I: pbuilder: network access will be disabled during build
I: Current time: Mon Jan  8 14:29:48 +14 2024
I: pbuilder-time-stamp: 1704673788
I: Building the build Environment
I: extracting base tarball [/var/cache/pbuilder/bookworm-reproducible-base.tgz]
I: copying local configuration
W: --override-config is not set; not updating apt.conf Read the manpage for details.
I: mounting /proc filesystem
I: mounting /sys filesystem
I: creating /{dev,run}/shm
I: mounting /dev/pts filesystem
I: redirecting /dev/ptmx to /dev/pts/ptmx
I: policy-rc.d already exists
I: using eatmydata during job
I: Copying source file
I: copying [golang-github-hashicorp-raft_1.1.2-1.dsc]
I: copying [./golang-github-hashicorp-raft_1.1.2.orig.tar.xz]
I: copying [./golang-github-hashicorp-raft_1.1.2-1.debian.tar.xz]
I: Extracting source
gpgv: Signature made Sun Jan 19 07:24:28 2020 
gpgv:                using RSA key 50BC7CF939D20C272A6B065652B6BBD953968D1B
gpgv: Can't check signature: No public key
dpkg-source: warning: cannot verify inline signature for ./golang-github-hashicorp-raft_1.1.2-1.dsc: no acceptable signature found
dpkg-source: info: extracting golang-github-hashicorp-raft in golang-github-hashicorp-raft-1.1.2
dpkg-source: info: unpacking golang-github-hashicorp-raft_1.1.2.orig.tar.xz
dpkg-source: info: unpacking golang-github-hashicorp-raft_1.1.2-1.debian.tar.xz
dpkg-source: info: using patch list from debian/patches/series
dpkg-source: info: applying t-skip--TestNetworkTransport_AppendEntriesPipeline_CloseStreams.patch
dpkg-source: info: applying t-skip--TestRaft_AutoSnapshot.patch
dpkg-source: info: applying t-skip--TestRaft_LeadershipTransferLeaderRejectsClientRequests.patch
dpkg-source: info: applying t-skip--TestRaft_RecoverCluster.patch
dpkg-source: info: applying t-skip--TestRaft_SnapshotRestore_PeerChange.patch
I: using fakeroot in build.
I: Installing the build-deps
I: user script /srv/workspace/pbuilder/15535/tmp/hooks/D01_modify_environment starting
debug: Running on ionos12-i386.
I: Changing host+domainname to test build reproducibility
I: Adding a custom variable just for the fun of it...
I: Changing /bin/sh to bash
'/bin/sh' -> '/bin/bash'
lrwxrwxrwx 1 root root 9 Jan  8 00:29 /bin/sh -> /bin/bash
I: Setting pbuilder2's login shell to /bin/bash
I: Setting pbuilder2's GECOS to second user,second room,second work-phone,second home-phone,second other
I: user script /srv/workspace/pbuilder/15535/tmp/hooks/D01_modify_environment finished
I: user script /srv/workspace/pbuilder/15535/tmp/hooks/D02_print_environment starting
I: set
  BASH=/bin/sh
  BASHOPTS=checkwinsize:cmdhist:complete_fullquote:extquote:force_fignore:globasciiranges:globskipdots:hostcomplete:interactive_comments:patsub_replacement:progcomp:promptvars:sourcepath
  BASH_ALIASES=()
  BASH_ARGC=()
  BASH_ARGV=()
  BASH_CMDS=()
  BASH_LINENO=([0]="12" [1]="0")
  BASH_LOADABLES_PATH=/usr/local/lib/bash:/usr/lib/bash:/opt/local/lib/bash:/usr/pkg/lib/bash:/opt/pkg/lib/bash:.
  BASH_SOURCE=([0]="/tmp/hooks/D02_print_environment" [1]="/tmp/hooks/D02_print_environment")
  BASH_VERSINFO=([0]="5" [1]="2" [2]="15" [3]="1" [4]="release" [5]="i686-pc-linux-gnu")
  BASH_VERSION='5.2.15(1)-release'
  BUILDDIR=/build/reproducible-path
  BUILDUSERGECOS='second user,second room,second work-phone,second home-phone,second other'
  BUILDUSERNAME=pbuilder2
  BUILD_ARCH=i386
  DEBIAN_FRONTEND=noninteractive
  DEB_BUILD_OPTIONS='buildinfo=+all reproducible=+all parallel=7 '
  DIRSTACK=()
  DISTRIBUTION=bookworm
  EUID=0
  FUNCNAME=([0]="Echo" [1]="main")
  GROUPS=()
  HOME=/root
  HOSTNAME=i-capture-the-hostname
  HOSTTYPE=i686
  HOST_ARCH=i386
  IFS=' 	
  '
  INVOCATION_ID=e8464e7aae43441c8e8a943dc6b44349
  LANG=C
  LANGUAGE=de_CH:de
  LC_ALL=C
  LD_LIBRARY_PATH=/usr/lib/libeatmydata
  LD_PRELOAD=libeatmydata.so
  MACHTYPE=i686-pc-linux-gnu
  MAIL=/var/mail/root
  OPTERR=1
  OPTIND=1
  OSTYPE=linux-gnu
  PATH=/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/i/capture/the/path
  PBCURRENTCOMMANDLINEOPERATION=build
  PBUILDER_OPERATION=build
  PBUILDER_PKGDATADIR=/usr/share/pbuilder
  PBUILDER_PKGLIBDIR=/usr/lib/pbuilder
  PBUILDER_SYSCONFDIR=/etc
  PIPESTATUS=([0]="0")
  POSIXLY_CORRECT=y
  PPID=15535
  PS4='+ '
  PWD=/
  SHELL=/bin/bash
  SHELLOPTS=braceexpand:errexit:hashall:interactive-comments:posix
  SHLVL=3
  SUDO_COMMAND='/usr/bin/timeout -k 24.1h 24h /usr/bin/ionice -c 3 /usr/bin/nice -n 11 /usr/bin/unshare --uts -- /usr/sbin/pbuilder --build --configfile /srv/reproducible-results/rbuild-debian/r-b-build.F1G7eTIx/pbuilderrc_L23S --distribution bookworm --hookdir /etc/pbuilder/rebuild-hooks --debbuildopts -b --basetgz /var/cache/pbuilder/bookworm-reproducible-base.tgz --buildresult /srv/reproducible-results/rbuild-debian/r-b-build.F1G7eTIx/b2 --logfile b2/build.log golang-github-hashicorp-raft_1.1.2-1.dsc'
  SUDO_GID=112
  SUDO_UID=107
  SUDO_USER=jenkins
  TERM=unknown
  TZ=/usr/share/zoneinfo/Etc/GMT-14
  UID=0
  USER=root
  _='I: set'
  http_proxy=http://78.137.99.97:3128
I: uname -a
  Linux i-capture-the-hostname 6.1.0-17-686-pae #1 SMP PREEMPT_DYNAMIC Debian 6.1.69-1 (2023-12-30) i686 GNU/Linux
I: ls -l /bin
  total 6036
  -rwxr-xr-x 1 root root 1408088 Apr 23  2023 bash
  -rwxr-xr-x 3 root root   38404 Sep 19  2022 bunzip2
  -rwxr-xr-x 3 root root   38404 Sep 19  2022 bzcat
  lrwxrwxrwx 1 root root       6 Sep 19  2022 bzcmp -> bzdiff
  -rwxr-xr-x 1 root root    2225 Sep 19  2022 bzdiff
  lrwxrwxrwx 1 root root       6 Sep 19  2022 bzegrep -> bzgrep
  -rwxr-xr-x 1 root root    4893 Nov 27  2021 bzexe
  lrwxrwxrwx 1 root root       6 Sep 19  2022 bzfgrep -> bzgrep
  -rwxr-xr-x 1 root root    3775 Sep 19  2022 bzgrep
  -rwxr-xr-x 3 root root   38404 Sep 19  2022 bzip2
  -rwxr-xr-x 1 root root   17892 Sep 19  2022 bzip2recover
  lrwxrwxrwx 1 root root       6 Sep 19  2022 bzless -> bzmore
  -rwxr-xr-x 1 root root    1297 Sep 19  2022 bzmore
  -rwxr-xr-x 1 root root   42920 Sep 20  2022 cat
  -rwxr-xr-x 1 root root   79816 Sep 20  2022 chgrp
  -rwxr-xr-x 1 root root   67496 Sep 20  2022 chmod
  -rwxr-xr-x 1 root root   79816 Sep 20  2022 chown
  -rwxr-xr-x 1 root root  162024 Sep 20  2022 cp
  -rwxr-xr-x 1 root root  136916 Jan  5  2023 dash
  -rwxr-xr-x 1 root root  137160 Sep 20  2022 date
  -rwxr-xr-x 1 root root  100364 Sep 20  2022 dd
  -rwxr-xr-x 1 root root  108940 Sep 20  2022 df
  -rwxr-xr-x 1 root root  162152 Sep 20  2022 dir
  -rwxr-xr-x 1 root root   87760 Mar 23  2023 dmesg
  lrwxrwxrwx 1 root root       8 Dec 19  2022 dnsdomainname -> hostname
  lrwxrwxrwx 1 root root       8 Dec 19  2022 domainname -> hostname
  -rwxr-xr-x 1 root root   38760 Sep 20  2022 echo
  -rwxr-xr-x 1 root root      41 Jan 24  2023 egrep
  -rwxr-xr-x 1 root root   34664 Sep 20  2022 false
  -rwxr-xr-x 1 root root      41 Jan 24  2023 fgrep
  -rwxr-xr-x 1 root root   84272 Mar 23  2023 findmnt
  -rwsr-xr-x 1 root root   30240 Mar 23  2023 fusermount
  -rwxr-xr-x 1 root root  218680 Jan 24  2023 grep
  -rwxr-xr-x 2 root root    2346 Apr 10  2022 gunzip
  -rwxr-xr-x 1 root root    6447 Apr 10  2022 gzexe
  -rwxr-xr-x 1 root root  100952 Apr 10  2022 gzip
  -rwxr-xr-x 1 root root   21916 Dec 19  2022 hostname
  -rwxr-xr-x 1 root root   75756 Sep 20  2022 ln
  -rwxr-xr-x 1 root root   55600 Mar 23  2023 login
  -rwxr-xr-x 1 root root  162152 Sep 20  2022 ls
  -rwxr-xr-x 1 root root  214568 Mar 23  2023 lsblk
  -rwxr-xr-x 1 root root   96328 Sep 20  2022 mkdir
  -rwxr-xr-x 1 root root   84008 Sep 20  2022 mknod
  -rwxr-xr-x 1 root root   38792 Sep 20  2022 mktemp
  -rwxr-xr-x 1 root root   63016 Mar 23  2023 more
  -rwsr-xr-x 1 root root   58912 Mar 23  2023 mount
  -rwxr-xr-x 1 root root   13856 Mar 23  2023 mountpoint
  -rwxr-xr-x 1 root root  157932 Sep 20  2022 mv
  lrwxrwxrwx 1 root root       8 Dec 19  2022 nisdomainname -> hostname
  lrwxrwxrwx 1 root root      14 Apr  3  2023 pidof -> /sbin/killall5
  -rwxr-xr-x 1 root root   38792 Sep 20  2022 pwd
  lrwxrwxrwx 1 root root       4 Apr 23  2023 rbash -> bash
  -rwxr-xr-x 1 root root   51080 Sep 20  2022 readlink
  -rwxr-xr-x 1 root root   75720 Sep 20  2022 rm
  -rwxr-xr-x 1 root root   51080 Sep 20  2022 rmdir
  -rwxr-xr-x 1 root root   22308 Jul 28 23:46 run-parts
  -rwxr-xr-x 1 root root  133224 Jan  5  2023 sed
  lrwxrwxrwx 1 root root       9 Jan  8 00:29 sh -> /bin/bash
  -rwxr-xr-x 1 root root   38760 Sep 20  2022 sleep
  -rwxr-xr-x 1 root root   87976 Sep 20  2022 stty
  -rwsr-xr-x 1 root root   83492 Mar 23  2023 su
  -rwxr-xr-x 1 root root   38792 Sep 20  2022 sync
  -rwxr-xr-x 1 root root  598456 Apr  6  2023 tar
  -rwxr-xr-x 1 root root   13860 Jul 28 23:46 tempfile
  -rwxr-xr-x 1 root root  120776 Sep 20  2022 touch
  -rwxr-xr-x 1 root root   34664 Sep 20  2022 true
  -rwxr-xr-x 1 root root   17892 Mar 23  2023 ulockmgr_server
  -rwsr-xr-x 1 root root   30236 Mar 23  2023 umount
  -rwxr-xr-x 1 root root   38760 Sep 20  2022 uname
  -rwxr-xr-x 2 root root    2346 Apr 10  2022 uncompress
  -rwxr-xr-x 1 root root  162152 Sep 20  2022 vdir
  -rwxr-xr-x 1 root root   71216 Mar 23  2023 wdctl
  lrwxrwxrwx 1 root root       8 Dec 19  2022 ypdomainname -> hostname
  -rwxr-xr-x 1 root root    1984 Apr 10  2022 zcat
  -rwxr-xr-x 1 root root    1678 Apr 10  2022 zcmp
  -rwxr-xr-x 1 root root    6460 Apr 10  2022 zdiff
  -rwxr-xr-x 1 root root      29 Apr 10  2022 zegrep
  -rwxr-xr-x 1 root root      29 Apr 10  2022 zfgrep
  -rwxr-xr-x 1 root root    2081 Apr 10  2022 zforce
  -rwxr-xr-x 1 root root    8103 Apr 10  2022 zgrep
  -rwxr-xr-x 1 root root    2206 Apr 10  2022 zless
  -rwxr-xr-x 1 root root    1842 Apr 10  2022 zmore
  -rwxr-xr-x 1 root root    4577 Apr 10  2022 znew
I: user script /srv/workspace/pbuilder/15535/tmp/hooks/D02_print_environment finished
 -> Attempting to satisfy build-dependencies
 -> Creating pbuilder-satisfydepends-dummy package
Package: pbuilder-satisfydepends-dummy
Version: 0.invalid.0
Architecture: i386
Maintainer: Debian Pbuilder Team <pbuilder-maint@lists.alioth.debian.org>
Description: Dummy package to satisfy dependencies with aptitude - created by pbuilder
 This package was created automatically by pbuilder to satisfy the
 build-dependencies of the package being currently built.
Depends: debhelper (>= 12~), dh-golang, golang-any, golang-github-armon-go-metrics-dev, golang-github-hashicorp-go-hclog-dev (>= 0.9.2~), golang-github-hashicorp-go-msgpack-dev (>= 0.5.5~)
dpkg-deb: building package 'pbuilder-satisfydepends-dummy' in '/tmp/satisfydepends-aptitude/pbuilder-satisfydepends-dummy.deb'.
Selecting previously unselected package pbuilder-satisfydepends-dummy.
(Reading database ... 18156 files and directories currently installed.)
Preparing to unpack .../pbuilder-satisfydepends-dummy.deb ...
Unpacking pbuilder-satisfydepends-dummy (0.invalid.0) ...
dpkg: pbuilder-satisfydepends-dummy: dependency problems, but configuring anyway as you requested:
 pbuilder-satisfydepends-dummy depends on debhelper (>= 12~); however:
  Package debhelper is not installed.
 pbuilder-satisfydepends-dummy depends on dh-golang; however:
  Package dh-golang is not installed.
 pbuilder-satisfydepends-dummy depends on golang-any; however:
  Package golang-any is not installed.
 pbuilder-satisfydepends-dummy depends on golang-github-armon-go-metrics-dev; however:
  Package golang-github-armon-go-metrics-dev is not installed.
 pbuilder-satisfydepends-dummy depends on golang-github-hashicorp-go-hclog-dev (>= 0.9.2~); however:
  Package golang-github-hashicorp-go-hclog-dev is not installed.
 pbuilder-satisfydepends-dummy depends on golang-github-hashicorp-go-msgpack-dev (>= 0.5.5~); however:
  Package golang-github-hashicorp-go-msgpack-dev is not installed.

Setting up pbuilder-satisfydepends-dummy (0.invalid.0) ...
Reading package lists...
Building dependency tree...
Reading state information...
Initializing package states...
Writing extended state information...
Building tag database...
pbuilder-satisfydepends-dummy is already installed at the requested version (0.invalid.0)
pbuilder-satisfydepends-dummy is already installed at the requested version (0.invalid.0)
The following NEW packages will be installed:
  autoconf{a} automake{a} autopoint{a} autotools-dev{a} bsdextrautils{a} debhelper{a} dh-autoreconf{a} dh-golang{a} dh-strip-nondeterminism{a} dwz{a} file{a} gettext{a} gettext-base{a} golang-1.19-go{a} golang-1.19-src{a} golang-any{a} golang-github-alecthomas-units-dev{a} golang-github-armon-go-metrics-dev{a} golang-github-aws-aws-sdk-go-dev{a} golang-github-beorn7-perks-dev{a} golang-github-cespare-xxhash-dev{a} golang-github-circonus-labs-circonus-gometrics-dev{a} golang-github-circonus-labs-circonusllhist-dev{a} golang-github-creack-pty-dev{a} golang-github-datadog-datadog-go-dev{a} golang-github-davecgh-go-spew-dev{a} golang-github-fatih-color-dev{a} golang-github-go-kit-log-dev{a} golang-github-go-logfmt-logfmt-dev{a} golang-github-golang-protobuf-1-3-dev{a} golang-github-hashicorp-go-cleanhttp-dev{a} golang-github-hashicorp-go-hclog-dev{a} golang-github-hashicorp-go-immutable-radix-dev{a} golang-github-hashicorp-go-msgpack-dev{a} golang-github-hashicorp-go-retryablehttp-dev{a} golang-github-hashicorp-go-uuid-dev{a} golang-github-hashicorp-golang-lru-dev{a} golang-github-jmespath-go-jmespath-dev{a} golang-github-jpillora-backoff-dev{a} golang-github-json-iterator-go-dev{a} golang-github-julienschmidt-httprouter-dev{a} golang-github-kr-pretty-dev{a} golang-github-kr-text-dev{a} golang-github-mattn-go-colorable-dev{a} golang-github-mattn-go-isatty-dev{a} golang-github-modern-go-concurrent-dev{a} golang-github-modern-go-reflect2-dev{a} golang-github-mwitkow-go-conntrack-dev{a} golang-github-pascaldekloe-goe-dev{a} golang-github-pkg-diff-dev{a} golang-github-pkg-errors-dev{a} golang-github-pmezard-go-difflib-dev{a} golang-github-prometheus-client-golang-dev{a} golang-github-prometheus-client-model-dev{a} golang-github-prometheus-common-dev{a} golang-github-prometheus-procfs-dev{a} golang-github-rogpeppe-go-internal-dev{a} golang-github-stretchr-objx-dev{a} golang-github-stretchr-testify-dev{a} golang-github-tv42-httpunix-dev{a} golang-github-vmihailenco-tagparser-dev{a} golang-go{a} golang-golang-x-net-dev{a} golang-golang-x-oauth2-dev{a} golang-golang-x-sync-dev{a} golang-golang-x-sys-dev{a} golang-golang-x-text-dev{a} golang-google-protobuf-dev{a} golang-gopkg-alecthomas-kingpin.v2-dev{a} golang-gopkg-check.v1-dev{a} golang-gopkg-mgo.v2-dev{a} golang-gopkg-tomb.v2-dev{a} golang-gopkg-vmihailenco-msgpack.v2-dev{a} golang-gopkg-yaml.v2-dev{a} golang-gopkg-yaml.v3-dev{a} golang-protobuf-extensions-dev{a} golang-src{a} groff-base{a} intltool-debian{a} libarchive-zip-perl{a} libdebhelper-perl{a} libelf1{a} libfile-stripnondeterminism-perl{a} libicu72{a} libmagic-mgc{a} libmagic1{a} libpipeline1{a} libsasl2-2{a} libsasl2-dev{a} libsasl2-modules-db{a} libsub-override-perl{a} libtool{a} libuchardet0{a} libxml2{a} m4{a} man-db{a} po-debconf{a} sensible-utils{a} 
The following packages are RECOMMENDED but will NOT be installed:
  curl libarchive-cpio-perl libltdl-dev libmail-sendmail-perl libsasl2-modules lynx pkg-config pkgconf wget 
0 packages upgraded, 98 newly installed, 0 to remove and 0 not upgraded.
Need to get 117 MB of archives. After unpacking 760 MB will be used.
Writing extended state information...
Get: 1 http://deb.debian.org/debian bookworm/main i386 sensible-utils all 0.0.17+nmu1 [19.0 kB]
Get: 2 http://deb.debian.org/debian bookworm/main i386 libmagic-mgc i386 1:5.44-3 [305 kB]
Get: 3 http://deb.debian.org/debian bookworm/main i386 libmagic1 i386 1:5.44-3 [114 kB]
Get: 4 http://deb.debian.org/debian bookworm/main i386 file i386 1:5.44-3 [42.5 kB]
Get: 5 http://deb.debian.org/debian bookworm/main i386 gettext-base i386 0.21-12 [162 kB]
Get: 6 http://deb.debian.org/debian bookworm/main i386 libuchardet0 i386 0.0.7-1 [67.9 kB]
Get: 7 http://deb.debian.org/debian bookworm/main i386 groff-base i386 1.22.4-10 [932 kB]
Get: 8 http://deb.debian.org/debian bookworm/main i386 bsdextrautils i386 2.38.1-5+b1 [90.3 kB]
Get: 9 http://deb.debian.org/debian bookworm/main i386 libpipeline1 i386 1.5.7-1 [40.0 kB]
Get: 10 http://deb.debian.org/debian bookworm/main i386 man-db i386 2.11.2-2 [1397 kB]
Get: 11 http://deb.debian.org/debian bookworm/main i386 m4 i386 1.4.19-3 [294 kB]
Get: 12 http://deb.debian.org/debian bookworm/main i386 autoconf all 2.71-3 [332 kB]
Get: 13 http://deb.debian.org/debian bookworm/main i386 autotools-dev all 20220109.1 [51.6 kB]
Get: 14 http://deb.debian.org/debian bookworm/main i386 automake all 1:1.16.5-1.3 [823 kB]
Get: 15 http://deb.debian.org/debian bookworm/main i386 autopoint all 0.21-12 [495 kB]
Get: 16 http://deb.debian.org/debian bookworm/main i386 libdebhelper-perl all 13.11.4 [81.2 kB]
Get: 17 http://deb.debian.org/debian bookworm/main i386 libtool all 2.4.7-5 [517 kB]
Get: 18 http://deb.debian.org/debian bookworm/main i386 dh-autoreconf all 20 [17.1 kB]
Get: 19 http://deb.debian.org/debian bookworm/main i386 libarchive-zip-perl all 1.68-1 [104 kB]
Get: 20 http://deb.debian.org/debian bookworm/main i386 libsub-override-perl all 0.09-4 [9304 B]
Get: 21 http://deb.debian.org/debian bookworm/main i386 libfile-stripnondeterminism-perl all 1.13.1-1 [19.4 kB]
Get: 22 http://deb.debian.org/debian bookworm/main i386 dh-strip-nondeterminism all 1.13.1-1 [8620 B]
Get: 23 http://deb.debian.org/debian bookworm/main i386 libelf1 i386 0.188-2.1 [179 kB]
Get: 24 http://deb.debian.org/debian bookworm/main i386 dwz i386 0.15-1 [118 kB]
Get: 25 http://deb.debian.org/debian bookworm/main i386 libicu72 i386 72.1-3 [9541 kB]
Get: 26 http://deb.debian.org/debian bookworm/main i386 libxml2 i386 2.9.14+dfsg-1.3~deb12u1 [720 kB]
Get: 27 http://deb.debian.org/debian bookworm/main i386 gettext i386 0.21-12 [1311 kB]
Get: 28 http://deb.debian.org/debian bookworm/main i386 intltool-debian all 0.35.0+20060710.6 [22.9 kB]
Get: 29 http://deb.debian.org/debian bookworm/main i386 po-debconf all 1.0.21+nmu1 [248 kB]
Get: 30 http://deb.debian.org/debian bookworm/main i386 debhelper all 13.11.4 [942 kB]
Get: 31 http://deb.debian.org/debian bookworm/main i386 dh-golang all 1.59 [27.8 kB]
Get: 32 http://deb.debian.org/debian bookworm/main i386 golang-1.19-src all 1.19.8-2 [18.3 MB]
Get: 33 http://deb.debian.org/debian bookworm/main i386 golang-1.19-go i386 1.19.8-2 [61.2 MB]
Get: 34 http://deb.debian.org/debian bookworm/main i386 golang-src all 2:1.19~1 [5756 B]
Get: 35 http://deb.debian.org/debian bookworm/main i386 golang-go i386 2:1.19~1 [45.0 kB]
Get: 36 http://deb.debian.org/debian bookworm/main i386 golang-any i386 2:1.19~1 [5872 B]
Get: 37 http://deb.debian.org/debian bookworm/main i386 golang-github-alecthomas-units-dev all 0.0~git20211218.b94a6e3-1 [8776 B]
Get: 38 http://deb.debian.org/debian bookworm/main i386 golang-github-circonus-labs-circonusllhist-dev all 0.0~git20191022.ec08cde-1 [11.1 kB]
Get: 39 http://deb.debian.org/debian bookworm/main i386 golang-github-hashicorp-go-cleanhttp-dev all 0.5.2-1 [10.8 kB]
Get: 40 http://deb.debian.org/debian bookworm/main i386 golang-golang-x-sys-dev all 0.3.0-1 [393 kB]
Get: 41 http://deb.debian.org/debian bookworm/main i386 golang-github-mattn-go-isatty-dev all 0.0.17-1 [6016 B]
Get: 42 http://deb.debian.org/debian bookworm/main i386 golang-github-mattn-go-colorable-dev all 0.1.13-1 [10.3 kB]
Get: 43 http://deb.debian.org/debian bookworm/main i386 golang-github-fatih-color-dev all 1.7.0-1 [11.4 kB]
Get: 44 http://deb.debian.org/debian bookworm/main i386 golang-github-davecgh-go-spew-dev all 1.1.1-3 [30.1 kB]
Get: 45 http://deb.debian.org/debian bookworm/main i386 golang-github-pmezard-go-difflib-dev all 1.0.0-3 [12.3 kB]
Get: 46 http://deb.debian.org/debian bookworm/main i386 golang-github-stretchr-objx-dev all 0.5.0-1 [25.5 kB]
Get: 47 http://deb.debian.org/debian bookworm/main i386 golang-gopkg-yaml.v3-dev all 3.0.1-3 [74.3 kB]
Get: 48 http://deb.debian.org/debian bookworm/main i386 golang-github-stretchr-testify-dev all 1.8.1-1 [67.9 kB]
Get: 49 http://deb.debian.org/debian bookworm/main i386 golang-github-hashicorp-go-hclog-dev all 0.11.0-1 [17.9 kB]
Get: 50 http://deb.debian.org/debian bookworm/main i386 golang-github-hashicorp-go-retryablehttp-dev all 0.7.1-1 [21.3 kB]
Get: 51 http://deb.debian.org/debian bookworm/main i386 golang-github-pkg-errors-dev all 0.9.1-2 [13.2 kB]
Get: 52 http://deb.debian.org/debian bookworm/main i386 golang-github-tv42-httpunix-dev all 0.0~git20150427.b75d861-3 [3956 B]
Get: 53 http://deb.debian.org/debian bookworm/main i386 golang-github-circonus-labs-circonus-gometrics-dev all 2.3.1-4 [65.7 kB]
Get: 54 http://deb.debian.org/debian bookworm/main i386 golang-github-datadog-datadog-go-dev all 2.1.0-3 [14.9 kB]
Get: 55 http://deb.debian.org/debian bookworm/main i386 golang-github-hashicorp-go-uuid-dev all 1.0.2-2 [8480 B]
Get: 56 http://deb.debian.org/debian bookworm/main i386 golang-github-hashicorp-golang-lru-dev all 0.5.4-2 [14.7 kB]
Get: 57 http://deb.debian.org/debian bookworm/main i386 golang-github-hashicorp-go-immutable-radix-dev all 1.3.1-3 [26.2 kB]
Get: 58 http://deb.debian.org/debian bookworm/main i386 golang-github-pascaldekloe-goe-dev all 0.1.0-4 [10.9 kB]
Get: 59 http://deb.debian.org/debian bookworm/main i386 golang-github-beorn7-perks-dev all 1.0.1-1 [12.1 kB]
Get: 60 http://deb.debian.org/debian bookworm/main i386 golang-github-cespare-xxhash-dev all 2.1.1-2 [8888 B]
Get: 61 http://deb.debian.org/debian bookworm/main i386 golang-github-golang-protobuf-1-3-dev all 1.3.5-4 [234 kB]
Get: 62 http://deb.debian.org/debian bookworm/main i386 golang-github-modern-go-concurrent-dev all 1.0.3-1.1 [4668 B]
Get: 63 http://deb.debian.org/debian bookworm/main i386 golang-github-modern-go-reflect2-dev all 1.0.2-2 [10.9 kB]
Get: 64 http://deb.debian.org/debian bookworm/main i386 golang-github-json-iterator-go-dev all 1.1.12-1 [62.2 kB]
Get: 65 http://deb.debian.org/debian bookworm/main i386 golang-github-prometheus-client-model-dev all 0.3.0-3 [10.5 kB]
Get: 66 http://deb.debian.org/debian bookworm/main i386 golang-gopkg-yaml.v2-dev all 2.4.0-4 [61.5 kB]
Get: 67 http://deb.debian.org/debian bookworm/main i386 golang-github-jmespath-go-jmespath-dev all 0.4.0-2 [91.6 kB]
Get: 68 http://deb.debian.org/debian bookworm/main i386 golang-golang-x-text-dev all 0.7.0-1 [3863 kB]
Get: 69 http://deb.debian.org/debian bookworm/main i386 golang-golang-x-net-dev all 1:0.7.0+dfsg-1 [731 kB]
Get: 70 http://deb.debian.org/debian bookworm/main i386 golang-github-aws-aws-sdk-go-dev all 1.44.133-1 [10.5 MB]
Get: 71 http://deb.debian.org/debian bookworm/main i386 golang-github-go-logfmt-logfmt-dev all 0.5.0-2 [13.4 kB]
Get: 72 http://deb.debian.org/debian bookworm/main i386 golang-github-go-kit-log-dev all 0.2.1-1 [26.9 kB]
Get: 73 http://deb.debian.org/debian bookworm/main i386 golang-github-julienschmidt-httprouter-dev all 1.3.0-2 [19.3 kB]
Get: 74 http://deb.debian.org/debian bookworm/main i386 golang-github-jpillora-backoff-dev all 1.0.0-1.1 [3712 B]
Get: 75 http://deb.debian.org/debian bookworm/main i386 golang-github-mwitkow-go-conntrack-dev all 0.0~git20190716.2f06839-3 [14.6 kB]
Get: 76 http://deb.debian.org/debian bookworm/main i386 golang-golang-x-oauth2-dev all 0.3.0-1 [35.0 kB]
Get: 77 http://deb.debian.org/debian bookworm/main i386 golang-google-protobuf-dev all 1.28.1-3 [542 kB]
Get: 78 http://deb.debian.org/debian bookworm/main i386 golang-gopkg-alecthomas-kingpin.v2-dev all 2.2.6-4 [42.8 kB]
Get: 79 http://deb.debian.org/debian bookworm/main i386 golang-protobuf-extensions-dev all 1.0.1-3 [29.5 kB]
Get: 80 http://deb.debian.org/debian bookworm/main i386 golang-github-prometheus-common-dev all 0.39.0-2 [110 kB]
Get: 81 http://deb.debian.org/debian bookworm/main i386 golang-golang-x-sync-dev all 0.1.0-1 [19.7 kB]
Get: 82 http://deb.debian.org/debian bookworm/main i386 golang-github-prometheus-procfs-dev all 0.8.0-3 [148 kB]
Get: 83 http://deb.debian.org/debian bookworm/main i386 golang-github-prometheus-client-golang-dev all 1.14.0-3 [174 kB]
Get: 84 http://deb.debian.org/debian bookworm/main i386 golang-github-armon-go-metrics-dev all 0.3.4-1 [30.5 kB]
Get: 85 http://deb.debian.org/debian bookworm/main i386 golang-github-creack-pty-dev all 1.1.18-1 [9796 B]
Get: 86 http://deb.debian.org/debian bookworm/main i386 golang-github-vmihailenco-tagparser-dev all 0.1.1-4 [4684 B]
Get: 87 http://deb.debian.org/debian bookworm/main i386 golang-gopkg-vmihailenco-msgpack.v2-dev all 4.3.1-2 [27.5 kB]
Get: 88 http://deb.debian.org/debian bookworm/main i386 golang-gopkg-tomb.v2-dev all 0.0~git20161208.d5d1b58-3 [6840 B]
Get: 89 http://deb.debian.org/debian bookworm/main i386 libsasl2-modules-db i386 2.1.28+dfsg-10 [21.4 kB]
Get: 90 http://deb.debian.org/debian bookworm/main i386 libsasl2-2 i386 2.1.28+dfsg-10 [62.7 kB]
Get: 91 http://deb.debian.org/debian bookworm/main i386 libsasl2-dev i386 2.1.28+dfsg-10 [248 kB]
Get: 92 http://deb.debian.org/debian bookworm/main i386 golang-github-kr-text-dev all 0.2.0-2 [10.2 kB]
Get: 93 http://deb.debian.org/debian bookworm/main i386 golang-github-pkg-diff-dev all 0.0~git20210226.20ebb0f-1 [93.6 kB]
Get: 94 http://deb.debian.org/debian bookworm/main i386 golang-github-rogpeppe-go-internal-dev all 1.9.0-1 [117 kB]
Get: 95 http://deb.debian.org/debian bookworm/main i386 golang-github-kr-pretty-dev all 0.3.0-1 [14.9 kB]
Get: 96 http://deb.debian.org/debian bookworm/main i386 golang-gopkg-check.v1-dev all 0.0+git20200902.038fdea-1 [32.1 kB]
Get: 97 http://deb.debian.org/debian bookworm/main i386 golang-gopkg-mgo.v2-dev all 2016.08.01-7 [316 kB]
Get: 98 http://deb.debian.org/debian bookworm/main i386 golang-github-hashicorp-go-msgpack-dev all 0.5.5-1 [43.3 kB]
Fetched 117 MB in 5s (23.0 MB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package sensible-utils.
(Reading database ... 
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%
(Reading database ... 75%
(Reading database ... 80%
(Reading database ... 85%
(Reading database ... 90%
(Reading database ... 95%
(Reading database ... 100%
(Reading database ... 18156 files and directories currently installed.)
Preparing to unpack .../00-sensible-utils_0.0.17+nmu1_all.deb ...
Unpacking sensible-utils (0.0.17+nmu1) ...
Selecting previously unselected package libmagic-mgc.
Preparing to unpack .../01-libmagic-mgc_1%3a5.44-3_i386.deb ...
Unpacking libmagic-mgc (1:5.44-3) ...
Selecting previously unselected package libmagic1:i386.
Preparing to unpack .../02-libmagic1_1%3a5.44-3_i386.deb ...
Unpacking libmagic1:i386 (1:5.44-3) ...
Selecting previously unselected package file.
Preparing to unpack .../03-file_1%3a5.44-3_i386.deb ...
Unpacking file (1:5.44-3) ...
Selecting previously unselected package gettext-base.
Preparing to unpack .../04-gettext-base_0.21-12_i386.deb ...
Unpacking gettext-base (0.21-12) ...
Selecting previously unselected package libuchardet0:i386.
Preparing to unpack .../05-libuchardet0_0.0.7-1_i386.deb ...
Unpacking libuchardet0:i386 (0.0.7-1) ...
Selecting previously unselected package groff-base.
Preparing to unpack .../06-groff-base_1.22.4-10_i386.deb ...
Unpacking groff-base (1.22.4-10) ...
Selecting previously unselected package bsdextrautils.
Preparing to unpack .../07-bsdextrautils_2.38.1-5+b1_i386.deb ...
Unpacking bsdextrautils (2.38.1-5+b1) ...
Selecting previously unselected package libpipeline1:i386.
Preparing to unpack .../08-libpipeline1_1.5.7-1_i386.deb ...
Unpacking libpipeline1:i386 (1.5.7-1) ...
Selecting previously unselected package man-db.
Preparing to unpack .../09-man-db_2.11.2-2_i386.deb ...
Unpacking man-db (2.11.2-2) ...
Selecting previously unselected package m4.
Preparing to unpack .../10-m4_1.4.19-3_i386.deb ...
Unpacking m4 (1.4.19-3) ...
Selecting previously unselected package autoconf.
Preparing to unpack .../11-autoconf_2.71-3_all.deb ...
Unpacking autoconf (2.71-3) ...
Selecting previously unselected package autotools-dev.
Preparing to unpack .../12-autotools-dev_20220109.1_all.deb ...
Unpacking autotools-dev (20220109.1) ...
Selecting previously unselected package automake.
Preparing to unpack .../13-automake_1%3a1.16.5-1.3_all.deb ...
Unpacking automake (1:1.16.5-1.3) ...
Selecting previously unselected package autopoint.
Preparing to unpack .../14-autopoint_0.21-12_all.deb ...
Unpacking autopoint (0.21-12) ...
Selecting previously unselected package libdebhelper-perl.
Preparing to unpack .../15-libdebhelper-perl_13.11.4_all.deb ...
Unpacking libdebhelper-perl (13.11.4) ...
Selecting previously unselected package libtool.
Preparing to unpack .../16-libtool_2.4.7-5_all.deb ...
Unpacking libtool (2.4.7-5) ...
Selecting previously unselected package dh-autoreconf.
Preparing to unpack .../17-dh-autoreconf_20_all.deb ...
Unpacking dh-autoreconf (20) ...
Selecting previously unselected package libarchive-zip-perl.
Preparing to unpack .../18-libarchive-zip-perl_1.68-1_all.deb ...
Unpacking libarchive-zip-perl (1.68-1) ...
Selecting previously unselected package libsub-override-perl.
Preparing to unpack .../19-libsub-override-perl_0.09-4_all.deb ...
Unpacking libsub-override-perl (0.09-4) ...
Selecting previously unselected package libfile-stripnondeterminism-perl.
Preparing to unpack .../20-libfile-stripnondeterminism-perl_1.13.1-1_all.deb ...
Unpacking libfile-stripnondeterminism-perl (1.13.1-1) ...
Selecting previously unselected package dh-strip-nondeterminism.
Preparing to unpack .../21-dh-strip-nondeterminism_1.13.1-1_all.deb ...
Unpacking dh-strip-nondeterminism (1.13.1-1) ...
Selecting previously unselected package libelf1:i386.
Preparing to unpack .../22-libelf1_0.188-2.1_i386.deb ...
Unpacking libelf1:i386 (0.188-2.1) ...
Selecting previously unselected package dwz.
Preparing to unpack .../23-dwz_0.15-1_i386.deb ...
Unpacking dwz (0.15-1) ...
Selecting previously unselected package libicu72:i386.
Preparing to unpack .../24-libicu72_72.1-3_i386.deb ...
Unpacking libicu72:i386 (72.1-3) ...
Selecting previously unselected package libxml2:i386.
Preparing to unpack .../25-libxml2_2.9.14+dfsg-1.3~deb12u1_i386.deb ...
Unpacking libxml2:i386 (2.9.14+dfsg-1.3~deb12u1) ...
Selecting previously unselected package gettext.
Preparing to unpack .../26-gettext_0.21-12_i386.deb ...
Unpacking gettext (0.21-12) ...
Selecting previously unselected package intltool-debian.
Preparing to unpack .../27-intltool-debian_0.35.0+20060710.6_all.deb ...
Unpacking intltool-debian (0.35.0+20060710.6) ...
Selecting previously unselected package po-debconf.
Preparing to unpack .../28-po-debconf_1.0.21+nmu1_all.deb ...
Unpacking po-debconf (1.0.21+nmu1) ...
Selecting previously unselected package debhelper.
Preparing to unpack .../29-debhelper_13.11.4_all.deb ...
Unpacking debhelper (13.11.4) ...
Selecting previously unselected package dh-golang.
Preparing to unpack .../30-dh-golang_1.59_all.deb ...
Unpacking dh-golang (1.59) ...
Selecting previously unselected package golang-1.19-src.
Preparing to unpack .../31-golang-1.19-src_1.19.8-2_all.deb ...
Unpacking golang-1.19-src (1.19.8-2) ...
Selecting previously unselected package golang-1.19-go.
Preparing to unpack .../32-golang-1.19-go_1.19.8-2_i386.deb ...
Unpacking golang-1.19-go (1.19.8-2) ...
Selecting previously unselected package golang-src.
Preparing to unpack .../33-golang-src_2%3a1.19~1_all.deb ...
Unpacking golang-src (2:1.19~1) ...
Selecting previously unselected package golang-go:i386.
Preparing to unpack .../34-golang-go_2%3a1.19~1_i386.deb ...
Unpacking golang-go:i386 (2:1.19~1) ...
Selecting previously unselected package golang-any:i386.
Preparing to unpack .../35-golang-any_2%3a1.19~1_i386.deb ...
Unpacking golang-any:i386 (2:1.19~1) ...
Selecting previously unselected package golang-github-alecthomas-units-dev.
Preparing to unpack .../36-golang-github-alecthomas-units-dev_0.0~git20211218.b94a6e3-1_all.deb ...
Unpacking golang-github-alecthomas-units-dev (0.0~git20211218.b94a6e3-1) ...
Selecting previously unselected package golang-github-circonus-labs-circonusllhist-dev.
Preparing to unpack .../37-golang-github-circonus-labs-circonusllhist-dev_0.0~git20191022.ec08cde-1_all.deb ...
Unpacking golang-github-circonus-labs-circonusllhist-dev (0.0~git20191022.ec08cde-1) ...
Selecting previously unselected package golang-github-hashicorp-go-cleanhttp-dev.
Preparing to unpack .../38-golang-github-hashicorp-go-cleanhttp-dev_0.5.2-1_all.deb ...
Unpacking golang-github-hashicorp-go-cleanhttp-dev (0.5.2-1) ...
Selecting previously unselected package golang-golang-x-sys-dev.
Preparing to unpack .../39-golang-golang-x-sys-dev_0.3.0-1_all.deb ...
Unpacking golang-golang-x-sys-dev (0.3.0-1) ...
Selecting previously unselected package golang-github-mattn-go-isatty-dev.
Preparing to unpack .../40-golang-github-mattn-go-isatty-dev_0.0.17-1_all.deb ...
Unpacking golang-github-mattn-go-isatty-dev (0.0.17-1) ...
Selecting previously unselected package golang-github-mattn-go-colorable-dev.
Preparing to unpack .../41-golang-github-mattn-go-colorable-dev_0.1.13-1_all.deb ...
Unpacking golang-github-mattn-go-colorable-dev (0.1.13-1) ...
Selecting previously unselected package golang-github-fatih-color-dev.
Preparing to unpack .../42-golang-github-fatih-color-dev_1.7.0-1_all.deb ...
Unpacking golang-github-fatih-color-dev (1.7.0-1) ...
Selecting previously unselected package golang-github-davecgh-go-spew-dev.
Preparing to unpack .../43-golang-github-davecgh-go-spew-dev_1.1.1-3_all.deb ...
Unpacking golang-github-davecgh-go-spew-dev (1.1.1-3) ...
Selecting previously unselected package golang-github-pmezard-go-difflib-dev.
Preparing to unpack .../44-golang-github-pmezard-go-difflib-dev_1.0.0-3_all.deb ...
Unpacking golang-github-pmezard-go-difflib-dev (1.0.0-3) ...
Selecting previously unselected package golang-github-stretchr-objx-dev.
Preparing to unpack .../45-golang-github-stretchr-objx-dev_0.5.0-1_all.deb ...
Unpacking golang-github-stretchr-objx-dev (0.5.0-1) ...
Selecting previously unselected package golang-gopkg-yaml.v3-dev.
Preparing to unpack .../46-golang-gopkg-yaml.v3-dev_3.0.1-3_all.deb ...
Unpacking golang-gopkg-yaml.v3-dev (3.0.1-3) ...
Selecting previously unselected package golang-github-stretchr-testify-dev.
Preparing to unpack .../47-golang-github-stretchr-testify-dev_1.8.1-1_all.deb ...
Unpacking golang-github-stretchr-testify-dev (1.8.1-1) ...
Selecting previously unselected package golang-github-hashicorp-go-hclog-dev.
Preparing to unpack .../48-golang-github-hashicorp-go-hclog-dev_0.11.0-1_all.deb ...
Unpacking golang-github-hashicorp-go-hclog-dev (0.11.0-1) ...
Selecting previously unselected package golang-github-hashicorp-go-retryablehttp-dev.
Preparing to unpack .../49-golang-github-hashicorp-go-retryablehttp-dev_0.7.1-1_all.deb ...
Unpacking golang-github-hashicorp-go-retryablehttp-dev (0.7.1-1) ...
Selecting previously unselected package golang-github-pkg-errors-dev.
Preparing to unpack .../50-golang-github-pkg-errors-dev_0.9.1-2_all.deb ...
Unpacking golang-github-pkg-errors-dev (0.9.1-2) ...
Selecting previously unselected package golang-github-tv42-httpunix-dev.
Preparing to unpack .../51-golang-github-tv42-httpunix-dev_0.0~git20150427.b75d861-3_all.deb ...
Unpacking golang-github-tv42-httpunix-dev (0.0~git20150427.b75d861-3) ...
Selecting previously unselected package golang-github-circonus-labs-circonus-gometrics-dev.
Preparing to unpack .../52-golang-github-circonus-labs-circonus-gometrics-dev_2.3.1-4_all.deb ...
Unpacking golang-github-circonus-labs-circonus-gometrics-dev (2.3.1-4) ...
Selecting previously unselected package golang-github-datadog-datadog-go-dev.
Preparing to unpack .../53-golang-github-datadog-datadog-go-dev_2.1.0-3_all.deb ...
Unpacking golang-github-datadog-datadog-go-dev (2.1.0-3) ...
Selecting previously unselected package golang-github-hashicorp-go-uuid-dev.
Preparing to unpack .../54-golang-github-hashicorp-go-uuid-dev_1.0.2-2_all.deb ...
Unpacking golang-github-hashicorp-go-uuid-dev (1.0.2-2) ...
Selecting previously unselected package golang-github-hashicorp-golang-lru-dev.
Preparing to unpack .../55-golang-github-hashicorp-golang-lru-dev_0.5.4-2_all.deb ...
Unpacking golang-github-hashicorp-golang-lru-dev (0.5.4-2) ...
Selecting previously unselected package golang-github-hashicorp-go-immutable-radix-dev.
Preparing to unpack .../56-golang-github-hashicorp-go-immutable-radix-dev_1.3.1-3_all.deb ...
Unpacking golang-github-hashicorp-go-immutable-radix-dev (1.3.1-3) ...
Selecting previously unselected package golang-github-pascaldekloe-goe-dev.
Preparing to unpack .../57-golang-github-pascaldekloe-goe-dev_0.1.0-4_all.deb ...
Unpacking golang-github-pascaldekloe-goe-dev (0.1.0-4) ...
Selecting previously unselected package golang-github-beorn7-perks-dev.
Preparing to unpack .../58-golang-github-beorn7-perks-dev_1.0.1-1_all.deb ...
Unpacking golang-github-beorn7-perks-dev (1.0.1-1) ...
Selecting previously unselected package golang-github-cespare-xxhash-dev.
Preparing to unpack .../59-golang-github-cespare-xxhash-dev_2.1.1-2_all.deb ...
Unpacking golang-github-cespare-xxhash-dev (2.1.1-2) ...
Selecting previously unselected package golang-github-golang-protobuf-1-3-dev.
Preparing to unpack .../60-golang-github-golang-protobuf-1-3-dev_1.3.5-4_all.deb ...
Unpacking golang-github-golang-protobuf-1-3-dev (1.3.5-4) ...
Selecting previously unselected package golang-github-modern-go-concurrent-dev.
Preparing to unpack .../61-golang-github-modern-go-concurrent-dev_1.0.3-1.1_all.deb ...
Unpacking golang-github-modern-go-concurrent-dev (1.0.3-1.1) ...
Selecting previously unselected package golang-github-modern-go-reflect2-dev.
Preparing to unpack .../62-golang-github-modern-go-reflect2-dev_1.0.2-2_all.deb ...
Unpacking golang-github-modern-go-reflect2-dev (1.0.2-2) ...
Selecting previously unselected package golang-github-json-iterator-go-dev.
Preparing to unpack .../63-golang-github-json-iterator-go-dev_1.1.12-1_all.deb ...
Unpacking golang-github-json-iterator-go-dev (1.1.12-1) ...
Selecting previously unselected package golang-github-prometheus-client-model-dev.
Preparing to unpack .../64-golang-github-prometheus-client-model-dev_0.3.0-3_all.deb ...
Unpacking golang-github-prometheus-client-model-dev (0.3.0-3) ...
Selecting previously unselected package golang-gopkg-yaml.v2-dev.
Preparing to unpack .../65-golang-gopkg-yaml.v2-dev_2.4.0-4_all.deb ...
Unpacking golang-gopkg-yaml.v2-dev (2.4.0-4) ...
Selecting previously unselected package golang-github-jmespath-go-jmespath-dev.
Preparing to unpack .../66-golang-github-jmespath-go-jmespath-dev_0.4.0-2_all.deb ...
Unpacking golang-github-jmespath-go-jmespath-dev (0.4.0-2) ...
Selecting previously unselected package golang-golang-x-text-dev.
Preparing to unpack .../67-golang-golang-x-text-dev_0.7.0-1_all.deb ...
Unpacking golang-golang-x-text-dev (0.7.0-1) ...
Selecting previously unselected package golang-golang-x-net-dev.
Preparing to unpack .../68-golang-golang-x-net-dev_1%3a0.7.0+dfsg-1_all.deb ...
Unpacking golang-golang-x-net-dev (1:0.7.0+dfsg-1) ...
Selecting previously unselected package golang-github-aws-aws-sdk-go-dev.
Preparing to unpack .../69-golang-github-aws-aws-sdk-go-dev_1.44.133-1_all.deb ...
Unpacking golang-github-aws-aws-sdk-go-dev (1.44.133-1) ...
Selecting previously unselected package golang-github-go-logfmt-logfmt-dev.
Preparing to unpack .../70-golang-github-go-logfmt-logfmt-dev_0.5.0-2_all.deb ...
Unpacking golang-github-go-logfmt-logfmt-dev (0.5.0-2) ...
Selecting previously unselected package golang-github-go-kit-log-dev.
Preparing to unpack .../71-golang-github-go-kit-log-dev_0.2.1-1_all.deb ...
Unpacking golang-github-go-kit-log-dev (0.2.1-1) ...
Selecting previously unselected package golang-github-julienschmidt-httprouter-dev.
Preparing to unpack .../72-golang-github-julienschmidt-httprouter-dev_1.3.0-2_all.deb ...
Unpacking golang-github-julienschmidt-httprouter-dev (1.3.0-2) ...
Selecting previously unselected package golang-github-jpillora-backoff-dev.
Preparing to unpack .../73-golang-github-jpillora-backoff-dev_1.0.0-1.1_all.deb ...
Unpacking golang-github-jpillora-backoff-dev (1.0.0-1.1) ...
Selecting previously unselected package golang-github-mwitkow-go-conntrack-dev.
Preparing to unpack .../74-golang-github-mwitkow-go-conntrack-dev_0.0~git20190716.2f06839-3_all.deb ...
Unpacking golang-github-mwitkow-go-conntrack-dev (0.0~git20190716.2f06839-3) ...
Selecting previously unselected package golang-golang-x-oauth2-dev.
Preparing to unpack .../75-golang-golang-x-oauth2-dev_0.3.0-1_all.deb ...
Unpacking golang-golang-x-oauth2-dev (0.3.0-1) ...
Selecting previously unselected package golang-google-protobuf-dev.
Preparing to unpack .../76-golang-google-protobuf-dev_1.28.1-3_all.deb ...
Unpacking golang-google-protobuf-dev (1.28.1-3) ...
Selecting previously unselected package golang-gopkg-alecthomas-kingpin.v2-dev.
Preparing to unpack .../77-golang-gopkg-alecthomas-kingpin.v2-dev_2.2.6-4_all.deb ...
Unpacking golang-gopkg-alecthomas-kingpin.v2-dev (2.2.6-4) ...
Selecting previously unselected package golang-protobuf-extensions-dev.
Preparing to unpack .../78-golang-protobuf-extensions-dev_1.0.1-3_all.deb ...
Unpacking golang-protobuf-extensions-dev (1.0.1-3) ...
Selecting previously unselected package golang-github-prometheus-common-dev.
Preparing to unpack .../79-golang-github-prometheus-common-dev_0.39.0-2_all.deb ...
Unpacking golang-github-prometheus-common-dev (0.39.0-2) ...
Selecting previously unselected package golang-golang-x-sync-dev.
Preparing to unpack .../80-golang-golang-x-sync-dev_0.1.0-1_all.deb ...
Unpacking golang-golang-x-sync-dev (0.1.0-1) ...
Selecting previously unselected package golang-github-prometheus-procfs-dev.
Preparing to unpack .../81-golang-github-prometheus-procfs-dev_0.8.0-3_all.deb ...
Unpacking golang-github-prometheus-procfs-dev (0.8.0-3) ...
Selecting previously unselected package golang-github-prometheus-client-golang-dev.
Preparing to unpack .../82-golang-github-prometheus-client-golang-dev_1.14.0-3_all.deb ...
Unpacking golang-github-prometheus-client-golang-dev (1.14.0-3) ...
Selecting previously unselected package golang-github-armon-go-metrics-dev.
Preparing to unpack .../83-golang-github-armon-go-metrics-dev_0.3.4-1_all.deb ...
Unpacking golang-github-armon-go-metrics-dev (0.3.4-1) ...
Selecting previously unselected package golang-github-creack-pty-dev.
Preparing to unpack .../84-golang-github-creack-pty-dev_1.1.18-1_all.deb ...
Unpacking golang-github-creack-pty-dev (1.1.18-1) ...
Selecting previously unselected package golang-github-vmihailenco-tagparser-dev.
Preparing to unpack .../85-golang-github-vmihailenco-tagparser-dev_0.1.1-4_all.deb ...
Unpacking golang-github-vmihailenco-tagparser-dev (0.1.1-4) ...
Selecting previously unselected package golang-gopkg-vmihailenco-msgpack.v2-dev.
Preparing to unpack .../86-golang-gopkg-vmihailenco-msgpack.v2-dev_4.3.1-2_all.deb ...
Unpacking golang-gopkg-vmihailenco-msgpack.v2-dev (4.3.1-2) ...
Selecting previously unselected package golang-gopkg-tomb.v2-dev.
Preparing to unpack .../87-golang-gopkg-tomb.v2-dev_0.0~git20161208.d5d1b58-3_all.deb ...
Unpacking golang-gopkg-tomb.v2-dev (0.0~git20161208.d5d1b58-3) ...
Selecting previously unselected package libsasl2-modules-db:i386.
Preparing to unpack .../88-libsasl2-modules-db_2.1.28+dfsg-10_i386.deb ...
Unpacking libsasl2-modules-db:i386 (2.1.28+dfsg-10) ...
Selecting previously unselected package libsasl2-2:i386.
Preparing to unpack .../89-libsasl2-2_2.1.28+dfsg-10_i386.deb ...
Unpacking libsasl2-2:i386 (2.1.28+dfsg-10) ...
Selecting previously unselected package libsasl2-dev.
Preparing to unpack .../90-libsasl2-dev_2.1.28+dfsg-10_i386.deb ...
Unpacking libsasl2-dev (2.1.28+dfsg-10) ...
Selecting previously unselected package golang-github-kr-text-dev.
Preparing to unpack .../91-golang-github-kr-text-dev_0.2.0-2_all.deb ...
Unpacking golang-github-kr-text-dev (0.2.0-2) ...
Selecting previously unselected package golang-github-pkg-diff-dev.
Preparing to unpack .../92-golang-github-pkg-diff-dev_0.0~git20210226.20ebb0f-1_all.deb ...
Unpacking golang-github-pkg-diff-dev (0.0~git20210226.20ebb0f-1) ...
Selecting previously unselected package golang-github-rogpeppe-go-internal-dev.
Preparing to unpack .../93-golang-github-rogpeppe-go-internal-dev_1.9.0-1_all.deb ...
Unpacking golang-github-rogpeppe-go-internal-dev (1.9.0-1) ...
Selecting previously unselected package golang-github-kr-pretty-dev.
Preparing to unpack .../94-golang-github-kr-pretty-dev_0.3.0-1_all.deb ...
Unpacking golang-github-kr-pretty-dev (0.3.0-1) ...
Selecting previously unselected package golang-gopkg-check.v1-dev.
Preparing to unpack .../95-golang-gopkg-check.v1-dev_0.0+git20200902.038fdea-1_all.deb ...
Unpacking golang-gopkg-check.v1-dev (0.0+git20200902.038fdea-1) ...
Selecting previously unselected package golang-gopkg-mgo.v2-dev.
Preparing to unpack .../96-golang-gopkg-mgo.v2-dev_2016.08.01-7_all.deb ...
Unpacking golang-gopkg-mgo.v2-dev (2016.08.01-7) ...
Selecting previously unselected package golang-github-hashicorp-go-msgpack-dev.
Preparing to unpack .../97-golang-github-hashicorp-go-msgpack-dev_0.5.5-1_all.deb ...
Unpacking golang-github-hashicorp-go-msgpack-dev (0.5.5-1) ...
Setting up libpipeline1:i386 (1.5.7-1) ...
Setting up golang-gopkg-yaml.v2-dev (2.4.0-4) ...
Setting up golang-github-hashicorp-go-uuid-dev (1.0.2-2) ...
Setting up libicu72:i386 (72.1-3) ...
Setting up bsdextrautils (2.38.1-5+b1) ...
Setting up golang-github-creack-pty-dev (1.1.18-1) ...
Setting up golang-github-pkg-diff-dev (0.0~git20210226.20ebb0f-1) ...
Setting up golang-github-stretchr-objx-dev (0.5.0-1) ...
Setting up libmagic-mgc (1:5.44-3) ...
Setting up golang-github-pkg-errors-dev (0.9.1-2) ...
Setting up golang-github-hashicorp-golang-lru-dev (0.5.4-2) ...
Setting up libarchive-zip-perl (1.68-1) ...
Setting up golang-gopkg-yaml.v3-dev (3.0.1-3) ...
Setting up libdebhelper-perl (13.11.4) ...
Setting up golang-golang-x-sys-dev (0.3.0-1) ...
Setting up libmagic1:i386 (1:5.44-3) ...
Setting up golang-github-pmezard-go-difflib-dev (1.0.0-3) ...
Setting up golang-github-modern-go-concurrent-dev (1.0.3-1.1) ...
Setting up gettext-base (0.21-12) ...
Setting up golang-1.19-src (1.19.8-2) ...
Setting up m4 (1.4.19-3) ...
Setting up golang-github-circonus-labs-circonusllhist-dev (0.0~git20191022.ec08cde-1) ...
Setting up golang-github-golang-protobuf-1-3-dev (1.3.5-4) ...
Setting up file (1:5.44-3) ...
Setting up golang-github-beorn7-perks-dev (1.0.1-1) ...
Setting up golang-github-hashicorp-go-cleanhttp-dev (0.5.2-1) ...
Setting up libsasl2-modules-db:i386 (2.1.28+dfsg-10) ...
Setting up golang-github-cespare-xxhash-dev (2.1.1-2) ...
Setting up golang-gopkg-tomb.v2-dev (0.0~git20161208.d5d1b58-3) ...
Setting up golang-github-jpillora-backoff-dev (1.0.0-1.1) ...
Setting up golang-github-davecgh-go-spew-dev (1.1.1-3) ...
Setting up autotools-dev (20220109.1) ...
Setting up golang-github-pascaldekloe-goe-dev (0.1.0-4) ...
Setting up golang-github-go-logfmt-logfmt-dev (0.5.0-2) ...
Setting up golang-google-protobuf-dev (1.28.1-3) ...
Setting up golang-github-kr-text-dev (0.2.0-2) ...
Setting up golang-github-stretchr-testify-dev (1.8.1-1) ...
Setting up golang-github-hashicorp-go-immutable-radix-dev (1.3.1-3) ...
Setting up golang-golang-x-sync-dev (0.1.0-1) ...
Setting up autopoint (0.21-12) ...
Setting up libsasl2-2:i386 (2.1.28+dfsg-10) ...
Setting up golang-github-julienschmidt-httprouter-dev (1.3.0-2) ...
Setting up autoconf (2.71-3) ...
Setting up golang-github-tv42-httpunix-dev (0.0~git20150427.b75d861-3) ...
Setting up sensible-utils (0.0.17+nmu1) ...
Setting up golang-golang-x-text-dev (0.7.0-1) ...
Setting up libuchardet0:i386 (0.0.7-1) ...
Setting up golang-github-prometheus-procfs-dev (0.8.0-3) ...
Setting up golang-github-alecthomas-units-dev (0.0~git20211218.b94a6e3-1) ...
Setting up golang-github-modern-go-reflect2-dev (1.0.2-2) ...
Setting up libsub-override-perl (0.09-4) ...
Setting up golang-github-prometheus-client-model-dev (0.3.0-3) ...
Setting up golang-github-datadog-datadog-go-dev (2.1.0-3) ...
Setting up golang-github-rogpeppe-go-internal-dev (1.9.0-1) ...
Setting up golang-src (2:1.19~1) ...
Setting up libelf1:i386 (0.188-2.1) ...
Setting up golang-github-vmihailenco-tagparser-dev (0.1.1-4) ...
Setting up libxml2:i386 (2.9.14+dfsg-1.3~deb12u1) ...
Setting up golang-github-jmespath-go-jmespath-dev (0.4.0-2) ...
Setting up golang-1.19-go (1.19.8-2) ...
Setting up automake (1:1.16.5-1.3) ...
update-alternatives: using /usr/bin/automake-1.16 to provide /usr/bin/automake (automake) in auto mode
Setting up golang-github-kr-pretty-dev (0.3.0-1) ...
Setting up libfile-stripnondeterminism-perl (1.13.1-1) ...
Setting up golang-github-mattn-go-isatty-dev (0.0.17-1) ...
Setting up golang-github-go-kit-log-dev (0.2.1-1) ...
Setting up gettext (0.21-12) ...
Setting up golang-gopkg-vmihailenco-msgpack.v2-dev (4.3.1-2) ...
Setting up golang-gopkg-alecthomas-kingpin.v2-dev (2.2.6-4) ...
Setting up libtool (2.4.7-5) ...
Setting up golang-go:i386 (2:1.19~1) ...
Setting up golang-github-mattn-go-colorable-dev (0.1.13-1) ...
Setting up golang-protobuf-extensions-dev (1.0.1-3) ...
Setting up golang-github-json-iterator-go-dev (1.1.12-1) ...
Setting up golang-gopkg-check.v1-dev (0.0+git20200902.038fdea-1) ...
Setting up golang-any:i386 (2:1.19~1) ...
Setting up intltool-debian (0.35.0+20060710.6) ...
Setting up dh-autoreconf (20) ...
Setting up libsasl2-dev (2.1.28+dfsg-10) ...
Setting up golang-gopkg-mgo.v2-dev (2016.08.01-7) ...
Setting up golang-golang-x-net-dev (1:0.7.0+dfsg-1) ...
Setting up dh-strip-nondeterminism (1.13.1-1) ...
Setting up dwz (0.15-1) ...
Setting up groff-base (1.22.4-10) ...
Setting up golang-golang-x-oauth2-dev (0.3.0-1) ...
Setting up golang-github-aws-aws-sdk-go-dev (1.44.133-1) ...
Setting up golang-github-fatih-color-dev (1.7.0-1) ...
Setting up golang-github-hashicorp-go-msgpack-dev (0.5.5-1) ...
Setting up po-debconf (1.0.21+nmu1) ...
Setting up man-db (2.11.2-2) ...
Not building database; man-db/auto-update is not 'true'.
Setting up golang-github-hashicorp-go-hclog-dev (0.11.0-1) ...
Setting up debhelper (13.11.4) ...
Setting up golang-github-hashicorp-go-retryablehttp-dev (0.7.1-1) ...
Setting up dh-golang (1.59) ...
Setting up golang-github-circonus-labs-circonus-gometrics-dev (2.3.1-4) ...
Setting up golang-github-mwitkow-go-conntrack-dev (0.0~git20190716.2f06839-3) ...
Setting up golang-github-prometheus-common-dev (0.39.0-2) ...
Setting up golang-github-prometheus-client-golang-dev (1.14.0-3) ...
Setting up golang-github-armon-go-metrics-dev (0.3.4-1) ...
Processing triggers for libc-bin (2.36-9+deb12u3) ...
Reading package lists...
Building dependency tree...
Reading state information...
Reading extended state information...
Initializing package states...
Writing extended state information...
Building tag database...
 -> Finished parsing the build-deps
Reading package lists...
Building dependency tree...
Reading state information...
fakeroot is already the newest version (1.31-1.2).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
I: Building the package
I: user script /srv/workspace/pbuilder/15535/tmp/hooks/A99_set_merged_usr starting
Not re-configuring usrmerge for bookworm
I: user script /srv/workspace/pbuilder/15535/tmp/hooks/A99_set_merged_usr finished
hostname: Name or service not known
I: Running cd /build/reproducible-path/golang-github-hashicorp-raft-1.1.2/ && env PATH="/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/i/capture/the/path" HOME="/nonexistent/second-build" dpkg-buildpackage -us -uc -b && env PATH="/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/i/capture/the/path" HOME="/nonexistent/second-build" dpkg-genchanges -S  > ../golang-github-hashicorp-raft_1.1.2-1_source.changes
dpkg-buildpackage: info: source package golang-github-hashicorp-raft
dpkg-buildpackage: info: source version 1.1.2-1
dpkg-buildpackage: info: source distribution unstable
dpkg-buildpackage: info: source changed by Dmitry Smirnov <onlyjob@debian.org>
 dpkg-source --before-build .
dpkg-buildpackage: info: host architecture i386
 fakeroot debian/rules clean
dh clean --buildsystem=golang --with=golang
   dh_auto_clean -O--buildsystem=golang
   dh_autoreconf_clean -O--buildsystem=golang
   dh_clean -O--buildsystem=golang
 debian/rules build
dh build --buildsystem=golang --with=golang
   dh_update_autotools_config -O--buildsystem=golang
   dh_autoreconf -O--buildsystem=golang
   dh_auto_configure -O--buildsystem=golang
   dh_auto_build -O--buildsystem=golang
	cd obj-i686-linux-gnu && go install -trimpath -v -p 7 github.com/hashicorp/raft github.com/hashicorp/raft/bench
internal/unsafeheader
internal/goarch
internal/goexperiment
internal/race
internal/cpu
internal/goos
runtime/internal/atomic
runtime/internal/syscall
internal/abi
runtime/internal/math
sync/atomic
runtime/internal/sys
unicode/utf8
unicode
container/list
internal/itoa
math/bits
crypto/internal/subtle
crypto/subtle
crypto/internal/boring/sig
encoding
internal/bytealg
unicode/utf16
internal/nettrace
vendor/golang.org/x/crypto/cryptobyte/asn1
vendor/golang.org/x/crypto/internal/subtle
math
runtime
internal/reflectlite
sync
internal/testlog
internal/singleflight
internal/sysinfo
math/rand
runtime/cgo
errors
sort
internal/oserror
io
internal/safefilepath
path
strconv
syscall
github.com/hashicorp/golang-lru/simplelru
vendor/golang.org/x/net/dns/dnsmessage
crypto/internal/nistec/fiat
bytes
strings
hash
reflect
bufio
crypto
crypto/internal/randutil
github.com/hashicorp/go-immutable-radix
hash/crc32
crypto/rc4
vendor/golang.org/x/text/transform
net/http/internal/ascii
time
internal/syscall/unix
internal/syscall/execenv
regexp/syntax
html
hash/crc64
crypto/internal/nistec
regexp
context
io/fs
internal/poll
os
internal/fmtsort
encoding/binary
encoding/base64
crypto/cipher
crypto/internal/edwards25519/field
crypto/md5
vendor/golang.org/x/crypto/internal/poly1305
vendor/golang.org/x/crypto/curve25519/internal/field
encoding/pem
golang.org/x/sys/unix
crypto/internal/edwards25519
fmt
internal/godebug
crypto/internal/boring
crypto/des
internal/intern
path/filepath
crypto/aes
net/netip
crypto/sha512
crypto/hmac
crypto/sha1
crypto/sha256
vendor/golang.org/x/crypto/chacha20
vendor/golang.org/x/crypto/hkdf
io/ioutil
os/signal
vendor/golang.org/x/crypto/chacha20poly1305
net
encoding/json
log
math/big
compress/flate
encoding/hex
vendor/golang.org/x/crypto/curve25519
net/url
vendor/golang.org/x/text/unicode/bidi
vendor/golang.org/x/text/unicode/norm
compress/gzip
vendor/golang.org/x/net/http2/hpack
mime
vendor/golang.org/x/text/secure/bidirule
mime/quotedprintable
net/http/internal
github.com/mattn/go-isatty
encoding/gob
go/token
text/template/parse
github.com/mattn/go-colorable
crypto/rand
crypto/elliptic
crypto/internal/boring/bbig
encoding/asn1
crypto/ed25519
crypto/rsa
crypto/dsa
vendor/golang.org/x/net/idna
github.com/fatih/color
flag
github.com/hashicorp/go-hclog
vendor/golang.org/x/crypto/cryptobyte
crypto/x509/pkix
text/template
runtime/debug
runtime/trace
testing
crypto/ecdsa
html/template
vendor/golang.org/x/net/http/httpproxy
crypto/x509
net/textproto
vendor/golang.org/x/net/http/httpguts
mime/multipart
crypto/tls
net/http/httptrace
net/http
net/rpc
github.com/armon/go-metrics
github.com/hashicorp/go-msgpack/codec
github.com/hashicorp/raft
github.com/hashicorp/raft/bench
   dh_auto_test -O--buildsystem=golang
	cd obj-i686-linux-gnu && go test -vet=off -v -p 7 github.com/hashicorp/raft github.com/hashicorp/raft/bench
=== RUN   TestCommitment_setVoters
--- PASS: TestCommitment_setVoters (0.00s)
=== RUN   TestCommitment_match_max
--- PASS: TestCommitment_match_max (0.00s)
=== RUN   TestCommitment_match_nonVoting
--- PASS: TestCommitment_match_nonVoting (0.00s)
=== RUN   TestCommitment_recalculate
--- PASS: TestCommitment_recalculate (0.00s)
=== RUN   TestCommitment_recalculate_startIndex
--- PASS: TestCommitment_recalculate_startIndex (0.00s)
=== RUN   TestCommitment_noVoterSanity
--- PASS: TestCommitment_noVoterSanity (0.00s)
=== RUN   TestCommitment_singleVoter
--- PASS: TestCommitment_singleVoter (0.00s)
=== RUN   TestConfiguration_Configuration_Clone
--- PASS: TestConfiguration_Configuration_Clone (0.00s)
=== RUN   TestConfiguration_configurations_Clone
--- PASS: TestConfiguration_configurations_Clone (0.00s)
=== RUN   TestConfiguration_hasVote
--- PASS: TestConfiguration_hasVote (0.00s)
=== RUN   TestConfiguration_checkConfiguration
--- PASS: TestConfiguration_checkConfiguration (0.00s)
=== RUN   TestConfiguration_nextConfiguration_table
--- PASS: TestConfiguration_nextConfiguration_table (0.00s)
=== RUN   TestConfiguration_nextConfiguration_prevIndex
--- PASS: TestConfiguration_nextConfiguration_prevIndex (0.00s)
=== RUN   TestConfiguration_nextConfiguration_checkConfiguration
--- PASS: TestConfiguration_nextConfiguration_checkConfiguration (0.00s)
=== RUN   TestConfiguration_encodeDecodePeers
--- PASS: TestConfiguration_encodeDecodePeers (0.00s)
=== RUN   TestConfiguration_encodeDecodeConfiguration
--- PASS: TestConfiguration_encodeDecodeConfiguration (0.00s)
=== RUN   TestDiscardSnapshotStoreImpl
--- PASS: TestDiscardSnapshotStoreImpl (0.00s)
=== RUN   TestDiscardSnapshotSinkImpl
--- PASS: TestDiscardSnapshotSinkImpl (0.00s)
=== RUN   TestFileSnapshotStoreImpl
--- PASS: TestFileSnapshotStoreImpl (0.00s)
=== RUN   TestFileSnapshotSinkImpl
--- PASS: TestFileSnapshotSinkImpl (0.00s)
=== RUN   TestFileSS_CreateSnapshotMissingParentDir
    testing.go:153: 2024-01-08T00:30:53.958Z [INFO]  creating new snapshot: path=/tmp/raft1326029728/raft553529258/snapshots/3-10-1704673853958.tmp
--- PASS: TestFileSS_CreateSnapshotMissingParentDir (0.00s)
=== RUN   TestFileSS_CreateSnapshot
    testing.go:153: 2024-01-08T00:30:53.962Z [INFO]  creating new snapshot: path=/tmp/raft2195550648/snapshots/3-10-1704673853962.tmp
    testing.go:153: 2024-01-08T00:30:53.964Z [WARN]  found temporary snapshot: name=3-10-1704673853962.tmp
--- PASS: TestFileSS_CreateSnapshot (0.01s)
=== RUN   TestFileSS_CancelSnapshot
    testing.go:153: 2024-01-08T00:30:53.972Z [INFO]  creating new snapshot: path=/tmp/raft2000699288/snapshots/3-10-1704673853972.tmp
--- PASS: TestFileSS_CancelSnapshot (0.00s)
=== RUN   TestFileSS_Retention
    testing.go:153: 2024-01-08T00:30:53.976Z [INFO]  creating new snapshot: path=/tmp/raft2965010602/snapshots/3-10-1704673853976.tmp
    testing.go:153: 2024-01-08T00:30:53.984Z [INFO]  creating new snapshot: path=/tmp/raft2965010602/snapshots/3-11-1704673853984.tmp
    testing.go:153: 2024-01-08T00:30:53.992Z [INFO]  creating new snapshot: path=/tmp/raft2965010602/snapshots/3-12-1704673853992.tmp
    testing.go:153: 2024-01-08T00:30:53.999Z [INFO]  reaping snapshot: path=/tmp/raft2965010602/snapshots/3-10-1704673853976
    testing.go:153: 2024-01-08T00:30:53.999Z [INFO]  creating new snapshot: path=/tmp/raft2965010602/snapshots/3-13-1704673853999.tmp
    testing.go:153: 2024-01-08T00:30:54.006Z [INFO]  reaping snapshot: path=/tmp/raft2965010602/snapshots/3-11-1704673853984
    testing.go:153: 2024-01-08T00:30:54.007Z [INFO]  creating new snapshot: path=/tmp/raft2965010602/snapshots/3-14-1704673854007.tmp
    testing.go:153: 2024-01-08T00:30:54.014Z [INFO]  reaping snapshot: path=/tmp/raft2965010602/snapshots/3-12-1704673853992
--- PASS: TestFileSS_Retention (0.04s)
=== RUN   TestFileSS_BadPerm
--- PASS: TestFileSS_BadPerm (0.00s)
=== RUN   TestFileSS_MissingParentDir
--- PASS: TestFileSS_MissingParentDir (0.00s)
=== RUN   TestFileSS_Ordering
    testing.go:153: 2024-01-08T00:30:54.015Z [INFO]  creating new snapshot: path=/tmp/raft1498870585/snapshots/5-130350-1704673854015.tmp
    testing.go:153: 2024-01-08T00:30:54.023Z [INFO]  creating new snapshot: path=/tmp/raft1498870585/snapshots/36-204917-1704673854023.tmp
--- PASS: TestFileSS_Ordering (0.02s)
=== RUN   TestDeferFutureSuccess
--- PASS: TestDeferFutureSuccess (0.00s)
=== RUN   TestDeferFutureError
--- PASS: TestDeferFutureError (0.00s)
=== RUN   TestDeferFutureConcurrent
--- PASS: TestDeferFutureConcurrent (0.00s)
=== RUN   TestInmemSnapshotStoreImpl
--- PASS: TestInmemSnapshotStoreImpl (0.00s)
=== RUN   TestInmemSnapshotSinkImpl
--- PASS: TestInmemSnapshotSinkImpl (0.00s)
=== RUN   TestInmemSS_CreateSnapshot
--- PASS: TestInmemSS_CreateSnapshot (0.00s)
=== RUN   TestInmemSS_OpenSnapshotTwice
--- PASS: TestInmemSS_OpenSnapshotTwice (0.00s)
=== RUN   TestInmemTransportImpl
--- PASS: TestInmemTransportImpl (0.00s)
=== RUN   TestInmemTransportWriteTimeout
--- PASS: TestInmemTransportWriteTimeout (0.01s)
=== RUN   TestRaft_Integ
--- SKIP: TestRaft_Integ (0.00s)
=== RUN   TestLogCache
--- PASS: TestLogCache (0.00s)
=== RUN   TestNetworkTransport_CloseStreams
--- PASS: TestNetworkTransport_CloseStreams (0.01s)
=== RUN   TestNetworkTransport_StartStop
--- PASS: TestNetworkTransport_StartStop (0.00s)
=== RUN   TestNetworkTransport_Heartbeat_FastPath
--- PASS: TestNetworkTransport_Heartbeat_FastPath (0.00s)
=== RUN   TestNetworkTransport_AppendEntries
--- PASS: TestNetworkTransport_AppendEntries (0.00s)
=== RUN   TestNetworkTransport_AppendEntriesPipeline
--- PASS: TestNetworkTransport_AppendEntriesPipeline (0.00s)
=== RUN   TestNetworkTransport_AppendEntriesPipeline_CloseStreams
    net_transport_test.go:339: Disabled unreliable test
--- SKIP: TestNetworkTransport_AppendEntriesPipeline_CloseStreams (0.00s)
=== RUN   TestNetworkTransport_RequestVote
--- PASS: TestNetworkTransport_RequestVote (0.00s)
=== RUN   TestNetworkTransport_InstallSnapshot
--- PASS: TestNetworkTransport_InstallSnapshot (0.00s)
=== RUN   TestNetworkTransport_EncodeDecode
--- PASS: TestNetworkTransport_EncodeDecode (0.00s)
=== RUN   TestNetworkTransport_EncodeDecode_AddressProvider
--- PASS: TestNetworkTransport_EncodeDecode_AddressProvider (0.00s)
=== RUN   TestNetworkTransport_PooledConn
--- PASS: TestNetworkTransport_PooledConn (0.03s)
=== RUN   TestNetworkTransport_ListenBackoff
    net_transport_test.go:718: countingWriter: 2024-01-08T00:30:54.089Z [ERROR] test: failed to accept connection: error="intentional error in test"
        
    net_transport_test.go:718: countingWriter: 2024-01-08T00:30:54.094Z [ERROR] test: failed to accept connection: error="intentional error in test"
        
    net_transport_test.go:718: countingWriter: 2024-01-08T00:30:54.105Z [ERROR] test: failed to accept connection: error="intentional error in test"
        
    net_transport_test.go:718: countingWriter: 2024-01-08T00:30:54.125Z [ERROR] test: failed to accept connection: error="intentional error in test"
        
    net_transport_test.go:718: countingWriter: 2024-01-08T00:30:54.165Z [ERROR] test: failed to accept connection: error="intentional error in test"
        
    net_transport_test.go:718: countingWriter: 2024-01-08T00:30:54.246Z [ERROR] test: failed to accept connection: error="intentional error in test"
        
    net_transport_test.go:718: countingWriter: 2024-01-08T00:30:54.407Z [ERROR] test: failed to accept connection: error="intentional error in test"
        
    net_transport_test.go:718: countingWriter: 2024-01-08T00:30:54.728Z [ERROR] test: failed to accept connection: error="intentional error in test"
        
    net_transport_test.go:718: countingWriter: 2024-01-08T00:30:55.368Z [ERROR] test: failed to accept connection: error="intentional error in test"
        
    net_transport_test.go:718: countingWriter: 2024-01-08T00:30:56.372Z [ERROR] test: failed to accept connection: error="intentional error in test"
        
    net_transport_test.go:718: countingWriter: 2024-01-08T00:30:57.376Z [ERROR] test: failed to accept connection: error="intentional error in test"
        
--- PASS: TestNetworkTransport_ListenBackoff (4.00s)
=== RUN   TestPeersJSON_BadConfiguration
--- PASS: TestPeersJSON_BadConfiguration (0.00s)
=== RUN   TestPeersJSON_ReadPeersJSON
--- PASS: TestPeersJSON_ReadPeersJSON (0.00s)
=== RUN   TestPeersJSON_ReadConfigJSON
--- PASS: TestPeersJSON_ReadConfigJSON (0.00s)
=== RUN   TestRaft_StartStop
testLoggerAdapter verbose: server-43a6be72-e193-8e23-a74a-0da1252ae32c: 2024-01-08T00:30:58.096Z [INFO]  server-43a6be72-e193-8e23-a74a-0da1252ae32c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-43a6be72-e193-8e23-a74a-0da1252ae32c Address:43a6be72-e193-8e23-a74a-0da1252ae32c}]"
    testing.go:149: server-43a6be72-e193-8e23-a74a-0da1252ae32c: 2024-01-08T00:30:58.096Z [INFO]  server-43a6be72-e193-8e23-a74a-0da1252ae32c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-43a6be72-e193-8e23-a74a-0da1252ae32c Address:43a6be72-e193-8e23-a74a-0da1252ae32c}]"
--- PASS: TestRaft_StartStop (0.00s)
=== RUN   TestRaft_AfterShutdown
testLoggerAdapter verbose: server-a683a6f6-84c9-0ab9-b8dc-a5b3b6309898: 2024-01-08T00:30:58.098Z [INFO]  server-a683a6f6-84c9-0ab9-b8dc-a5b3b6309898: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a683a6f6-84c9-0ab9-b8dc-a5b3b6309898 Address:a683a6f6-84c9-0ab9-b8dc-a5b3b6309898}]"
    testing.go:149: server-a683a6f6-84c9-0ab9-b8dc-a5b3b6309898: 2024-01-08T00:30:58.098Z [INFO]  server-a683a6f6-84c9-0ab9-b8dc-a5b3b6309898: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-a683a6f6-84c9-0ab9-b8dc-a5b3b6309898 Address:a683a6f6-84c9-0ab9-b8dc-a5b3b6309898}]"
--- PASS: TestRaft_AfterShutdown (0.00s)
=== RUN   TestRaft_LiveBootstrap
testLoggerAdapter verbose: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.099Z [INFO]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: initial configuration: index=0 servers=[]
    testing.go:149: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.099Z [INFO]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: initial configuration: index=0 servers=[]
testLoggerAdapter verbose: server-973d02ab-ccbc-3aa3-dfdd-7014fee58249: 2024-01-08T00:30:58.099Z [INFO]  server-973d02ab-ccbc-3aa3-dfdd-7014fee58249: initial configuration: index=0 servers=[]
    testing.go:149: server-973d02ab-ccbc-3aa3-dfdd-7014fee58249: 2024-01-08T00:30:58.099Z [INFO]  server-973d02ab-ccbc-3aa3-dfdd-7014fee58249: initial configuration: index=0 servers=[]
testLoggerAdapter verbose: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.099Z [INFO]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: entering follower state: follower="Node at a2bf7e73-f626-d3e2-e4ac-44b433be2cbb [Follower]" leader=
    testing.go:149: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.099Z [INFO]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: entering follower state: follower="Node at a2bf7e73-f626-d3e2-e4ac-44b433be2cbb [Follower]" leader=
testLoggerAdapter verbose: server-973d02ab-ccbc-3aa3-dfdd-7014fee58249: 2024-01-08T00:30:58.099Z [INFO]  server-973d02ab-ccbc-3aa3-dfdd-7014fee58249: entering follower state: follower="Node at 973d02ab-ccbc-3aa3-dfdd-7014fee58249 [Follower]" leader=
    testing.go:149: server-973d02ab-ccbc-3aa3-dfdd-7014fee58249: 2024-01-08T00:30:58.099Z [INFO]  server-973d02ab-ccbc-3aa3-dfdd-7014fee58249: entering follower state: follower="Node at 973d02ab-ccbc-3aa3-dfdd-7014fee58249 [Follower]" leader=
testLoggerAdapter verbose: server-a29e7003-fbdb-813d-959e-b1862c65567c: 2024-01-08T00:30:58.099Z [INFO]  server-a29e7003-fbdb-813d-959e-b1862c65567c: initial configuration: index=0 servers=[]
    testing.go:149: server-a29e7003-fbdb-813d-959e-b1862c65567c: 2024-01-08T00:30:58.099Z [INFO]  server-a29e7003-fbdb-813d-959e-b1862c65567c: initial configuration: index=0 servers=[]
testLoggerAdapter verbose: server-a29e7003-fbdb-813d-959e-b1862c65567c: 2024-01-08T00:30:58.099Z [INFO]  server-a29e7003-fbdb-813d-959e-b1862c65567c: entering follower state: follower="Node at a29e7003-fbdb-813d-959e-b1862c65567c [Follower]" leader=
    testing.go:149: server-a29e7003-fbdb-813d-959e-b1862c65567c: 2024-01-08T00:30:58.099Z [INFO]  server-a29e7003-fbdb-813d-959e-b1862c65567c: entering follower state: follower="Node at a29e7003-fbdb-813d-959e-b1862c65567c [Follower]" leader=
testLoggerAdapter verbose: cluster: 2024-01-08T00:30:58.099Z [INFO]  starting stability test: raft-state=Follower
    testing.go:149: cluster: 2024-01-08T00:30:58.099Z [INFO]  starting stability test: raft-state=Follower
testLoggerAdapter verbose: server-973d02ab-ccbc-3aa3-dfdd-7014fee58249: 2024-01-08T00:30:58.164Z [WARN]  server-973d02ab-ccbc-3aa3-dfdd-7014fee58249: no known peers, aborting election
    testing.go:149: server-973d02ab-ccbc-3aa3-dfdd-7014fee58249: 2024-01-08T00:30:58.164Z [WARN]  server-973d02ab-ccbc-3aa3-dfdd-7014fee58249: no known peers, aborting election
testLoggerAdapter verbose: server-a29e7003-fbdb-813d-959e-b1862c65567c: 2024-01-08T00:30:58.174Z [WARN]  server-a29e7003-fbdb-813d-959e-b1862c65567c: no known peers, aborting election
    testing.go:149: server-a29e7003-fbdb-813d-959e-b1862c65567c: 2024-01-08T00:30:58.174Z [WARN]  server-a29e7003-fbdb-813d-959e-b1862c65567c: no known peers, aborting election
testLoggerAdapter verbose: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.181Z [WARN]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.181Z [WARN]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.181Z [INFO]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: entering candidate state: node="Node at a2bf7e73-f626-d3e2-e4ac-44b433be2cbb [Candidate]" term=2
    testing.go:149: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.181Z [INFO]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: entering candidate state: node="Node at a2bf7e73-f626-d3e2-e4ac-44b433be2cbb [Candidate]" term=2
testLoggerAdapter verbose: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.181Z [INFO]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: election won: tally=2
    testing.go:149: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.181Z [INFO]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: election won: tally=2
testLoggerAdapter verbose: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.181Z [INFO]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: entering leader state: leader="Node at a2bf7e73-f626-d3e2-e4ac-44b433be2cbb [Leader]"
    testing.go:149: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.181Z [INFO]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: entering leader state: leader="Node at a2bf7e73-f626-d3e2-e4ac-44b433be2cbb [Leader]"
testLoggerAdapter verbose: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.181Z [INFO]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: added peer, starting replication: peer=server-973d02ab-ccbc-3aa3-dfdd-7014fee58249
    testing.go:149: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.181Z [INFO]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: added peer, starting replication: peer=server-973d02ab-ccbc-3aa3-dfdd-7014fee58249
testLoggerAdapter verbose: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.181Z [INFO]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: added peer, starting replication: peer=server-a29e7003-fbdb-813d-959e-b1862c65567c
    testing.go:149: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.181Z [INFO]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: added peer, starting replication: peer=server-a29e7003-fbdb-813d-959e-b1862c65567c
testLoggerAdapter verbose: server-a29e7003-fbdb-813d-959e-b1862c65567c: 2024-01-08T00:30:58.182Z [WARN]  server-a29e7003-fbdb-813d-959e-b1862c65567c: failed to get previous log: previous-index=1 last-index=0 error="log not found"
    testing.go:149: server-a29e7003-fbdb-813d-959e-b1862c65567c: 2024-01-08T00:30:58.182Z [WARN]  server-a29e7003-fbdb-813d-959e-b1862c65567c: failed to get previous log: previous-index=1 last-index=0 error="log not found"
testLoggerAdapter verbose: server-973d02ab-ccbc-3aa3-dfdd-7014fee58249: 2024-01-08T00:30:58.182Z [WARN]  server-973d02ab-ccbc-3aa3-dfdd-7014fee58249: failed to get previous log: previous-index=1 last-index=0 error="log not found"
    testing.go:149: server-973d02ab-ccbc-3aa3-dfdd-7014fee58249: 2024-01-08T00:30:58.182Z [WARN]  server-973d02ab-ccbc-3aa3-dfdd-7014fee58249: failed to get previous log: previous-index=1 last-index=0 error="log not found"
testLoggerAdapter verbose: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.182Z [WARN]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: appendEntries rejected, sending older logs: peer="{Voter server-a29e7003-fbdb-813d-959e-b1862c65567c a29e7003-fbdb-813d-959e-b1862c65567c}" next=1
    testing.go:149: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.182Z [WARN]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: appendEntries rejected, sending older logs: peer="{Voter server-a29e7003-fbdb-813d-959e-b1862c65567c a29e7003-fbdb-813d-959e-b1862c65567c}" next=1
testLoggerAdapter verbose: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.182Z [WARN]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: appendEntries rejected, sending older logs: peer="{Voter server-973d02ab-ccbc-3aa3-dfdd-7014fee58249 973d02ab-ccbc-3aa3-dfdd-7014fee58249}" next=1
    testing.go:149: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.182Z [WARN]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: appendEntries rejected, sending older logs: peer="{Voter server-973d02ab-ccbc-3aa3-dfdd-7014fee58249 973d02ab-ccbc-3aa3-dfdd-7014fee58249}" next=1
testLoggerAdapter verbose: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.182Z [INFO]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: pipelining replication: peer="{Voter server-a29e7003-fbdb-813d-959e-b1862c65567c a29e7003-fbdb-813d-959e-b1862c65567c}"
    testing.go:149: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.182Z [INFO]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: pipelining replication: peer="{Voter server-a29e7003-fbdb-813d-959e-b1862c65567c a29e7003-fbdb-813d-959e-b1862c65567c}"
testLoggerAdapter verbose: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.182Z [INFO]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: pipelining replication: peer="{Voter server-973d02ab-ccbc-3aa3-dfdd-7014fee58249 973d02ab-ccbc-3aa3-dfdd-7014fee58249}"
    testing.go:149: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.182Z [INFO]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: pipelining replication: peer="{Voter server-973d02ab-ccbc-3aa3-dfdd-7014fee58249 973d02ab-ccbc-3aa3-dfdd-7014fee58249}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:30:58.287Z [INFO]  stable state for Follower reached at 2024-01-08 00:30:58.181969687 +0000 UTC m=+4.228858754 (2 nodes), 82.086071ms from start of poll, 82.684371ms from cluster start. Timeout at 2024-01-08 00:30:58.286983493 +0000 UTC m=+4.333872527, 105.013773ms after stability
    testing.go:149: cluster: 2024-01-08T00:30:58.287Z [INFO]  stable state for Follower reached at 2024-01-08 00:30:58.181969687 +0000 UTC m=+4.228858754 (2 nodes), 82.086071ms from start of poll, 82.684371ms from cluster start. Timeout at 2024-01-08 00:30:58.286983493 +0000 UTC m=+4.333872527, 105.013773ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:30:58.287Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:30:58.287Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: cluster: 2024-01-08T00:30:58.392Z [INFO]  stable state for Leader reached at 2024-01-08 00:30:58.287143559 +0000 UTC m=+4.334032597 (1 nodes), 2.047µs from start of poll, 187.858214ms from cluster start. Timeout at 2024-01-08 00:30:58.39280871 +0000 UTC m=+4.439697752, 105.665155ms after stability
    testing.go:149: cluster: 2024-01-08T00:30:58.392Z [INFO]  stable state for Leader reached at 2024-01-08 00:30:58.287143559 +0000 UTC m=+4.334032597 (1 nodes), 2.047µs from start of poll, 187.858214ms from cluster start. Timeout at 2024-01-08 00:30:58.39280871 +0000 UTC m=+4.439697752, 105.665155ms after stability
testLoggerAdapter verbose: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.404Z [INFO]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: aborting pipeline replication: peer="{Voter server-a29e7003-fbdb-813d-959e-b1862c65567c a29e7003-fbdb-813d-959e-b1862c65567c}"
    testing.go:149: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.404Z [INFO]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: aborting pipeline replication: peer="{Voter server-a29e7003-fbdb-813d-959e-b1862c65567c a29e7003-fbdb-813d-959e-b1862c65567c}"
testLoggerAdapter verbose: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.404Z [INFO]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: aborting pipeline replication: peer="{Voter server-973d02ab-ccbc-3aa3-dfdd-7014fee58249 973d02ab-ccbc-3aa3-dfdd-7014fee58249}"
    testing.go:149: server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: 2024-01-08T00:30:58.404Z [INFO]  server-a2bf7e73-f626-d3e2-e4ac-44b433be2cbb: aborting pipeline replication: peer="{Voter server-973d02ab-ccbc-3aa3-dfdd-7014fee58249 973d02ab-ccbc-3aa3-dfdd-7014fee58249}"
--- PASS: TestRaft_LiveBootstrap (0.31s)
=== RUN   TestRaft_RecoverCluster_NoState
testLoggerAdapter verbose: server-2b6174a5-f346-120a-ff5e-ae3a9dbffd11: 2024-01-08T00:30:58.405Z [INFO]  server-2b6174a5-f346-120a-ff5e-ae3a9dbffd11: initial configuration: index=0 servers=[]
    testing.go:149: server-2b6174a5-f346-120a-ff5e-ae3a9dbffd11: 2024-01-08T00:30:58.405Z [INFO]  server-2b6174a5-f346-120a-ff5e-ae3a9dbffd11: initial configuration: index=0 servers=[]
testLoggerAdapter verbose: server-2b6174a5-f346-120a-ff5e-ae3a9dbffd11: 2024-01-08T00:30:58.405Z [INFO]  server-2b6174a5-f346-120a-ff5e-ae3a9dbffd11: entering follower state: follower="Node at 2b6174a5-f346-120a-ff5e-ae3a9dbffd11 [Shutdown]" leader=
    testing.go:149: server-2b6174a5-f346-120a-ff5e-ae3a9dbffd11: 2024-01-08T00:30:58.405Z [INFO]  server-2b6174a5-f346-120a-ff5e-ae3a9dbffd11: entering follower state: follower="Node at 2b6174a5-f346-120a-ff5e-ae3a9dbffd11 [Shutdown]" leader=
--- PASS: TestRaft_RecoverCluster_NoState (0.00s)
=== RUN   TestRaft_RecoverCluster
    raft_test.go:119: Disabled unreliable test
--- SKIP: TestRaft_RecoverCluster (0.00s)
=== RUN   TestRaft_HasExistingState
testLoggerAdapter verbose: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.406Z [INFO]  server-8e70590a-d0ae-be79-3970-60fa015aafee: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8e70590a-d0ae-be79-3970-60fa015aafee Address:8e70590a-d0ae-be79-3970-60fa015aafee} {Suffrage:Voter ID:server-f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c Address:f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c}]"
    testing.go:149: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.406Z [INFO]  server-8e70590a-d0ae-be79-3970-60fa015aafee: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8e70590a-d0ae-be79-3970-60fa015aafee Address:8e70590a-d0ae-be79-3970-60fa015aafee} {Suffrage:Voter ID:server-f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c Address:f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c}]"
testLoggerAdapter verbose: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.406Z [INFO]  server-8e70590a-d0ae-be79-3970-60fa015aafee: entering follower state: follower="Node at 8e70590a-d0ae-be79-3970-60fa015aafee [Follower]" leader=
    testing.go:149: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.406Z [INFO]  server-8e70590a-d0ae-be79-3970-60fa015aafee: entering follower state: follower="Node at 8e70590a-d0ae-be79-3970-60fa015aafee [Follower]" leader=
testLoggerAdapter verbose: server-f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c: 2024-01-08T00:30:58.406Z [INFO]  server-f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8e70590a-d0ae-be79-3970-60fa015aafee Address:8e70590a-d0ae-be79-3970-60fa015aafee} {Suffrage:Voter ID:server-f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c Address:f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c}]"
    testing.go:149: server-f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c: 2024-01-08T00:30:58.406Z [INFO]  server-f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8e70590a-d0ae-be79-3970-60fa015aafee Address:8e70590a-d0ae-be79-3970-60fa015aafee} {Suffrage:Voter ID:server-f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c Address:f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c}]"
testLoggerAdapter verbose: server-f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c: 2024-01-08T00:30:58.406Z [INFO]  server-f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c: entering follower state: follower="Node at f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c [Follower]" leader=
    testing.go:149: server-f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c: 2024-01-08T00:30:58.406Z [INFO]  server-f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c: entering follower state: follower="Node at f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c [Follower]" leader=
testLoggerAdapter verbose: server-58d4ba3f-7492-7c34-c358-c714c349e806: 2024-01-08T00:30:58.407Z [INFO]  server-58d4ba3f-7492-7c34-c358-c714c349e806: initial configuration: index=0 servers=[]
    testing.go:149: server-58d4ba3f-7492-7c34-c358-c714c349e806: 2024-01-08T00:30:58.407Z [INFO]  server-58d4ba3f-7492-7c34-c358-c714c349e806: initial configuration: index=0 servers=[]
testLoggerAdapter verbose: cluster: 2024-01-08T00:30:58.407Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-58d4ba3f-7492-7c34-c358-c714c349e806: 2024-01-08T00:30:58.407Z [INFO]  server-58d4ba3f-7492-7c34-c358-c714c349e806: entering follower state: follower="Node at 58d4ba3f-7492-7c34-c358-c714c349e806 [Follower]" leader=
    testing.go:149: cluster: 2024-01-08T00:30:58.407Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: server-58d4ba3f-7492-7c34-c358-c714c349e806: 2024-01-08T00:30:58.407Z [INFO]  server-58d4ba3f-7492-7c34-c358-c714c349e806: entering follower state: follower="Node at 58d4ba3f-7492-7c34-c358-c714c349e806 [Follower]" leader=
testLoggerAdapter verbose: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.460Z [WARN]  server-8e70590a-d0ae-be79-3970-60fa015aafee: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.460Z [WARN]  server-8e70590a-d0ae-be79-3970-60fa015aafee: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.460Z [INFO]  server-8e70590a-d0ae-be79-3970-60fa015aafee: entering candidate state: node="Node at 8e70590a-d0ae-be79-3970-60fa015aafee [Candidate]" term=2
    testing.go:149: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.460Z [INFO]  server-8e70590a-d0ae-be79-3970-60fa015aafee: entering candidate state: node="Node at 8e70590a-d0ae-be79-3970-60fa015aafee [Candidate]" term=2
testLoggerAdapter verbose: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.460Z [INFO]  server-8e70590a-d0ae-be79-3970-60fa015aafee: election won: tally=2
    testing.go:149: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.460Z [INFO]  server-8e70590a-d0ae-be79-3970-60fa015aafee: election won: tally=2
testLoggerAdapter verbose: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.461Z [INFO]  server-8e70590a-d0ae-be79-3970-60fa015aafee: entering leader state: leader="Node at 8e70590a-d0ae-be79-3970-60fa015aafee [Leader]"
    testing.go:149: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.461Z [INFO]  server-8e70590a-d0ae-be79-3970-60fa015aafee: entering leader state: leader="Node at 8e70590a-d0ae-be79-3970-60fa015aafee [Leader]"
testLoggerAdapter verbose: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.461Z [INFO]  server-8e70590a-d0ae-be79-3970-60fa015aafee: added peer, starting replication: peer=server-f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c
    testing.go:149: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.461Z [INFO]  server-8e70590a-d0ae-be79-3970-60fa015aafee: added peer, starting replication: peer=server-f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c
testLoggerAdapter verbose: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.461Z [INFO]  server-8e70590a-d0ae-be79-3970-60fa015aafee: pipelining replication: peer="{Voter server-f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c}"
    testing.go:149: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.461Z [INFO]  server-8e70590a-d0ae-be79-3970-60fa015aafee: pipelining replication: peer="{Voter server-f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c}"
testLoggerAdapter verbose: server-58d4ba3f-7492-7c34-c358-c714c349e806: 2024-01-08T00:30:58.489Z [WARN]  server-58d4ba3f-7492-7c34-c358-c714c349e806: no known peers, aborting election
    testing.go:149: server-58d4ba3f-7492-7c34-c358-c714c349e806: 2024-01-08T00:30:58.489Z [WARN]  server-58d4ba3f-7492-7c34-c358-c714c349e806: no known peers, aborting election
testLoggerAdapter verbose: cluster: 2024-01-08T00:30:58.567Z [INFO]  stable state for Leader reached at 2024-01-08 00:30:58.461016792 +0000 UTC m=+4.507905846 (1 nodes), 53.682203ms from start of poll, 54.524866ms from cluster start. Timeout at 2024-01-08 00:30:58.567001104 +0000 UTC m=+4.613890312, 105.984466ms after stability
    testing.go:149: cluster: 2024-01-08T00:30:58.567Z [INFO]  stable state for Leader reached at 2024-01-08 00:30:58.461016792 +0000 UTC m=+4.507905846 (1 nodes), 53.682203ms from start of poll, 54.524866ms from cluster start. Timeout at 2024-01-08 00:30:58.567001104 +0000 UTC m=+4.613890312, 105.984466ms after stability
testLoggerAdapter verbose: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.567Z [INFO]  server-8e70590a-d0ae-be79-3970-60fa015aafee: updating configuration: command=AddStaging server-id=server-58d4ba3f-7492-7c34-c358-c714c349e806 server-addr=58d4ba3f-7492-7c34-c358-c714c349e806 servers="[{Suffrage:Voter ID:server-8e70590a-d0ae-be79-3970-60fa015aafee Address:8e70590a-d0ae-be79-3970-60fa015aafee} {Suffrage:Voter ID:server-f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c Address:f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c} {Suffrage:Voter ID:server-58d4ba3f-7492-7c34-c358-c714c349e806 Address:58d4ba3f-7492-7c34-c358-c714c349e806}]"
    testing.go:149: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.567Z [INFO]  server-8e70590a-d0ae-be79-3970-60fa015aafee: updating configuration: command=AddStaging server-id=server-58d4ba3f-7492-7c34-c358-c714c349e806 server-addr=58d4ba3f-7492-7c34-c358-c714c349e806 servers="[{Suffrage:Voter ID:server-8e70590a-d0ae-be79-3970-60fa015aafee Address:8e70590a-d0ae-be79-3970-60fa015aafee} {Suffrage:Voter ID:server-f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c Address:f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c} {Suffrage:Voter ID:server-58d4ba3f-7492-7c34-c358-c714c349e806 Address:58d4ba3f-7492-7c34-c358-c714c349e806}]"
testLoggerAdapter verbose: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.567Z [INFO]  server-8e70590a-d0ae-be79-3970-60fa015aafee: added peer, starting replication: peer=server-58d4ba3f-7492-7c34-c358-c714c349e806
    testing.go:149: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.567Z [INFO]  server-8e70590a-d0ae-be79-3970-60fa015aafee: added peer, starting replication: peer=server-58d4ba3f-7492-7c34-c358-c714c349e806
testLoggerAdapter verbose: server-58d4ba3f-7492-7c34-c358-c714c349e806: 2024-01-08T00:30:58.567Z [WARN]  server-58d4ba3f-7492-7c34-c358-c714c349e806: failed to get previous log: previous-index=3 last-index=0 error="log not found"
    testing.go:149: server-58d4ba3f-7492-7c34-c358-c714c349e806: 2024-01-08T00:30:58.567Z [WARN]  server-58d4ba3f-7492-7c34-c358-c714c349e806: failed to get previous log: previous-index=3 last-index=0 error="log not found"
testLoggerAdapter verbose: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.567Z [WARN]  server-8e70590a-d0ae-be79-3970-60fa015aafee: appendEntries rejected, sending older logs: peer="{Voter server-58d4ba3f-7492-7c34-c358-c714c349e806 58d4ba3f-7492-7c34-c358-c714c349e806}" next=1
    testing.go:149: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.567Z [WARN]  server-8e70590a-d0ae-be79-3970-60fa015aafee: appendEntries rejected, sending older logs: peer="{Voter server-58d4ba3f-7492-7c34-c358-c714c349e806 58d4ba3f-7492-7c34-c358-c714c349e806}" next=1
testLoggerAdapter verbose: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.567Z [INFO]  server-8e70590a-d0ae-be79-3970-60fa015aafee: pipelining replication: peer="{Voter server-58d4ba3f-7492-7c34-c358-c714c349e806 58d4ba3f-7492-7c34-c358-c714c349e806}"
    testing.go:149: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.567Z [INFO]  server-8e70590a-d0ae-be79-3970-60fa015aafee: pipelining replication: peer="{Voter server-58d4ba3f-7492-7c34-c358-c714c349e806 58d4ba3f-7492-7c34-c358-c714c349e806}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:30:58.573Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:30:58.573Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: cluster: 2024-01-08T00:30:58.679Z [INFO]  stable state for Leader reached at 2024-01-08 00:30:58.573465699 +0000 UTC m=+4.620354750 (1 nodes), 2.015µs from start of poll, 166.97377ms from cluster start. Timeout at 2024-01-08 00:30:58.678984501 +0000 UTC m=+4.725873640, 105.51889ms after stability
    testing.go:149: cluster: 2024-01-08T00:30:58.679Z [INFO]  stable state for Leader reached at 2024-01-08 00:30:58.573465699 +0000 UTC m=+4.620354750 (1 nodes), 2.015µs from start of poll, 166.97377ms from cluster start. Timeout at 2024-01-08 00:30:58.678984501 +0000 UTC m=+4.725873640, 105.51889ms after stability
testLoggerAdapter verbose: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.679Z [INFO]  server-8e70590a-d0ae-be79-3970-60fa015aafee: aborting pipeline replication: peer="{Voter server-58d4ba3f-7492-7c34-c358-c714c349e806 58d4ba3f-7492-7c34-c358-c714c349e806}"
    testing.go:149: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.679Z [INFO]  server-8e70590a-d0ae-be79-3970-60fa015aafee: aborting pipeline replication: peer="{Voter server-58d4ba3f-7492-7c34-c358-c714c349e806 58d4ba3f-7492-7c34-c358-c714c349e806}"
testLoggerAdapter verbose: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.679Z [INFO]  server-8e70590a-d0ae-be79-3970-60fa015aafee: aborting pipeline replication: peer="{Voter server-f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c}"
    testing.go:149: server-8e70590a-d0ae-be79-3970-60fa015aafee: 2024-01-08T00:30:58.679Z [INFO]  server-8e70590a-d0ae-be79-3970-60fa015aafee: aborting pipeline replication: peer="{Voter server-f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c f03a1c4e-3b0d-ee16-9c68-5bc28760cd1c}"
--- PASS: TestRaft_HasExistingState (0.27s)
=== RUN   TestRaft_SingleNode
testLoggerAdapter verbose: server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b: 2024-01-08T00:30:58.680Z [INFO]  server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b Address:75d84fa3-09ba-8ebe-0fa9-ffd50de8294b}]"
    testing.go:149: server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b: 2024-01-08T00:30:58.680Z [INFO]  server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b Address:75d84fa3-09ba-8ebe-0fa9-ffd50de8294b}]"
testLoggerAdapter verbose: server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b: 2024-01-08T00:30:58.680Z [INFO]  server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b: entering follower state: follower="Node at 75d84fa3-09ba-8ebe-0fa9-ffd50de8294b [Follower]" leader=
    testing.go:149: server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b: 2024-01-08T00:30:58.680Z [INFO]  server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b: entering follower state: follower="Node at 75d84fa3-09ba-8ebe-0fa9-ffd50de8294b [Follower]" leader=
testLoggerAdapter verbose: server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b: 2024-01-08T00:30:58.769Z [WARN]  server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b: 2024-01-08T00:30:58.769Z [WARN]  server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b: 2024-01-08T00:30:58.769Z [INFO]  server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b: entering candidate state: node="Node at 75d84fa3-09ba-8ebe-0fa9-ffd50de8294b [Candidate]" term=2
    testing.go:149: server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b: 2024-01-08T00:30:58.769Z [INFO]  server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b: entering candidate state: node="Node at 75d84fa3-09ba-8ebe-0fa9-ffd50de8294b [Candidate]" term=2
testLoggerAdapter verbose: server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b: 2024-01-08T00:30:58.769Z [INFO]  server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b: election won: tally=1
    testing.go:149: server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b: 2024-01-08T00:30:58.769Z [INFO]  server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b: election won: tally=1
testLoggerAdapter verbose: server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b: 2024-01-08T00:30:58.769Z [INFO]  server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b: entering leader state: leader="Node at 75d84fa3-09ba-8ebe-0fa9-ffd50de8294b [Leader]"
    testing.go:149: server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b: 2024-01-08T00:30:58.769Z [INFO]  server-75d84fa3-09ba-8ebe-0fa9-ffd50de8294b: entering leader state: leader="Node at 75d84fa3-09ba-8ebe-0fa9-ffd50de8294b [Leader]"
--- PASS: TestRaft_SingleNode (0.09s)
=== RUN   TestRaft_TripleNode
testLoggerAdapter verbose: server-27979bdb-2e77-d2c5-e95a-fcc3441736e7: 2024-01-08T00:30:58.770Z [INFO]  server-27979bdb-2e77-d2c5-e95a-fcc3441736e7: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-27979bdb-2e77-d2c5-e95a-fcc3441736e7 Address:27979bdb-2e77-d2c5-e95a-fcc3441736e7} {Suffrage:Voter ID:server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28 Address:cb8bea42-e1e6-d359-7b3f-f3bde7859a28} {Suffrage:Voter ID:server-183056d4-fff1-d1ea-0306-8d317b8fc38d Address:183056d4-fff1-d1ea-0306-8d317b8fc38d}]"
    testing.go:149: server-27979bdb-2e77-d2c5-e95a-fcc3441736e7: 2024-01-08T00:30:58.770Z [INFO]  server-27979bdb-2e77-d2c5-e95a-fcc3441736e7: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-27979bdb-2e77-d2c5-e95a-fcc3441736e7 Address:27979bdb-2e77-d2c5-e95a-fcc3441736e7} {Suffrage:Voter ID:server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28 Address:cb8bea42-e1e6-d359-7b3f-f3bde7859a28} {Suffrage:Voter ID:server-183056d4-fff1-d1ea-0306-8d317b8fc38d Address:183056d4-fff1-d1ea-0306-8d317b8fc38d}]"
testLoggerAdapter verbose: server-27979bdb-2e77-d2c5-e95a-fcc3441736e7: 2024-01-08T00:30:58.770Z [INFO]  server-27979bdb-2e77-d2c5-e95a-fcc3441736e7: entering follower state: follower="Node at 27979bdb-2e77-d2c5-e95a-fcc3441736e7 [Follower]" leader=
    testing.go:149: server-27979bdb-2e77-d2c5-e95a-fcc3441736e7: 2024-01-08T00:30:58.770Z [INFO]  server-27979bdb-2e77-d2c5-e95a-fcc3441736e7: entering follower state: follower="Node at 27979bdb-2e77-d2c5-e95a-fcc3441736e7 [Follower]" leader=
testLoggerAdapter verbose: server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28: 2024-01-08T00:30:58.770Z [INFO]  server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-27979bdb-2e77-d2c5-e95a-fcc3441736e7 Address:27979bdb-2e77-d2c5-e95a-fcc3441736e7} {Suffrage:Voter ID:server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28 Address:cb8bea42-e1e6-d359-7b3f-f3bde7859a28} {Suffrage:Voter ID:server-183056d4-fff1-d1ea-0306-8d317b8fc38d Address:183056d4-fff1-d1ea-0306-8d317b8fc38d}]"
    testing.go:149: server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28: 2024-01-08T00:30:58.770Z [INFO]  server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-27979bdb-2e77-d2c5-e95a-fcc3441736e7 Address:27979bdb-2e77-d2c5-e95a-fcc3441736e7} {Suffrage:Voter ID:server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28 Address:cb8bea42-e1e6-d359-7b3f-f3bde7859a28} {Suffrage:Voter ID:server-183056d4-fff1-d1ea-0306-8d317b8fc38d Address:183056d4-fff1-d1ea-0306-8d317b8fc38d}]"
testLoggerAdapter verbose: server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28: 2024-01-08T00:30:58.771Z [INFO]  server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28: entering follower state: follower="Node at cb8bea42-e1e6-d359-7b3f-f3bde7859a28 [Follower]" leader=
    testing.go:149: server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28: 2024-01-08T00:30:58.771Z [INFO]  server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28: entering follower state: follower="Node at cb8bea42-e1e6-d359-7b3f-f3bde7859a28 [Follower]" leader=
testLoggerAdapter verbose: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:58.771Z [INFO]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-27979bdb-2e77-d2c5-e95a-fcc3441736e7 Address:27979bdb-2e77-d2c5-e95a-fcc3441736e7} {Suffrage:Voter ID:server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28 Address:cb8bea42-e1e6-d359-7b3f-f3bde7859a28} {Suffrage:Voter ID:server-183056d4-fff1-d1ea-0306-8d317b8fc38d Address:183056d4-fff1-d1ea-0306-8d317b8fc38d}]"
    testing.go:149: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:58.771Z [INFO]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-27979bdb-2e77-d2c5-e95a-fcc3441736e7 Address:27979bdb-2e77-d2c5-e95a-fcc3441736e7} {Suffrage:Voter ID:server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28 Address:cb8bea42-e1e6-d359-7b3f-f3bde7859a28} {Suffrage:Voter ID:server-183056d4-fff1-d1ea-0306-8d317b8fc38d Address:183056d4-fff1-d1ea-0306-8d317b8fc38d}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:30:58.771Z [INFO]  starting stability test: raft-state=Follower
    testing.go:149: cluster: 2024-01-08T00:30:58.771Z [INFO]  starting stability test: raft-state=Follower
testLoggerAdapter verbose: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:58.771Z [INFO]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: entering follower state: follower="Node at 183056d4-fff1-d1ea-0306-8d317b8fc38d [Follower]" leader=
    testing.go:149: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:58.771Z [INFO]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: entering follower state: follower="Node at 183056d4-fff1-d1ea-0306-8d317b8fc38d [Follower]" leader=
testLoggerAdapter verbose: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:58.839Z [WARN]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:58.839Z [WARN]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:58.839Z [INFO]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: entering candidate state: node="Node at 183056d4-fff1-d1ea-0306-8d317b8fc38d [Candidate]" term=2
    testing.go:149: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:58.839Z [INFO]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: entering candidate state: node="Node at 183056d4-fff1-d1ea-0306-8d317b8fc38d [Candidate]" term=2
testLoggerAdapter verbose: server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28: 2024-01-08T00:30:58.839Z [WARN]  server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28: 2024-01-08T00:30:58.839Z [WARN]  server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28: 2024-01-08T00:30:58.839Z [INFO]  server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28: entering candidate state: node="Node at cb8bea42-e1e6-d359-7b3f-f3bde7859a28 [Candidate]" term=2
    testing.go:149: server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28: 2024-01-08T00:30:58.839Z [INFO]  server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28: entering candidate state: node="Node at cb8bea42-e1e6-d359-7b3f-f3bde7859a28 [Candidate]" term=2
testLoggerAdapter verbose: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:58.839Z [INFO]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: election won: tally=2
    testing.go:149: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:58.839Z [INFO]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: election won: tally=2
testLoggerAdapter verbose: server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28: 2024-01-08T00:30:58.839Z [INFO]  server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28: duplicate requestVote for same term: term=2
testLoggerAdapter verbose: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:58.839Z [INFO]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: entering leader state: leader="Node at 183056d4-fff1-d1ea-0306-8d317b8fc38d [Leader]"
    testing.go:149: server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28: 2024-01-08T00:30:58.839Z [INFO]  server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28: duplicate requestVote for same term: term=2
    testing.go:149: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:58.839Z [INFO]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: entering leader state: leader="Node at 183056d4-fff1-d1ea-0306-8d317b8fc38d [Leader]"
testLoggerAdapter verbose: server-27979bdb-2e77-d2c5-e95a-fcc3441736e7: 2024-01-08T00:30:58.839Z [INFO]  server-27979bdb-2e77-d2c5-e95a-fcc3441736e7: duplicate requestVote for same term: term=2
testLoggerAdapter verbose: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:58.839Z [INFO]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: added peer, starting replication: peer=server-27979bdb-2e77-d2c5-e95a-fcc3441736e7
    testing.go:149: server-27979bdb-2e77-d2c5-e95a-fcc3441736e7: 2024-01-08T00:30:58.839Z [INFO]  server-27979bdb-2e77-d2c5-e95a-fcc3441736e7: duplicate requestVote for same term: term=2
    testing.go:149: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:58.839Z [INFO]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: added peer, starting replication: peer=server-27979bdb-2e77-d2c5-e95a-fcc3441736e7
testLoggerAdapter verbose: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:58.839Z [INFO]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: added peer, starting replication: peer=server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28
    testing.go:149: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:58.839Z [INFO]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: added peer, starting replication: peer=server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28
testLoggerAdapter verbose: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:58.839Z [WARN]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: rejecting vote request since we have a leader: from=cb8bea42-e1e6-d359-7b3f-f3bde7859a28 leader=183056d4-fff1-d1ea-0306-8d317b8fc38d
    testing.go:149: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:58.839Z [WARN]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: rejecting vote request since we have a leader: from=cb8bea42-e1e6-d359-7b3f-f3bde7859a28 leader=183056d4-fff1-d1ea-0306-8d317b8fc38d
testLoggerAdapter verbose: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:58.839Z [INFO]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: pipelining replication: peer="{Voter server-27979bdb-2e77-d2c5-e95a-fcc3441736e7 27979bdb-2e77-d2c5-e95a-fcc3441736e7}"
testLoggerAdapter verbose: server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28: 2024-01-08T00:30:58.839Z [INFO]  server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28: entering follower state: follower="Node at cb8bea42-e1e6-d359-7b3f-f3bde7859a28 [Follower]" leader=183056d4-fff1-d1ea-0306-8d317b8fc38d
    testing.go:149: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:58.839Z [INFO]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: pipelining replication: peer="{Voter server-27979bdb-2e77-d2c5-e95a-fcc3441736e7 27979bdb-2e77-d2c5-e95a-fcc3441736e7}"
testLoggerAdapter verbose: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:58.839Z [INFO]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: pipelining replication: peer="{Voter server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28 cb8bea42-e1e6-d359-7b3f-f3bde7859a28}"
    testing.go:149: server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28: 2024-01-08T00:30:58.839Z [INFO]  server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28: entering follower state: follower="Node at cb8bea42-e1e6-d359-7b3f-f3bde7859a28 [Follower]" leader=183056d4-fff1-d1ea-0306-8d317b8fc38d
    testing.go:149: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:58.839Z [INFO]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: pipelining replication: peer="{Voter server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28 cb8bea42-e1e6-d359-7b3f-f3bde7859a28}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:30:58.945Z [INFO]  stable state for Follower reached at 2024-01-08 00:30:58.839812924 +0000 UTC m=+4.886702045 (2 nodes), 68.471664ms from start of poll, 69.285934ms from cluster start. Timeout at 2024-01-08 00:30:58.944968828 +0000 UTC m=+4.991857920, 105.155875ms after stability
    testing.go:149: cluster: 2024-01-08T00:30:58.945Z [INFO]  stable state for Follower reached at 2024-01-08 00:30:58.839812924 +0000 UTC m=+4.886702045 (2 nodes), 68.471664ms from start of poll, 69.285934ms from cluster start. Timeout at 2024-01-08 00:30:58.944968828 +0000 UTC m=+4.991857920, 105.155875ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:30:58.945Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:30:58.945Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: cluster: 2024-01-08T00:30:59.050Z [INFO]  stable state for Leader reached at 2024-01-08 00:30:58.945100214 +0000 UTC m=+4.991989249 (1 nodes), 2.209µs from start of poll, 174.573138ms from cluster start. Timeout at 2024-01-08 00:30:59.050841434 +0000 UTC m=+5.097730509, 105.74126ms after stability
    testing.go:149: cluster: 2024-01-08T00:30:59.050Z [INFO]  stable state for Leader reached at 2024-01-08 00:30:58.945100214 +0000 UTC m=+4.991989249 (1 nodes), 2.209µs from start of poll, 174.573138ms from cluster start. Timeout at 2024-01-08 00:30:59.050841434 +0000 UTC m=+5.097730509, 105.74126ms after stability
testLoggerAdapter verbose: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:59.062Z [INFO]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: aborting pipeline replication: peer="{Voter server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28 cb8bea42-e1e6-d359-7b3f-f3bde7859a28}"
    testing.go:149: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:59.062Z [INFO]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: aborting pipeline replication: peer="{Voter server-cb8bea42-e1e6-d359-7b3f-f3bde7859a28 cb8bea42-e1e6-d359-7b3f-f3bde7859a28}"
testLoggerAdapter verbose: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:59.062Z [INFO]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: aborting pipeline replication: peer="{Voter server-27979bdb-2e77-d2c5-e95a-fcc3441736e7 27979bdb-2e77-d2c5-e95a-fcc3441736e7}"
    testing.go:149: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:59.062Z [INFO]  server-183056d4-fff1-d1ea-0306-8d317b8fc38d: aborting pipeline replication: peer="{Voter server-27979bdb-2e77-d2c5-e95a-fcc3441736e7 27979bdb-2e77-d2c5-e95a-fcc3441736e7}"
testLoggerAdapter verbose: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:59.112Z [ERROR] server-183056d4-fff1-d1ea-0306-8d317b8fc38d: failed to heartbeat to: peer=27979bdb-2e77-d2c5-e95a-fcc3441736e7 error="command timed out"
    testing.go:149: server-183056d4-fff1-d1ea-0306-8d317b8fc38d: 2024-01-08T00:30:59.112Z [ERROR] server-183056d4-fff1-d1ea-0306-8d317b8fc38d: failed to heartbeat to: peer=27979bdb-2e77-d2c5-e95a-fcc3441736e7 error="command timed out"
--- PASS: TestRaft_TripleNode (0.36s)
=== RUN   TestRaft_LeaderFail
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.130Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 Address:9c603e47-dcb5-0e30-0afa-aaca8df23da9} {Suffrage:Voter ID:server-aef27578-0e68-b160-b9b7-f17b6fd6c17e Address:aef27578-0e68-b160-b9b7-f17b6fd6c17e} {Suffrage:Voter ID:server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 Address:4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}]"
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.130Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 Address:9c603e47-dcb5-0e30-0afa-aaca8df23da9} {Suffrage:Voter ID:server-aef27578-0e68-b160-b9b7-f17b6fd6c17e Address:aef27578-0e68-b160-b9b7-f17b6fd6c17e} {Suffrage:Voter ID:server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 Address:4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}]"
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.131Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: entering follower state: follower="Node at 9c603e47-dcb5-0e30-0afa-aaca8df23da9 [Follower]" leader=
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.131Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: entering follower state: follower="Node at 9c603e47-dcb5-0e30-0afa-aaca8df23da9 [Follower]" leader=
testLoggerAdapter verbose: server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: 2024-01-08T00:30:59.131Z [INFO]  server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 Address:9c603e47-dcb5-0e30-0afa-aaca8df23da9} {Suffrage:Voter ID:server-aef27578-0e68-b160-b9b7-f17b6fd6c17e Address:aef27578-0e68-b160-b9b7-f17b6fd6c17e} {Suffrage:Voter ID:server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 Address:4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}]"
    testing.go:149: server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: 2024-01-08T00:30:59.131Z [INFO]  server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 Address:9c603e47-dcb5-0e30-0afa-aaca8df23da9} {Suffrage:Voter ID:server-aef27578-0e68-b160-b9b7-f17b6fd6c17e Address:aef27578-0e68-b160-b9b7-f17b6fd6c17e} {Suffrage:Voter ID:server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 Address:4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}]"
testLoggerAdapter verbose: server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: 2024-01-08T00:30:59.131Z [INFO]  server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: entering follower state: follower="Node at aef27578-0e68-b160-b9b7-f17b6fd6c17e [Follower]" leader=
    testing.go:149: server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: 2024-01-08T00:30:59.131Z [INFO]  server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: entering follower state: follower="Node at aef27578-0e68-b160-b9b7-f17b6fd6c17e [Follower]" leader=
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.131Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 Address:9c603e47-dcb5-0e30-0afa-aaca8df23da9} {Suffrage:Voter ID:server-aef27578-0e68-b160-b9b7-f17b6fd6c17e Address:aef27578-0e68-b160-b9b7-f17b6fd6c17e} {Suffrage:Voter ID:server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 Address:4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}]"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.131Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 Address:9c603e47-dcb5-0e30-0afa-aaca8df23da9} {Suffrage:Voter ID:server-aef27578-0e68-b160-b9b7-f17b6fd6c17e Address:aef27578-0e68-b160-b9b7-f17b6fd6c17e} {Suffrage:Voter ID:server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 Address:4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:30:59.132Z [INFO]  starting stability test: raft-state=Follower
    testing.go:149: cluster: 2024-01-08T00:30:59.132Z [INFO]  starting stability test: raft-state=Follower
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.132Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: entering follower state: follower="Node at 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 [Follower]" leader=
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.132Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: entering follower state: follower="Node at 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 [Follower]" leader=
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.197Z [WARN]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.197Z [WARN]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.197Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: entering candidate state: node="Node at 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 [Candidate]" term=2
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.197Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: entering candidate state: node="Node at 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 [Candidate]" term=2
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.198Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: election won: tally=2
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.198Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: election won: tally=2
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.198Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: entering leader state: leader="Node at 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 [Leader]"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.198Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: entering leader state: leader="Node at 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 [Leader]"
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.198Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: added peer, starting replication: peer=server-9c603e47-dcb5-0e30-0afa-aaca8df23da9
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.198Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: added peer, starting replication: peer=server-9c603e47-dcb5-0e30-0afa-aaca8df23da9
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.198Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: added peer, starting replication: peer=server-aef27578-0e68-b160-b9b7-f17b6fd6c17e
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.198Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: added peer, starting replication: peer=server-aef27578-0e68-b160-b9b7-f17b6fd6c17e
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.198Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: pipelining replication: peer="{Voter server-aef27578-0e68-b160-b9b7-f17b6fd6c17e aef27578-0e68-b160-b9b7-f17b6fd6c17e}"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.198Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: pipelining replication: peer="{Voter server-aef27578-0e68-b160-b9b7-f17b6fd6c17e aef27578-0e68-b160-b9b7-f17b6fd6c17e}"
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.198Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: pipelining replication: peer="{Voter server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 9c603e47-dcb5-0e30-0afa-aaca8df23da9}"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.198Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: pipelining replication: peer="{Voter server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 9c603e47-dcb5-0e30-0afa-aaca8df23da9}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:30:59.303Z [INFO]  stable state for Follower reached at 2024-01-08 00:30:59.198212299 +0000 UTC m=+5.245101418 (2 nodes), 65.971574ms from start of poll, 67.4763ms from cluster start. Timeout at 2024-01-08 00:30:59.303843107 +0000 UTC m=+5.350732154, 105.630736ms after stability
    testing.go:149: cluster: 2024-01-08T00:30:59.303Z [INFO]  stable state for Follower reached at 2024-01-08 00:30:59.198212299 +0000 UTC m=+5.245101418 (2 nodes), 65.971574ms from start of poll, 67.4763ms from cluster start. Timeout at 2024-01-08 00:30:59.303843107 +0000 UTC m=+5.350732154, 105.630736ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:30:59.304Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:30:59.304Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: cluster: 2024-01-08T00:30:59.409Z [INFO]  stable state for Leader reached at 2024-01-08 00:30:59.304038833 +0000 UTC m=+5.350927877 (1 nodes), 2.002µs from start of poll, 173.302759ms from cluster start. Timeout at 2024-01-08 00:30:59.409050697 +0000 UTC m=+5.455939736, 105.011859ms after stability
    testing.go:149: cluster: 2024-01-08T00:30:59.409Z [INFO]  stable state for Leader reached at 2024-01-08 00:30:59.304038833 +0000 UTC m=+5.350927877 (1 nodes), 2.002µs from start of poll, 173.302759ms from cluster start. Timeout at 2024-01-08 00:30:59.409050697 +0000 UTC m=+5.455939736, 105.011859ms after stability
    raft_test.go:336: [INFO] Disconnecting Node at 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 [Leader]
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.422Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to pipeline appendEntries: peer="{Voter server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 9c603e47-dcb5-0e30-0afa-aaca8df23da9}" error="append pipeline closed"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.422Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to pipeline appendEntries: peer="{Voter server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 9c603e47-dcb5-0e30-0afa-aaca8df23da9}" error="append pipeline closed"
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.422Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: aborting pipeline replication: peer="{Voter server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 9c603e47-dcb5-0e30-0afa-aaca8df23da9}"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.422Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: aborting pipeline replication: peer="{Voter server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 9c603e47-dcb5-0e30-0afa-aaca8df23da9}"
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.423Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to heartbeat to: peer=9c603e47-dcb5-0e30-0afa-aaca8df23da9 error="failed to connect to peer: 9c603e47-dcb5-0e30-0afa-aaca8df23da9"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.423Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to heartbeat to: peer=9c603e47-dcb5-0e30-0afa-aaca8df23da9 error="failed to connect to peer: 9c603e47-dcb5-0e30-0afa-aaca8df23da9"
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.426Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to heartbeat to: peer=aef27578-0e68-b160-b9b7-f17b6fd6c17e error="failed to connect to peer: aef27578-0e68-b160-b9b7-f17b6fd6c17e"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.426Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to heartbeat to: peer=aef27578-0e68-b160-b9b7-f17b6fd6c17e error="failed to connect to peer: aef27578-0e68-b160-b9b7-f17b6fd6c17e"
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.426Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to pipeline appendEntries: peer="{Voter server-aef27578-0e68-b160-b9b7-f17b6fd6c17e aef27578-0e68-b160-b9b7-f17b6fd6c17e}" error="append pipeline closed"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.426Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to pipeline appendEntries: peer="{Voter server-aef27578-0e68-b160-b9b7-f17b6fd6c17e aef27578-0e68-b160-b9b7-f17b6fd6c17e}" error="append pipeline closed"
testLoggerAdapter verbose: cluster: 2024-01-08T00:30:59.426Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.426Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: aborting pipeline replication: peer="{Voter server-aef27578-0e68-b160-b9b7-f17b6fd6c17e aef27578-0e68-b160-b9b7-f17b6fd6c17e}"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.426Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: aborting pipeline replication: peer="{Voter server-aef27578-0e68-b160-b9b7-f17b6fd6c17e aef27578-0e68-b160-b9b7-f17b6fd6c17e}"
    testing.go:149: cluster: 2024-01-08T00:30:59.426Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.431Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to appendEntries to: peer="{Voter server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 9c603e47-dcb5-0e30-0afa-aaca8df23da9}" error="failed to connect to peer: 9c603e47-dcb5-0e30-0afa-aaca8df23da9"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.431Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to appendEntries to: peer="{Voter server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 9c603e47-dcb5-0e30-0afa-aaca8df23da9}" error="failed to connect to peer: 9c603e47-dcb5-0e30-0afa-aaca8df23da9"
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.433Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to appendEntries to: peer="{Voter server-aef27578-0e68-b160-b9b7-f17b6fd6c17e aef27578-0e68-b160-b9b7-f17b6fd6c17e}" error="failed to connect to peer: aef27578-0e68-b160-b9b7-f17b6fd6c17e"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.433Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to appendEntries to: peer="{Voter server-aef27578-0e68-b160-b9b7-f17b6fd6c17e aef27578-0e68-b160-b9b7-f17b6fd6c17e}" error="failed to connect to peer: aef27578-0e68-b160-b9b7-f17b6fd6c17e"
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.442Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to heartbeat to: peer=9c603e47-dcb5-0e30-0afa-aaca8df23da9 error="failed to connect to peer: 9c603e47-dcb5-0e30-0afa-aaca8df23da9"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.442Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to heartbeat to: peer=9c603e47-dcb5-0e30-0afa-aaca8df23da9 error="failed to connect to peer: 9c603e47-dcb5-0e30-0afa-aaca8df23da9"
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.446Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to heartbeat to: peer=aef27578-0e68-b160-b9b7-f17b6fd6c17e error="failed to connect to peer: aef27578-0e68-b160-b9b7-f17b6fd6c17e"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.446Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to heartbeat to: peer=aef27578-0e68-b160-b9b7-f17b6fd6c17e error="failed to connect to peer: aef27578-0e68-b160-b9b7-f17b6fd6c17e"
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.449Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to appendEntries to: peer="{Voter server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 9c603e47-dcb5-0e30-0afa-aaca8df23da9}" error="failed to connect to peer: 9c603e47-dcb5-0e30-0afa-aaca8df23da9"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.449Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to appendEntries to: peer="{Voter server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 9c603e47-dcb5-0e30-0afa-aaca8df23da9}" error="failed to connect to peer: 9c603e47-dcb5-0e30-0afa-aaca8df23da9"
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.454Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to appendEntries to: peer="{Voter server-aef27578-0e68-b160-b9b7-f17b6fd6c17e aef27578-0e68-b160-b9b7-f17b6fd6c17e}" error="failed to connect to peer: aef27578-0e68-b160-b9b7-f17b6fd6c17e"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.454Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to appendEntries to: peer="{Voter server-aef27578-0e68-b160-b9b7-f17b6fd6c17e aef27578-0e68-b160-b9b7-f17b6fd6c17e}" error="failed to connect to peer: aef27578-0e68-b160-b9b7-f17b6fd6c17e"
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.460Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to heartbeat to: peer=9c603e47-dcb5-0e30-0afa-aaca8df23da9 error="failed to connect to peer: 9c603e47-dcb5-0e30-0afa-aaca8df23da9"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.460Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to heartbeat to: peer=9c603e47-dcb5-0e30-0afa-aaca8df23da9 error="failed to connect to peer: 9c603e47-dcb5-0e30-0afa-aaca8df23da9"
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.466Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to heartbeat to: peer=aef27578-0e68-b160-b9b7-f17b6fd6c17e error="failed to connect to peer: aef27578-0e68-b160-b9b7-f17b6fd6c17e"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.466Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to heartbeat to: peer=aef27578-0e68-b160-b9b7-f17b6fd6c17e error="failed to connect to peer: aef27578-0e68-b160-b9b7-f17b6fd6c17e"
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.467Z [WARN]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to contact: server-id=server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 time=50.364282ms
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.467Z [WARN]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to contact: server-id=server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 time=50.364282ms
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.467Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to appendEntries to: peer="{Voter server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 9c603e47-dcb5-0e30-0afa-aaca8df23da9}" error="failed to connect to peer: 9c603e47-dcb5-0e30-0afa-aaca8df23da9"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.467Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to appendEntries to: peer="{Voter server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 9c603e47-dcb5-0e30-0afa-aaca8df23da9}" error="failed to connect to peer: 9c603e47-dcb5-0e30-0afa-aaca8df23da9"
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.474Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to appendEntries to: peer="{Voter server-aef27578-0e68-b160-b9b7-f17b6fd6c17e aef27578-0e68-b160-b9b7-f17b6fd6c17e}" error="failed to connect to peer: aef27578-0e68-b160-b9b7-f17b6fd6c17e"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.474Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to appendEntries to: peer="{Voter server-aef27578-0e68-b160-b9b7-f17b6fd6c17e aef27578-0e68-b160-b9b7-f17b6fd6c17e}" error="failed to connect to peer: aef27578-0e68-b160-b9b7-f17b6fd6c17e"
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.478Z [WARN]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to contact: server-id=server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 time=61.161791ms
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.478Z [WARN]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to contact: server-id=server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 time=61.161791ms
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.478Z [WARN]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to contact: server-id=server-aef27578-0e68-b160-b9b7-f17b6fd6c17e time=59.788984ms
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.478Z [WARN]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to contact: server-id=server-aef27578-0e68-b160-b9b7-f17b6fd6c17e time=59.788984ms
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.478Z [WARN]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to contact quorum of nodes, stepping down
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.478Z [WARN]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to contact quorum of nodes, stepping down
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.478Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: entering follower state: follower="Node at 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 [Follower]" leader=
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.478Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: entering follower state: follower="Node at 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 [Follower]" leader=
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.490Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to heartbeat to: peer=9c603e47-dcb5-0e30-0afa-aaca8df23da9 error="failed to connect to peer: 9c603e47-dcb5-0e30-0afa-aaca8df23da9"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.490Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to heartbeat to: peer=9c603e47-dcb5-0e30-0afa-aaca8df23da9 error="failed to connect to peer: 9c603e47-dcb5-0e30-0afa-aaca8df23da9"
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.494Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to appendEntries to: peer="{Voter server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 9c603e47-dcb5-0e30-0afa-aaca8df23da9}" error="failed to connect to peer: 9c603e47-dcb5-0e30-0afa-aaca8df23da9"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.494Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to appendEntries to: peer="{Voter server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 9c603e47-dcb5-0e30-0afa-aaca8df23da9}" error="failed to connect to peer: 9c603e47-dcb5-0e30-0afa-aaca8df23da9"
testLoggerAdapter verbose: cluster: 2024-01-08T00:30:59.531Z [INFO]  stable state for Leader reached at 2024-01-08 00:30:59.426538546 +0000 UTC m=+5.473427672 (1 nodes), 2.382µs from start of poll, 295.802554ms from cluster start. Timeout at 2024-01-08 00:30:59.531796261 +0000 UTC m=+5.578685296, 105.257624ms after stability
    testing.go:149: cluster: 2024-01-08T00:30:59.531Z [INFO]  stable state for Leader reached at 2024-01-08 00:30:59.426538546 +0000 UTC m=+5.473427672 (1 nodes), 2.382µs from start of poll, 295.802554ms from cluster start. Timeout at 2024-01-08 00:30:59.531796261 +0000 UTC m=+5.578685296, 105.257624ms after stability
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.537Z [WARN]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: heartbeat timeout reached, starting election: last-leader=4ff51c45-01a1-521b-0e7f-4a59ba7c74c9
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.537Z [WARN]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: heartbeat timeout reached, starting election: last-leader=4ff51c45-01a1-521b-0e7f-4a59ba7c74c9
testLoggerAdapter verbose: cluster: 2024-01-08T00:30:59.537Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.537Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: entering candidate state: node="Node at 9c603e47-dcb5-0e30-0afa-aaca8df23da9 [Candidate]" term=3
    testing.go:149: cluster: 2024-01-08T00:30:59.537Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.537Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: entering candidate state: node="Node at 9c603e47-dcb5-0e30-0afa-aaca8df23da9 [Candidate]" term=3
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.537Z [ERROR] server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: failed to make requestVote RPC: target="{Voter server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}" error="failed to connect to peer: 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9"
testLoggerAdapter verbose: server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: 2024-01-08T00:30:59.537Z [WARN]  server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: rejecting vote request since we have a leader: from=9c603e47-dcb5-0e30-0afa-aaca8df23da9 leader=4ff51c45-01a1-521b-0e7f-4a59ba7c74c9
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.537Z [ERROR] server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: failed to make requestVote RPC: target="{Voter server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}" error="failed to connect to peer: 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9"
    testing.go:149: server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: 2024-01-08T00:30:59.537Z [WARN]  server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: rejecting vote request since we have a leader: from=9c603e47-dcb5-0e30-0afa-aaca8df23da9 leader=4ff51c45-01a1-521b-0e7f-4a59ba7c74c9
testLoggerAdapter verbose: server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: 2024-01-08T00:30:59.545Z [WARN]  server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: heartbeat timeout reached, starting election: last-leader=4ff51c45-01a1-521b-0e7f-4a59ba7c74c9
    testing.go:149: server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: 2024-01-08T00:30:59.545Z [WARN]  server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: heartbeat timeout reached, starting election: last-leader=4ff51c45-01a1-521b-0e7f-4a59ba7c74c9
testLoggerAdapter verbose: server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: 2024-01-08T00:30:59.545Z [INFO]  server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: entering candidate state: node="Node at aef27578-0e68-b160-b9b7-f17b6fd6c17e [Candidate]" term=3
    testing.go:149: server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: 2024-01-08T00:30:59.545Z [INFO]  server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: entering candidate state: node="Node at aef27578-0e68-b160-b9b7-f17b6fd6c17e [Candidate]" term=3
testLoggerAdapter verbose: server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: 2024-01-08T00:30:59.545Z [ERROR] server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: failed to make requestVote RPC: target="{Voter server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}" error="failed to connect to peer: 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9"
    testing.go:149: server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: 2024-01-08T00:30:59.545Z [ERROR] server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: failed to make requestVote RPC: target="{Voter server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}" error="failed to connect to peer: 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9"
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.545Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: duplicate requestVote for same term: term=3
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.545Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: duplicate requestVote for same term: term=3
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.572Z [WARN]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.572Z [WARN]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.572Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: entering candidate state: node="Node at 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 [Candidate]" term=3
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.572Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: entering candidate state: node="Node at 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 [Candidate]" term=3
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.572Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to make requestVote RPC: target="{Voter server-aef27578-0e68-b160-b9b7-f17b6fd6c17e aef27578-0e68-b160-b9b7-f17b6fd6c17e}" error="failed to connect to peer: aef27578-0e68-b160-b9b7-f17b6fd6c17e"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.572Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to make requestVote RPC: target="{Voter server-aef27578-0e68-b160-b9b7-f17b6fd6c17e aef27578-0e68-b160-b9b7-f17b6fd6c17e}" error="failed to connect to peer: aef27578-0e68-b160-b9b7-f17b6fd6c17e"
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.572Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to make requestVote RPC: target="{Voter server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 9c603e47-dcb5-0e30-0afa-aaca8df23da9}" error="failed to connect to peer: 9c603e47-dcb5-0e30-0afa-aaca8df23da9"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.572Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to make requestVote RPC: target="{Voter server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 9c603e47-dcb5-0e30-0afa-aaca8df23da9}" error="failed to connect to peer: 9c603e47-dcb5-0e30-0afa-aaca8df23da9"
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.612Z [WARN]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: Election timeout reached, restarting election
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.612Z [WARN]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: Election timeout reached, restarting election
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.612Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: entering candidate state: node="Node at 9c603e47-dcb5-0e30-0afa-aaca8df23da9 [Candidate]" term=4
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.612Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: entering candidate state: node="Node at 9c603e47-dcb5-0e30-0afa-aaca8df23da9 [Candidate]" term=4
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.612Z [ERROR] server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: failed to make requestVote RPC: target="{Voter server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}" error="failed to connect to peer: 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9"
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.612Z [ERROR] server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: failed to make requestVote RPC: target="{Voter server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}" error="failed to connect to peer: 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9"
testLoggerAdapter verbose: server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: 2024-01-08T00:30:59.612Z [INFO]  server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: entering follower state: follower="Node at aef27578-0e68-b160-b9b7-f17b6fd6c17e [Follower]" leader=
    testing.go:149: server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: 2024-01-08T00:30:59.612Z [INFO]  server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: entering follower state: follower="Node at aef27578-0e68-b160-b9b7-f17b6fd6c17e [Follower]" leader=
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.612Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: election won: tally=2
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.612Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: election won: tally=2
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.612Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: entering leader state: leader="Node at 9c603e47-dcb5-0e30-0afa-aaca8df23da9 [Leader]"
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.612Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: entering leader state: leader="Node at 9c603e47-dcb5-0e30-0afa-aaca8df23da9 [Leader]"
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.612Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: added peer, starting replication: peer=server-aef27578-0e68-b160-b9b7-f17b6fd6c17e
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.612Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: added peer, starting replication: peer=server-aef27578-0e68-b160-b9b7-f17b6fd6c17e
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.613Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: added peer, starting replication: peer=server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.613Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: added peer, starting replication: peer=server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.613Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: pipelining replication: peer="{Voter server-aef27578-0e68-b160-b9b7-f17b6fd6c17e aef27578-0e68-b160-b9b7-f17b6fd6c17e}"
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.613Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: pipelining replication: peer="{Voter server-aef27578-0e68-b160-b9b7-f17b6fd6c17e aef27578-0e68-b160-b9b7-f17b6fd6c17e}"
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.613Z [ERROR] server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: failed to appendEntries to: peer="{Voter server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}" error="failed to connect to peer: 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9"
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.613Z [ERROR] server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: failed to appendEntries to: peer="{Voter server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}" error="failed to connect to peer: 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9"
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.621Z [ERROR] server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: failed to heartbeat to: peer=4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 error="failed to connect to peer: 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9"
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.621Z [ERROR] server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: failed to heartbeat to: peer=4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 error="failed to connect to peer: 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9"
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.633Z [ERROR] server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: failed to appendEntries to: peer="{Voter server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}" error="failed to connect to peer: 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9"
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.633Z [ERROR] server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: failed to appendEntries to: peer="{Voter server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}" error="failed to connect to peer: 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9"
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.639Z [ERROR] server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: failed to heartbeat to: peer=4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 error="failed to connect to peer: 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9"
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.639Z [ERROR] server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: failed to heartbeat to: peer=4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 error="failed to connect to peer: 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9"
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.653Z [ERROR] server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: failed to appendEntries to: peer="{Voter server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}" error="failed to connect to peer: 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9"
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.653Z [ERROR] server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: failed to appendEntries to: peer="{Voter server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}" error="failed to connect to peer: 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9"
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.660Z [ERROR] server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: failed to heartbeat to: peer=4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 error="failed to connect to peer: 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9"
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.660Z [ERROR] server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: failed to heartbeat to: peer=4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 error="failed to connect to peer: 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9"
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.663Z [WARN]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: failed to contact: server-id=server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 time=50.094154ms
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.663Z [WARN]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: failed to contact: server-id=server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 time=50.094154ms
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.668Z [WARN]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: Election timeout reached, restarting election
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.668Z [WARN]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: Election timeout reached, restarting election
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.668Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: entering candidate state: node="Node at 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 [Candidate]" term=4
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.668Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: entering candidate state: node="Node at 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 [Candidate]" term=4
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.668Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to make requestVote RPC: target="{Voter server-aef27578-0e68-b160-b9b7-f17b6fd6c17e aef27578-0e68-b160-b9b7-f17b6fd6c17e}" error="failed to connect to peer: aef27578-0e68-b160-b9b7-f17b6fd6c17e"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.668Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to make requestVote RPC: target="{Voter server-aef27578-0e68-b160-b9b7-f17b6fd6c17e aef27578-0e68-b160-b9b7-f17b6fd6c17e}" error="failed to connect to peer: aef27578-0e68-b160-b9b7-f17b6fd6c17e"
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.668Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to make requestVote RPC: target="{Voter server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 9c603e47-dcb5-0e30-0afa-aaca8df23da9}" error="failed to connect to peer: 9c603e47-dcb5-0e30-0afa-aaca8df23da9"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.668Z [ERROR] server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to make requestVote RPC: target="{Voter server-9c603e47-dcb5-0e30-0afa-aaca8df23da9 9c603e47-dcb5-0e30-0afa-aaca8df23da9}" error="failed to connect to peer: 9c603e47-dcb5-0e30-0afa-aaca8df23da9"
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.679Z [ERROR] server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: failed to appendEntries to: peer="{Voter server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}" error="failed to connect to peer: 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9"
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.679Z [ERROR] server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: failed to appendEntries to: peer="{Voter server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}" error="failed to connect to peer: 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9"
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.691Z [ERROR] server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: failed to heartbeat to: peer=4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 error="failed to connect to peer: 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9"
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.691Z [ERROR] server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: failed to heartbeat to: peer=4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 error="failed to connect to peer: 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9"
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.712Z [WARN]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: failed to contact: server-id=server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 time=99.58262ms
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.712Z [WARN]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: failed to contact: server-id=server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 time=99.58262ms
testLoggerAdapter verbose: cluster: 2024-01-08T00:30:59.718Z [INFO]  stable state for Leader reached at 2024-01-08 00:30:59.612988044 +0000 UTC m=+5.659877252 (1 nodes), 75.757545ms from start of poll, 482.252134ms from cluster start. Timeout at 2024-01-08 00:30:59.718179924 +0000 UTC m=+5.765069007, 105.191755ms after stability
    testing.go:149: cluster: 2024-01-08T00:30:59.718Z [INFO]  stable state for Leader reached at 2024-01-08 00:30:59.612988044 +0000 UTC m=+5.659877252 (1 nodes), 75.757545ms from start of poll, 482.252134ms from cluster start. Timeout at 2024-01-08 00:30:59.718179924 +0000 UTC m=+5.765069007, 105.191755ms after stability
    raft_test.go:371: [INFO] Reconnecting Node at 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 [Candidate]
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.724Z [WARN]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: Election timeout reached, restarting election
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.724Z [WARN]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: Election timeout reached, restarting election
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.725Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: entering candidate state: node="Node at 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 [Candidate]" term=5
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.725Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: entering candidate state: node="Node at 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 [Candidate]" term=5
testLoggerAdapter verbose: server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: 2024-01-08T00:30:59.725Z [WARN]  server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: rejecting vote request since we have a leader: from=4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 leader=9c603e47-dcb5-0e30-0afa-aaca8df23da9
    testing.go:149: server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: 2024-01-08T00:30:59.725Z [WARN]  server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: rejecting vote request since we have a leader: from=4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 leader=9c603e47-dcb5-0e30-0afa-aaca8df23da9
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.725Z [WARN]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: rejecting vote request since we have a leader: from=4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 leader=9c603e47-dcb5-0e30-0afa-aaca8df23da9
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.725Z [WARN]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: rejecting vote request since we have a leader: from=4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 leader=9c603e47-dcb5-0e30-0afa-aaca8df23da9
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.729Z [ERROR] server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: peer has newer term, stopping replication: peer="{Voter server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}"
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.729Z [ERROR] server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: peer has newer term, stopping replication: peer="{Voter server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}"
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.729Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: entering follower state: follower="Node at 9c603e47-dcb5-0e30-0afa-aaca8df23da9 [Follower]" leader=
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.729Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: entering follower state: follower="Node at 9c603e47-dcb5-0e30-0afa-aaca8df23da9 [Follower]" leader=
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.729Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: aborting pipeline replication: peer="{Voter server-aef27578-0e68-b160-b9b7-f17b6fd6c17e aef27578-0e68-b160-b9b7-f17b6fd6c17e}"
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.729Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: aborting pipeline replication: peer="{Voter server-aef27578-0e68-b160-b9b7-f17b6fd6c17e aef27578-0e68-b160-b9b7-f17b6fd6c17e}"
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.776Z [WARN]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: Election timeout reached, restarting election
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.776Z [WARN]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: Election timeout reached, restarting election
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.776Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: entering candidate state: node="Node at 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 [Candidate]" term=6
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.776Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: entering candidate state: node="Node at 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 [Candidate]" term=6
testLoggerAdapter verbose: server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: 2024-01-08T00:30:59.776Z [WARN]  server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: rejecting vote request since we have a leader: from=4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 leader=9c603e47-dcb5-0e30-0afa-aaca8df23da9
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.776Z [WARN]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: rejecting vote request since our last term is greater: candidate=4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 last-term=4 last-candidate-term=2
    testing.go:149: server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: 2024-01-08T00:30:59.776Z [WARN]  server-aef27578-0e68-b160-b9b7-f17b6fd6c17e: rejecting vote request since we have a leader: from=4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 leader=9c603e47-dcb5-0e30-0afa-aaca8df23da9
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.776Z [WARN]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: rejecting vote request since our last term is greater: candidate=4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 last-term=4 last-candidate-term=2
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.781Z [WARN]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.781Z [WARN]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.781Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: entering candidate state: node="Node at 9c603e47-dcb5-0e30-0afa-aaca8df23da9 [Candidate]" term=7
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.781Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: entering candidate state: node="Node at 9c603e47-dcb5-0e30-0afa-aaca8df23da9 [Candidate]" term=7
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.782Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: entering follower state: follower="Node at 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 [Follower]" leader=
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.782Z [INFO]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: entering follower state: follower="Node at 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 [Follower]" leader=
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.782Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: election won: tally=2
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.782Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: election won: tally=2
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.782Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: entering leader state: leader="Node at 9c603e47-dcb5-0e30-0afa-aaca8df23da9 [Leader]"
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.782Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: entering leader state: leader="Node at 9c603e47-dcb5-0e30-0afa-aaca8df23da9 [Leader]"
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.782Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: added peer, starting replication: peer=server-aef27578-0e68-b160-b9b7-f17b6fd6c17e
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.782Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: added peer, starting replication: peer=server-aef27578-0e68-b160-b9b7-f17b6fd6c17e
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.782Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: added peer, starting replication: peer=server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.782Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: added peer, starting replication: peer=server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9
testLoggerAdapter verbose: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.782Z [WARN]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to get previous log: previous-index=5 last-index=3 error="log not found"
    testing.go:149: server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: 2024-01-08T00:30:59.782Z [WARN]  server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9: failed to get previous log: previous-index=5 last-index=3 error="log not found"
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.782Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: pipelining replication: peer="{Voter server-aef27578-0e68-b160-b9b7-f17b6fd6c17e aef27578-0e68-b160-b9b7-f17b6fd6c17e}"
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.782Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: pipelining replication: peer="{Voter server-aef27578-0e68-b160-b9b7-f17b6fd6c17e aef27578-0e68-b160-b9b7-f17b6fd6c17e}"
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.782Z [WARN]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: appendEntries rejected, sending older logs: peer="{Voter server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}" next=4
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.782Z [WARN]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: appendEntries rejected, sending older logs: peer="{Voter server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}" next=4
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.782Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: pipelining replication: peer="{Voter server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}"
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.782Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: pipelining replication: peer="{Voter server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}"
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.788Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: aborting pipeline replication: peer="{Voter server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}"
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.788Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: aborting pipeline replication: peer="{Voter server-4ff51c45-01a1-521b-0e7f-4a59ba7c74c9 4ff51c45-01a1-521b-0e7f-4a59ba7c74c9}"
testLoggerAdapter verbose: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.788Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: aborting pipeline replication: peer="{Voter server-aef27578-0e68-b160-b9b7-f17b6fd6c17e aef27578-0e68-b160-b9b7-f17b6fd6c17e}"
    testing.go:149: server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: 2024-01-08T00:30:59.788Z [INFO]  server-9c603e47-dcb5-0e30-0afa-aaca8df23da9: aborting pipeline replication: peer="{Voter server-aef27578-0e68-b160-b9b7-f17b6fd6c17e aef27578-0e68-b160-b9b7-f17b6fd6c17e}"
--- PASS: TestRaft_LeaderFail (0.66s)
=== RUN   TestRaft_BehindFollower
testLoggerAdapter verbose: server-3d691ad1-3554-8502-a80d-ba74a8ccad96: 2024-01-08T00:30:59.789Z [INFO]  server-3d691ad1-3554-8502-a80d-ba74a8ccad96: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3d691ad1-3554-8502-a80d-ba74a8ccad96 Address:3d691ad1-3554-8502-a80d-ba74a8ccad96} {Suffrage:Voter ID:server-2332666b-cbed-2be3-6042-5e64ba1cd321 Address:2332666b-cbed-2be3-6042-5e64ba1cd321} {Suffrage:Voter ID:server-06746be3-e087-5413-084f-0a85eafee938 Address:06746be3-e087-5413-084f-0a85eafee938}]"
    testing.go:149: server-3d691ad1-3554-8502-a80d-ba74a8ccad96: 2024-01-08T00:30:59.789Z [INFO]  server-3d691ad1-3554-8502-a80d-ba74a8ccad96: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3d691ad1-3554-8502-a80d-ba74a8ccad96 Address:3d691ad1-3554-8502-a80d-ba74a8ccad96} {Suffrage:Voter ID:server-2332666b-cbed-2be3-6042-5e64ba1cd321 Address:2332666b-cbed-2be3-6042-5e64ba1cd321} {Suffrage:Voter ID:server-06746be3-e087-5413-084f-0a85eafee938 Address:06746be3-e087-5413-084f-0a85eafee938}]"
testLoggerAdapter verbose: server-3d691ad1-3554-8502-a80d-ba74a8ccad96: 2024-01-08T00:30:59.789Z [INFO]  server-3d691ad1-3554-8502-a80d-ba74a8ccad96: entering follower state: follower="Node at 3d691ad1-3554-8502-a80d-ba74a8ccad96 [Follower]" leader=
testLoggerAdapter verbose: server-2332666b-cbed-2be3-6042-5e64ba1cd321: 2024-01-08T00:30:59.789Z [INFO]  server-2332666b-cbed-2be3-6042-5e64ba1cd321: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3d691ad1-3554-8502-a80d-ba74a8ccad96 Address:3d691ad1-3554-8502-a80d-ba74a8ccad96} {Suffrage:Voter ID:server-2332666b-cbed-2be3-6042-5e64ba1cd321 Address:2332666b-cbed-2be3-6042-5e64ba1cd321} {Suffrage:Voter ID:server-06746be3-e087-5413-084f-0a85eafee938 Address:06746be3-e087-5413-084f-0a85eafee938}]"
    testing.go:149: server-3d691ad1-3554-8502-a80d-ba74a8ccad96: 2024-01-08T00:30:59.789Z [INFO]  server-3d691ad1-3554-8502-a80d-ba74a8ccad96: entering follower state: follower="Node at 3d691ad1-3554-8502-a80d-ba74a8ccad96 [Follower]" leader=
    testing.go:149: server-2332666b-cbed-2be3-6042-5e64ba1cd321: 2024-01-08T00:30:59.789Z [INFO]  server-2332666b-cbed-2be3-6042-5e64ba1cd321: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3d691ad1-3554-8502-a80d-ba74a8ccad96 Address:3d691ad1-3554-8502-a80d-ba74a8ccad96} {Suffrage:Voter ID:server-2332666b-cbed-2be3-6042-5e64ba1cd321 Address:2332666b-cbed-2be3-6042-5e64ba1cd321} {Suffrage:Voter ID:server-06746be3-e087-5413-084f-0a85eafee938 Address:06746be3-e087-5413-084f-0a85eafee938}]"
testLoggerAdapter verbose: server-2332666b-cbed-2be3-6042-5e64ba1cd321: 2024-01-08T00:30:59.789Z [INFO]  server-2332666b-cbed-2be3-6042-5e64ba1cd321: entering follower state: follower="Node at 2332666b-cbed-2be3-6042-5e64ba1cd321 [Follower]" leader=
testLoggerAdapter verbose: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:30:59.789Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3d691ad1-3554-8502-a80d-ba74a8ccad96 Address:3d691ad1-3554-8502-a80d-ba74a8ccad96} {Suffrage:Voter ID:server-2332666b-cbed-2be3-6042-5e64ba1cd321 Address:2332666b-cbed-2be3-6042-5e64ba1cd321} {Suffrage:Voter ID:server-06746be3-e087-5413-084f-0a85eafee938 Address:06746be3-e087-5413-084f-0a85eafee938}]"
    testing.go:149: server-2332666b-cbed-2be3-6042-5e64ba1cd321: 2024-01-08T00:30:59.789Z [INFO]  server-2332666b-cbed-2be3-6042-5e64ba1cd321: entering follower state: follower="Node at 2332666b-cbed-2be3-6042-5e64ba1cd321 [Follower]" leader=
    testing.go:149: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:30:59.789Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3d691ad1-3554-8502-a80d-ba74a8ccad96 Address:3d691ad1-3554-8502-a80d-ba74a8ccad96} {Suffrage:Voter ID:server-2332666b-cbed-2be3-6042-5e64ba1cd321 Address:2332666b-cbed-2be3-6042-5e64ba1cd321} {Suffrage:Voter ID:server-06746be3-e087-5413-084f-0a85eafee938 Address:06746be3-e087-5413-084f-0a85eafee938}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:30:59.789Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:30:59.789Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:30:59.789Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: entering follower state: follower="Node at 06746be3-e087-5413-084f-0a85eafee938 [Follower]" leader=
    testing.go:149: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:30:59.789Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: entering follower state: follower="Node at 06746be3-e087-5413-084f-0a85eafee938 [Follower]" leader=
testLoggerAdapter verbose: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:30:59.867Z [WARN]  server-06746be3-e087-5413-084f-0a85eafee938: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:30:59.867Z [WARN]  server-06746be3-e087-5413-084f-0a85eafee938: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:30:59.867Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: entering candidate state: node="Node at 06746be3-e087-5413-084f-0a85eafee938 [Candidate]" term=2
    testing.go:149: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:30:59.867Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: entering candidate state: node="Node at 06746be3-e087-5413-084f-0a85eafee938 [Candidate]" term=2
testLoggerAdapter verbose: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:30:59.867Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: election won: tally=2
    testing.go:149: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:30:59.867Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: election won: tally=2
testLoggerAdapter verbose: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:30:59.867Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: entering leader state: leader="Node at 06746be3-e087-5413-084f-0a85eafee938 [Leader]"
    testing.go:149: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:30:59.867Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: entering leader state: leader="Node at 06746be3-e087-5413-084f-0a85eafee938 [Leader]"
testLoggerAdapter verbose: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:30:59.867Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: added peer, starting replication: peer=server-3d691ad1-3554-8502-a80d-ba74a8ccad96
    testing.go:149: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:30:59.867Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: added peer, starting replication: peer=server-3d691ad1-3554-8502-a80d-ba74a8ccad96
testLoggerAdapter verbose: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:30:59.867Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: added peer, starting replication: peer=server-2332666b-cbed-2be3-6042-5e64ba1cd321
    testing.go:149: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:30:59.867Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: added peer, starting replication: peer=server-2332666b-cbed-2be3-6042-5e64ba1cd321
testLoggerAdapter verbose: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:30:59.867Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: pipelining replication: peer="{Voter server-3d691ad1-3554-8502-a80d-ba74a8ccad96 3d691ad1-3554-8502-a80d-ba74a8ccad96}"
    testing.go:149: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:30:59.867Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: pipelining replication: peer="{Voter server-3d691ad1-3554-8502-a80d-ba74a8ccad96 3d691ad1-3554-8502-a80d-ba74a8ccad96}"
testLoggerAdapter verbose: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:30:59.867Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: pipelining replication: peer="{Voter server-2332666b-cbed-2be3-6042-5e64ba1cd321 2332666b-cbed-2be3-6042-5e64ba1cd321}"
    testing.go:149: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:30:59.867Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: pipelining replication: peer="{Voter server-2332666b-cbed-2be3-6042-5e64ba1cd321 2332666b-cbed-2be3-6042-5e64ba1cd321}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:30:59.972Z [INFO]  stable state for Leader reached at 2024-01-08 00:30:59.867712268 +0000 UTC m=+5.914601325 (1 nodes), 77.726084ms from start of poll, 78.310338ms from cluster start. Timeout at 2024-01-08 00:30:59.97287267 +0000 UTC m=+6.019761714, 105.160389ms after stability
    testing.go:149: cluster: 2024-01-08T00:30:59.972Z [INFO]  stable state for Leader reached at 2024-01-08 00:30:59.867712268 +0000 UTC m=+5.914601325 (1 nodes), 77.726084ms from start of poll, 78.310338ms from cluster start. Timeout at 2024-01-08 00:30:59.97287267 +0000 UTC m=+6.019761714, 105.160389ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:30:59.972Z [INFO]  starting stability test: raft-state=Follower
    testing.go:149: cluster: 2024-01-08T00:30:59.972Z [INFO]  starting stability test: raft-state=Follower
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:00.078Z [INFO]  stable state for Follower reached at 2024-01-08 00:30:59.973000959 +0000 UTC m=+6.019890014 (2 nodes), 2.153µs from start of poll, 183.599027ms from cluster start. Timeout at 2024-01-08 00:31:00.078007259 +0000 UTC m=+6.124896322, 105.006308ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:00.078Z [INFO]  stable state for Follower reached at 2024-01-08 00:30:59.973000959 +0000 UTC m=+6.019890014 (2 nodes), 2.153µs from start of poll, 183.599027ms from cluster start. Timeout at 2024-01-08 00:31:00.078007259 +0000 UTC m=+6.124896322, 105.006308ms after stability
testLoggerAdapter verbose: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:31:00.078Z [ERROR] server-06746be3-e087-5413-084f-0a85eafee938: failed to pipeline appendEntries: peer="{Voter server-3d691ad1-3554-8502-a80d-ba74a8ccad96 3d691ad1-3554-8502-a80d-ba74a8ccad96}" error="append pipeline closed"
    testing.go:149: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:31:00.078Z [ERROR] server-06746be3-e087-5413-084f-0a85eafee938: failed to pipeline appendEntries: peer="{Voter server-3d691ad1-3554-8502-a80d-ba74a8ccad96 3d691ad1-3554-8502-a80d-ba74a8ccad96}" error="append pipeline closed"
testLoggerAdapter verbose: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:31:00.078Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: aborting pipeline replication: peer="{Voter server-3d691ad1-3554-8502-a80d-ba74a8ccad96 3d691ad1-3554-8502-a80d-ba74a8ccad96}"
    testing.go:149: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:31:00.078Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: aborting pipeline replication: peer="{Voter server-3d691ad1-3554-8502-a80d-ba74a8ccad96 3d691ad1-3554-8502-a80d-ba74a8ccad96}"
testLoggerAdapter verbose: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:31:00.078Z [ERROR] server-06746be3-e087-5413-084f-0a85eafee938: failed to appendEntries to: peer="{Voter server-3d691ad1-3554-8502-a80d-ba74a8ccad96 3d691ad1-3554-8502-a80d-ba74a8ccad96}" error="failed to connect to peer: 3d691ad1-3554-8502-a80d-ba74a8ccad96"
    testing.go:149: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:31:00.078Z [ERROR] server-06746be3-e087-5413-084f-0a85eafee938: failed to appendEntries to: peer="{Voter server-3d691ad1-3554-8502-a80d-ba74a8ccad96 3d691ad1-3554-8502-a80d-ba74a8ccad96}" error="failed to connect to peer: 3d691ad1-3554-8502-a80d-ba74a8ccad96"
    raft_test.go:420: [INFO] Finished apply without behind follower
testLoggerAdapter verbose: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:31:00.088Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: pipelining replication: peer="{Voter server-3d691ad1-3554-8502-a80d-ba74a8ccad96 3d691ad1-3554-8502-a80d-ba74a8ccad96}"
    testing.go:149: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:31:00.088Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: pipelining replication: peer="{Voter server-3d691ad1-3554-8502-a80d-ba74a8ccad96 3d691ad1-3554-8502-a80d-ba74a8ccad96}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:00.097Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:00.097Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:00.203Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:00.097714237 +0000 UTC m=+6.144603273 (1 nodes), 1.815µs from start of poll, 308.312286ms from cluster start. Timeout at 2024-01-08 00:31:00.203381996 +0000 UTC m=+6.250271061, 105.667788ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:00.203Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:00.097714237 +0000 UTC m=+6.144603273 (1 nodes), 1.815µs from start of poll, 308.312286ms from cluster start. Timeout at 2024-01-08 00:31:00.203381996 +0000 UTC m=+6.250271061, 105.667788ms after stability
testLoggerAdapter verbose: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:31:00.203Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: aborting pipeline replication: peer="{Voter server-2332666b-cbed-2be3-6042-5e64ba1cd321 2332666b-cbed-2be3-6042-5e64ba1cd321}"
    testing.go:149: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:31:00.203Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: aborting pipeline replication: peer="{Voter server-2332666b-cbed-2be3-6042-5e64ba1cd321 2332666b-cbed-2be3-6042-5e64ba1cd321}"
testLoggerAdapter verbose: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:31:00.203Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: aborting pipeline replication: peer="{Voter server-3d691ad1-3554-8502-a80d-ba74a8ccad96 3d691ad1-3554-8502-a80d-ba74a8ccad96}"
    testing.go:149: server-06746be3-e087-5413-084f-0a85eafee938: 2024-01-08T00:31:00.203Z [INFO]  server-06746be3-e087-5413-084f-0a85eafee938: aborting pipeline replication: peer="{Voter server-3d691ad1-3554-8502-a80d-ba74a8ccad96 3d691ad1-3554-8502-a80d-ba74a8ccad96}"
--- PASS: TestRaft_BehindFollower (0.42s)
=== RUN   TestRaft_ApplyNonLeader
testLoggerAdapter verbose: server-c94ae48a-40c5-cead-43b2-426a0d62a506: 2024-01-08T00:31:00.204Z [INFO]  server-c94ae48a-40c5-cead-43b2-426a0d62a506: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c94ae48a-40c5-cead-43b2-426a0d62a506 Address:c94ae48a-40c5-cead-43b2-426a0d62a506} {Suffrage:Voter ID:server-ee235919-0126-d33f-6a9a-fbd6e9bd2216 Address:ee235919-0126-d33f-6a9a-fbd6e9bd2216} {Suffrage:Voter ID:server-567da5c9-fcd6-028c-64c5-eac7045ef0b7 Address:567da5c9-fcd6-028c-64c5-eac7045ef0b7}]"
    testing.go:149: server-c94ae48a-40c5-cead-43b2-426a0d62a506: 2024-01-08T00:31:00.204Z [INFO]  server-c94ae48a-40c5-cead-43b2-426a0d62a506: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c94ae48a-40c5-cead-43b2-426a0d62a506 Address:c94ae48a-40c5-cead-43b2-426a0d62a506} {Suffrage:Voter ID:server-ee235919-0126-d33f-6a9a-fbd6e9bd2216 Address:ee235919-0126-d33f-6a9a-fbd6e9bd2216} {Suffrage:Voter ID:server-567da5c9-fcd6-028c-64c5-eac7045ef0b7 Address:567da5c9-fcd6-028c-64c5-eac7045ef0b7}]"
testLoggerAdapter verbose: server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: 2024-01-08T00:31:00.204Z [INFO]  server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c94ae48a-40c5-cead-43b2-426a0d62a506 Address:c94ae48a-40c5-cead-43b2-426a0d62a506} {Suffrage:Voter ID:server-ee235919-0126-d33f-6a9a-fbd6e9bd2216 Address:ee235919-0126-d33f-6a9a-fbd6e9bd2216} {Suffrage:Voter ID:server-567da5c9-fcd6-028c-64c5-eac7045ef0b7 Address:567da5c9-fcd6-028c-64c5-eac7045ef0b7}]"
    testing.go:149: server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: 2024-01-08T00:31:00.204Z [INFO]  server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c94ae48a-40c5-cead-43b2-426a0d62a506 Address:c94ae48a-40c5-cead-43b2-426a0d62a506} {Suffrage:Voter ID:server-ee235919-0126-d33f-6a9a-fbd6e9bd2216 Address:ee235919-0126-d33f-6a9a-fbd6e9bd2216} {Suffrage:Voter ID:server-567da5c9-fcd6-028c-64c5-eac7045ef0b7 Address:567da5c9-fcd6-028c-64c5-eac7045ef0b7}]"
testLoggerAdapter verbose: server-567da5c9-fcd6-028c-64c5-eac7045ef0b7: 2024-01-08T00:31:00.205Z [INFO]  server-567da5c9-fcd6-028c-64c5-eac7045ef0b7: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c94ae48a-40c5-cead-43b2-426a0d62a506 Address:c94ae48a-40c5-cead-43b2-426a0d62a506} {Suffrage:Voter ID:server-ee235919-0126-d33f-6a9a-fbd6e9bd2216 Address:ee235919-0126-d33f-6a9a-fbd6e9bd2216} {Suffrage:Voter ID:server-567da5c9-fcd6-028c-64c5-eac7045ef0b7 Address:567da5c9-fcd6-028c-64c5-eac7045ef0b7}]"
    testing.go:149: server-567da5c9-fcd6-028c-64c5-eac7045ef0b7: 2024-01-08T00:31:00.205Z [INFO]  server-567da5c9-fcd6-028c-64c5-eac7045ef0b7: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c94ae48a-40c5-cead-43b2-426a0d62a506 Address:c94ae48a-40c5-cead-43b2-426a0d62a506} {Suffrage:Voter ID:server-ee235919-0126-d33f-6a9a-fbd6e9bd2216 Address:ee235919-0126-d33f-6a9a-fbd6e9bd2216} {Suffrage:Voter ID:server-567da5c9-fcd6-028c-64c5-eac7045ef0b7 Address:567da5c9-fcd6-028c-64c5-eac7045ef0b7}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:00.205Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:00.205Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-c94ae48a-40c5-cead-43b2-426a0d62a506: 2024-01-08T00:31:00.205Z [INFO]  server-c94ae48a-40c5-cead-43b2-426a0d62a506: entering follower state: follower="Node at c94ae48a-40c5-cead-43b2-426a0d62a506 [Follower]" leader=
    testing.go:149: server-c94ae48a-40c5-cead-43b2-426a0d62a506: 2024-01-08T00:31:00.205Z [INFO]  server-c94ae48a-40c5-cead-43b2-426a0d62a506: entering follower state: follower="Node at c94ae48a-40c5-cead-43b2-426a0d62a506 [Follower]" leader=
testLoggerAdapter verbose: server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: 2024-01-08T00:31:00.205Z [INFO]  server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: entering follower state: follower="Node at ee235919-0126-d33f-6a9a-fbd6e9bd2216 [Follower]" leader=
    testing.go:149: server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: 2024-01-08T00:31:00.205Z [INFO]  server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: entering follower state: follower="Node at ee235919-0126-d33f-6a9a-fbd6e9bd2216 [Follower]" leader=
testLoggerAdapter verbose: server-567da5c9-fcd6-028c-64c5-eac7045ef0b7: 2024-01-08T00:31:00.205Z [INFO]  server-567da5c9-fcd6-028c-64c5-eac7045ef0b7: entering follower state: follower="Node at 567da5c9-fcd6-028c-64c5-eac7045ef0b7 [Follower]" leader=
    testing.go:149: server-567da5c9-fcd6-028c-64c5-eac7045ef0b7: 2024-01-08T00:31:00.205Z [INFO]  server-567da5c9-fcd6-028c-64c5-eac7045ef0b7: entering follower state: follower="Node at 567da5c9-fcd6-028c-64c5-eac7045ef0b7 [Follower]" leader=
testLoggerAdapter verbose: server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: 2024-01-08T00:31:00.298Z [WARN]  server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: 2024-01-08T00:31:00.298Z [WARN]  server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: 2024-01-08T00:31:00.298Z [INFO]  server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: entering candidate state: node="Node at ee235919-0126-d33f-6a9a-fbd6e9bd2216 [Candidate]" term=2
    testing.go:149: server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: 2024-01-08T00:31:00.298Z [INFO]  server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: entering candidate state: node="Node at ee235919-0126-d33f-6a9a-fbd6e9bd2216 [Candidate]" term=2
testLoggerAdapter verbose: server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: 2024-01-08T00:31:00.298Z [INFO]  server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: election won: tally=2
    testing.go:149: server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: 2024-01-08T00:31:00.298Z [INFO]  server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: election won: tally=2
testLoggerAdapter verbose: server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: 2024-01-08T00:31:00.298Z [INFO]  server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: entering leader state: leader="Node at ee235919-0126-d33f-6a9a-fbd6e9bd2216 [Leader]"
    testing.go:149: server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: 2024-01-08T00:31:00.298Z [INFO]  server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: entering leader state: leader="Node at ee235919-0126-d33f-6a9a-fbd6e9bd2216 [Leader]"
testLoggerAdapter verbose: server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: 2024-01-08T00:31:00.298Z [INFO]  server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: added peer, starting replication: peer=server-c94ae48a-40c5-cead-43b2-426a0d62a506
    testing.go:149: server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: 2024-01-08T00:31:00.298Z [INFO]  server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: added peer, starting replication: peer=server-c94ae48a-40c5-cead-43b2-426a0d62a506
testLoggerAdapter verbose: server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: 2024-01-08T00:31:00.298Z [INFO]  server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: added peer, starting replication: peer=server-567da5c9-fcd6-028c-64c5-eac7045ef0b7
    testing.go:149: server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: 2024-01-08T00:31:00.298Z [INFO]  server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: added peer, starting replication: peer=server-567da5c9-fcd6-028c-64c5-eac7045ef0b7
testLoggerAdapter verbose: server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: 2024-01-08T00:31:00.298Z [INFO]  server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: pipelining replication: peer="{Voter server-567da5c9-fcd6-028c-64c5-eac7045ef0b7 567da5c9-fcd6-028c-64c5-eac7045ef0b7}"
    testing.go:149: server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: 2024-01-08T00:31:00.298Z [INFO]  server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: pipelining replication: peer="{Voter server-567da5c9-fcd6-028c-64c5-eac7045ef0b7 567da5c9-fcd6-028c-64c5-eac7045ef0b7}"
testLoggerAdapter verbose: server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: 2024-01-08T00:31:00.299Z [INFO]  server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: pipelining replication: peer="{Voter server-c94ae48a-40c5-cead-43b2-426a0d62a506 c94ae48a-40c5-cead-43b2-426a0d62a506}"
    testing.go:149: server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: 2024-01-08T00:31:00.299Z [INFO]  server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: pipelining replication: peer="{Voter server-c94ae48a-40c5-cead-43b2-426a0d62a506 c94ae48a-40c5-cead-43b2-426a0d62a506}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:00.405Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:00.299191428 +0000 UTC m=+6.346080483 (1 nodes), 94.064449ms from start of poll, 94.579875ms from cluster start. Timeout at 2024-01-08 00:31:00.405083111 +0000 UTC m=+6.451972167, 105.891684ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:00.405Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:00.299191428 +0000 UTC m=+6.346080483 (1 nodes), 94.064449ms from start of poll, 94.579875ms from cluster start. Timeout at 2024-01-08 00:31:00.405083111 +0000 UTC m=+6.451972167, 105.891684ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:00.405Z [INFO]  starting stability test: raft-state=Follower
    testing.go:149: cluster: 2024-01-08T00:31:00.405Z [INFO]  starting stability test: raft-state=Follower
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:00.510Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:00.4052561 +0000 UTC m=+6.452145314 (2 nodes), 2.707µs from start of poll, 200.644706ms from cluster start. Timeout at 2024-01-08 00:31:00.510524944 +0000 UTC m=+6.557414102, 105.268788ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:00.510Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:00.4052561 +0000 UTC m=+6.452145314 (2 nodes), 2.707µs from start of poll, 200.644706ms from cluster start. Timeout at 2024-01-08 00:31:00.510524944 +0000 UTC m=+6.557414102, 105.268788ms after stability
testLoggerAdapter verbose: server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: 2024-01-08T00:31:00.510Z [INFO]  server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: aborting pipeline replication: peer="{Voter server-567da5c9-fcd6-028c-64c5-eac7045ef0b7 567da5c9-fcd6-028c-64c5-eac7045ef0b7}"
    testing.go:149: server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: 2024-01-08T00:31:00.510Z [INFO]  server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: aborting pipeline replication: peer="{Voter server-567da5c9-fcd6-028c-64c5-eac7045ef0b7 567da5c9-fcd6-028c-64c5-eac7045ef0b7}"
testLoggerAdapter verbose: server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: 2024-01-08T00:31:00.510Z [INFO]  server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: aborting pipeline replication: peer="{Voter server-c94ae48a-40c5-cead-43b2-426a0d62a506 c94ae48a-40c5-cead-43b2-426a0d62a506}"
    testing.go:149: server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: 2024-01-08T00:31:00.510Z [INFO]  server-ee235919-0126-d33f-6a9a-fbd6e9bd2216: aborting pipeline replication: peer="{Voter server-c94ae48a-40c5-cead-43b2-426a0d62a506 c94ae48a-40c5-cead-43b2-426a0d62a506}"
--- PASS: TestRaft_ApplyNonLeader (0.31s)
=== RUN   TestRaft_ApplyConcurrent
testLoggerAdapter verbose: server-e34a038b-075e-3cbe-6d6c-013f2913909e: 2024-01-08T00:31:00.512Z [INFO]  server-e34a038b-075e-3cbe-6d6c-013f2913909e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e34a038b-075e-3cbe-6d6c-013f2913909e Address:e34a038b-075e-3cbe-6d6c-013f2913909e} {Suffrage:Voter ID:server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507 Address:4c99d2b2-3331-681e-5fb0-42ad6ea7e507} {Suffrage:Voter ID:server-c7c7daf6-a912-c817-eea7-cd8c3644e532 Address:c7c7daf6-a912-c817-eea7-cd8c3644e532}]"
    testing.go:149: server-e34a038b-075e-3cbe-6d6c-013f2913909e: 2024-01-08T00:31:00.512Z [INFO]  server-e34a038b-075e-3cbe-6d6c-013f2913909e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e34a038b-075e-3cbe-6d6c-013f2913909e Address:e34a038b-075e-3cbe-6d6c-013f2913909e} {Suffrage:Voter ID:server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507 Address:4c99d2b2-3331-681e-5fb0-42ad6ea7e507} {Suffrage:Voter ID:server-c7c7daf6-a912-c817-eea7-cd8c3644e532 Address:c7c7daf6-a912-c817-eea7-cd8c3644e532}]"
testLoggerAdapter verbose: server-e34a038b-075e-3cbe-6d6c-013f2913909e: 2024-01-08T00:31:00.512Z [INFO]  server-e34a038b-075e-3cbe-6d6c-013f2913909e: entering follower state: follower="Node at e34a038b-075e-3cbe-6d6c-013f2913909e [Follower]" leader=
testLoggerAdapter verbose: server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: 2024-01-08T00:31:00.512Z [INFO]  server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e34a038b-075e-3cbe-6d6c-013f2913909e Address:e34a038b-075e-3cbe-6d6c-013f2913909e} {Suffrage:Voter ID:server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507 Address:4c99d2b2-3331-681e-5fb0-42ad6ea7e507} {Suffrage:Voter ID:server-c7c7daf6-a912-c817-eea7-cd8c3644e532 Address:c7c7daf6-a912-c817-eea7-cd8c3644e532}]"
    testing.go:149: server-e34a038b-075e-3cbe-6d6c-013f2913909e: 2024-01-08T00:31:00.512Z [INFO]  server-e34a038b-075e-3cbe-6d6c-013f2913909e: entering follower state: follower="Node at e34a038b-075e-3cbe-6d6c-013f2913909e [Follower]" leader=
    testing.go:149: server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: 2024-01-08T00:31:00.512Z [INFO]  server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e34a038b-075e-3cbe-6d6c-013f2913909e Address:e34a038b-075e-3cbe-6d6c-013f2913909e} {Suffrage:Voter ID:server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507 Address:4c99d2b2-3331-681e-5fb0-42ad6ea7e507} {Suffrage:Voter ID:server-c7c7daf6-a912-c817-eea7-cd8c3644e532 Address:c7c7daf6-a912-c817-eea7-cd8c3644e532}]"
testLoggerAdapter verbose: server-c7c7daf6-a912-c817-eea7-cd8c3644e532: 2024-01-08T00:31:00.512Z [INFO]  server-c7c7daf6-a912-c817-eea7-cd8c3644e532: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e34a038b-075e-3cbe-6d6c-013f2913909e Address:e34a038b-075e-3cbe-6d6c-013f2913909e} {Suffrage:Voter ID:server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507 Address:4c99d2b2-3331-681e-5fb0-42ad6ea7e507} {Suffrage:Voter ID:server-c7c7daf6-a912-c817-eea7-cd8c3644e532 Address:c7c7daf6-a912-c817-eea7-cd8c3644e532}]"
    testing.go:149: server-c7c7daf6-a912-c817-eea7-cd8c3644e532: 2024-01-08T00:31:00.512Z [INFO]  server-c7c7daf6-a912-c817-eea7-cd8c3644e532: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e34a038b-075e-3cbe-6d6c-013f2913909e Address:e34a038b-075e-3cbe-6d6c-013f2913909e} {Suffrage:Voter ID:server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507 Address:4c99d2b2-3331-681e-5fb0-42ad6ea7e507} {Suffrage:Voter ID:server-c7c7daf6-a912-c817-eea7-cd8c3644e532 Address:c7c7daf6-a912-c817-eea7-cd8c3644e532}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:00.512Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:00.512Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: 2024-01-08T00:31:00.512Z [INFO]  server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: entering follower state: follower="Node at 4c99d2b2-3331-681e-5fb0-42ad6ea7e507 [Follower]" leader=
    testing.go:149: server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: 2024-01-08T00:31:00.512Z [INFO]  server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: entering follower state: follower="Node at 4c99d2b2-3331-681e-5fb0-42ad6ea7e507 [Follower]" leader=
testLoggerAdapter verbose: server-c7c7daf6-a912-c817-eea7-cd8c3644e532: 2024-01-08T00:31:00.512Z [INFO]  server-c7c7daf6-a912-c817-eea7-cd8c3644e532: entering follower state: follower="Node at c7c7daf6-a912-c817-eea7-cd8c3644e532 [Follower]" leader=
    testing.go:149: server-c7c7daf6-a912-c817-eea7-cd8c3644e532: 2024-01-08T00:31:00.512Z [INFO]  server-c7c7daf6-a912-c817-eea7-cd8c3644e532: entering follower state: follower="Node at c7c7daf6-a912-c817-eea7-cd8c3644e532 [Follower]" leader=
testLoggerAdapter verbose: server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: 2024-01-08T00:31:00.662Z [WARN]  server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: 2024-01-08T00:31:00.662Z [WARN]  server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: 2024-01-08T00:31:00.662Z [INFO]  server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: entering candidate state: node="Node at 4c99d2b2-3331-681e-5fb0-42ad6ea7e507 [Candidate]" term=2
    testing.go:149: server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: 2024-01-08T00:31:00.662Z [INFO]  server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: entering candidate state: node="Node at 4c99d2b2-3331-681e-5fb0-42ad6ea7e507 [Candidate]" term=2
testLoggerAdapter verbose: server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: 2024-01-08T00:31:00.662Z [INFO]  server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: election won: tally=2
    testing.go:149: server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: 2024-01-08T00:31:00.662Z [INFO]  server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: election won: tally=2
testLoggerAdapter verbose: server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: 2024-01-08T00:31:00.662Z [INFO]  server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: entering leader state: leader="Node at 4c99d2b2-3331-681e-5fb0-42ad6ea7e507 [Leader]"
    testing.go:149: server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: 2024-01-08T00:31:00.662Z [INFO]  server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: entering leader state: leader="Node at 4c99d2b2-3331-681e-5fb0-42ad6ea7e507 [Leader]"
testLoggerAdapter verbose: server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: 2024-01-08T00:31:00.662Z [INFO]  server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: added peer, starting replication: peer=server-e34a038b-075e-3cbe-6d6c-013f2913909e
    testing.go:149: server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: 2024-01-08T00:31:00.662Z [INFO]  server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: added peer, starting replication: peer=server-e34a038b-075e-3cbe-6d6c-013f2913909e
testLoggerAdapter verbose: server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: 2024-01-08T00:31:00.662Z [INFO]  server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: added peer, starting replication: peer=server-c7c7daf6-a912-c817-eea7-cd8c3644e532
    testing.go:149: server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: 2024-01-08T00:31:00.662Z [INFO]  server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: added peer, starting replication: peer=server-c7c7daf6-a912-c817-eea7-cd8c3644e532
testLoggerAdapter verbose: server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: 2024-01-08T00:31:00.662Z [INFO]  server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: pipelining replication: peer="{Voter server-c7c7daf6-a912-c817-eea7-cd8c3644e532 c7c7daf6-a912-c817-eea7-cd8c3644e532}"
    testing.go:149: server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: 2024-01-08T00:31:00.662Z [INFO]  server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: pipelining replication: peer="{Voter server-c7c7daf6-a912-c817-eea7-cd8c3644e532 c7c7daf6-a912-c817-eea7-cd8c3644e532}"
testLoggerAdapter verbose: server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: 2024-01-08T00:31:00.662Z [INFO]  server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: pipelining replication: peer="{Voter server-e34a038b-075e-3cbe-6d6c-013f2913909e e34a038b-075e-3cbe-6d6c-013f2913909e}"
    testing.go:149: server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: 2024-01-08T00:31:00.662Z [INFO]  server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: pipelining replication: peer="{Voter server-e34a038b-075e-3cbe-6d6c-013f2913909e e34a038b-075e-3cbe-6d6c-013f2913909e}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:00.869Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:00.662925761 +0000 UTC m=+6.709814815 (1 nodes), 150.20566ms from start of poll, 150.859475ms from cluster start. Timeout at 2024-01-08 00:31:00.868704599 +0000 UTC m=+6.915593685, 205.77887ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:00.869Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:00.662925761 +0000 UTC m=+6.709814815 (1 nodes), 150.20566ms from start of poll, 150.859475ms from cluster start. Timeout at 2024-01-08 00:31:00.868704599 +0000 UTC m=+6.915593685, 205.77887ms after stability
testLoggerAdapter verbose: server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: 2024-01-08T00:31:00.882Z [INFO]  server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: aborting pipeline replication: peer="{Voter server-c7c7daf6-a912-c817-eea7-cd8c3644e532 c7c7daf6-a912-c817-eea7-cd8c3644e532}"
    testing.go:149: server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: 2024-01-08T00:31:00.882Z [INFO]  server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: aborting pipeline replication: peer="{Voter server-c7c7daf6-a912-c817-eea7-cd8c3644e532 c7c7daf6-a912-c817-eea7-cd8c3644e532}"
testLoggerAdapter verbose: server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: 2024-01-08T00:31:00.882Z [INFO]  server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: aborting pipeline replication: peer="{Voter server-e34a038b-075e-3cbe-6d6c-013f2913909e e34a038b-075e-3cbe-6d6c-013f2913909e}"
    testing.go:149: server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: 2024-01-08T00:31:00.882Z [INFO]  server-4c99d2b2-3331-681e-5fb0-42ad6ea7e507: aborting pipeline replication: peer="{Voter server-e34a038b-075e-3cbe-6d6c-013f2913909e e34a038b-075e-3cbe-6d6c-013f2913909e}"
--- PASS: TestRaft_ApplyConcurrent (0.37s)
=== RUN   TestRaft_ApplyConcurrent_Timeout
testLoggerAdapter verbose: server-42711a1b-8ca5-3d4a-dbff-88b96e634082: 2024-01-08T00:31:00.883Z [INFO]  server-42711a1b-8ca5-3d4a-dbff-88b96e634082: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42711a1b-8ca5-3d4a-dbff-88b96e634082 Address:42711a1b-8ca5-3d4a-dbff-88b96e634082}]"
    testing.go:149: server-42711a1b-8ca5-3d4a-dbff-88b96e634082: 2024-01-08T00:31:00.883Z [INFO]  server-42711a1b-8ca5-3d4a-dbff-88b96e634082: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42711a1b-8ca5-3d4a-dbff-88b96e634082 Address:42711a1b-8ca5-3d4a-dbff-88b96e634082}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:00.883Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:00.883Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-42711a1b-8ca5-3d4a-dbff-88b96e634082: 2024-01-08T00:31:00.883Z [INFO]  server-42711a1b-8ca5-3d4a-dbff-88b96e634082: entering follower state: follower="Node at 42711a1b-8ca5-3d4a-dbff-88b96e634082 [Follower]" leader=
    testing.go:149: server-42711a1b-8ca5-3d4a-dbff-88b96e634082: 2024-01-08T00:31:00.883Z [INFO]  server-42711a1b-8ca5-3d4a-dbff-88b96e634082: entering follower state: follower="Node at 42711a1b-8ca5-3d4a-dbff-88b96e634082 [Follower]" leader=
testLoggerAdapter verbose: server-42711a1b-8ca5-3d4a-dbff-88b96e634082: 2024-01-08T00:31:01.054Z [WARN]  server-42711a1b-8ca5-3d4a-dbff-88b96e634082: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-42711a1b-8ca5-3d4a-dbff-88b96e634082: 2024-01-08T00:31:01.054Z [WARN]  server-42711a1b-8ca5-3d4a-dbff-88b96e634082: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-42711a1b-8ca5-3d4a-dbff-88b96e634082: 2024-01-08T00:31:01.054Z [INFO]  server-42711a1b-8ca5-3d4a-dbff-88b96e634082: entering candidate state: node="Node at 42711a1b-8ca5-3d4a-dbff-88b96e634082 [Candidate]" term=2
    testing.go:149: server-42711a1b-8ca5-3d4a-dbff-88b96e634082: 2024-01-08T00:31:01.054Z [INFO]  server-42711a1b-8ca5-3d4a-dbff-88b96e634082: entering candidate state: node="Node at 42711a1b-8ca5-3d4a-dbff-88b96e634082 [Candidate]" term=2
testLoggerAdapter verbose: server-42711a1b-8ca5-3d4a-dbff-88b96e634082: 2024-01-08T00:31:01.054Z [INFO]  server-42711a1b-8ca5-3d4a-dbff-88b96e634082: election won: tally=1
    testing.go:149: server-42711a1b-8ca5-3d4a-dbff-88b96e634082: 2024-01-08T00:31:01.054Z [INFO]  server-42711a1b-8ca5-3d4a-dbff-88b96e634082: election won: tally=1
testLoggerAdapter verbose: server-42711a1b-8ca5-3d4a-dbff-88b96e634082: 2024-01-08T00:31:01.054Z [INFO]  server-42711a1b-8ca5-3d4a-dbff-88b96e634082: entering leader state: leader="Node at 42711a1b-8ca5-3d4a-dbff-88b96e634082 [Leader]"
    testing.go:149: server-42711a1b-8ca5-3d4a-dbff-88b96e634082: 2024-01-08T00:31:01.054Z [INFO]  server-42711a1b-8ca5-3d4a-dbff-88b96e634082: entering leader state: leader="Node at 42711a1b-8ca5-3d4a-dbff-88b96e634082 [Leader]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:01.256Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:01.054795003 +0000 UTC m=+7.101684045 (1 nodes), 171.672614ms from start of poll, 171.855826ms from cluster start. Timeout at 2024-01-08 00:31:01.25694606 +0000 UTC m=+7.303835185, 202.15114ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:01.256Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:01.054795003 +0000 UTC m=+7.101684045 (1 nodes), 171.672614ms from start of poll, 171.855826ms from cluster start. Timeout at 2024-01-08 00:31:01.25694606 +0000 UTC m=+7.303835185, 202.15114ms after stability
--- PASS: TestRaft_ApplyConcurrent_Timeout (0.38s)
=== RUN   TestRaft_JoinNode
testLoggerAdapter verbose: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.259Z [INFO]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7198e2e6-b1c3-0003-aae7-a998e679b2be Address:7198e2e6-b1c3-0003-aae7-a998e679b2be} {Suffrage:Voter ID:server-04f12023-517d-7f2b-f1cf-06545fbcde04 Address:04f12023-517d-7f2b-f1cf-06545fbcde04}]"
    testing.go:149: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.259Z [INFO]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7198e2e6-b1c3-0003-aae7-a998e679b2be Address:7198e2e6-b1c3-0003-aae7-a998e679b2be} {Suffrage:Voter ID:server-04f12023-517d-7f2b-f1cf-06545fbcde04 Address:04f12023-517d-7f2b-f1cf-06545fbcde04}]"
testLoggerAdapter verbose: server-04f12023-517d-7f2b-f1cf-06545fbcde04: 2024-01-08T00:31:01.260Z [INFO]  server-04f12023-517d-7f2b-f1cf-06545fbcde04: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7198e2e6-b1c3-0003-aae7-a998e679b2be Address:7198e2e6-b1c3-0003-aae7-a998e679b2be} {Suffrage:Voter ID:server-04f12023-517d-7f2b-f1cf-06545fbcde04 Address:04f12023-517d-7f2b-f1cf-06545fbcde04}]"
    testing.go:149: server-04f12023-517d-7f2b-f1cf-06545fbcde04: 2024-01-08T00:31:01.260Z [INFO]  server-04f12023-517d-7f2b-f1cf-06545fbcde04: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7198e2e6-b1c3-0003-aae7-a998e679b2be Address:7198e2e6-b1c3-0003-aae7-a998e679b2be} {Suffrage:Voter ID:server-04f12023-517d-7f2b-f1cf-06545fbcde04 Address:04f12023-517d-7f2b-f1cf-06545fbcde04}]"
testLoggerAdapter verbose: server-6a4b3372-729a-f122-9465-d18f6df32d65: 2024-01-08T00:31:01.260Z [INFO]  server-6a4b3372-729a-f122-9465-d18f6df32d65: initial configuration: index=0 servers=[]
    testing.go:149: server-6a4b3372-729a-f122-9465-d18f6df32d65: 2024-01-08T00:31:01.260Z [INFO]  server-6a4b3372-729a-f122-9465-d18f6df32d65: initial configuration: index=0 servers=[]
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:01.260Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:01.260Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.260Z [INFO]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: entering follower state: follower="Node at 7198e2e6-b1c3-0003-aae7-a998e679b2be [Follower]" leader=
    testing.go:149: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.260Z [INFO]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: entering follower state: follower="Node at 7198e2e6-b1c3-0003-aae7-a998e679b2be [Follower]" leader=
testLoggerAdapter verbose: server-04f12023-517d-7f2b-f1cf-06545fbcde04: 2024-01-08T00:31:01.260Z [INFO]  server-04f12023-517d-7f2b-f1cf-06545fbcde04: entering follower state: follower="Node at 04f12023-517d-7f2b-f1cf-06545fbcde04 [Follower]" leader=
    testing.go:149: server-04f12023-517d-7f2b-f1cf-06545fbcde04: 2024-01-08T00:31:01.260Z [INFO]  server-04f12023-517d-7f2b-f1cf-06545fbcde04: entering follower state: follower="Node at 04f12023-517d-7f2b-f1cf-06545fbcde04 [Follower]" leader=
testLoggerAdapter verbose: server-6a4b3372-729a-f122-9465-d18f6df32d65: 2024-01-08T00:31:01.260Z [INFO]  server-6a4b3372-729a-f122-9465-d18f6df32d65: entering follower state: follower="Node at 6a4b3372-729a-f122-9465-d18f6df32d65 [Follower]" leader=
    testing.go:149: server-6a4b3372-729a-f122-9465-d18f6df32d65: 2024-01-08T00:31:01.260Z [INFO]  server-6a4b3372-729a-f122-9465-d18f6df32d65: entering follower state: follower="Node at 6a4b3372-729a-f122-9465-d18f6df32d65 [Follower]" leader=
testLoggerAdapter verbose: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.312Z [WARN]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.312Z [WARN]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.312Z [INFO]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: entering candidate state: node="Node at 7198e2e6-b1c3-0003-aae7-a998e679b2be [Candidate]" term=2
    testing.go:149: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.312Z [INFO]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: entering candidate state: node="Node at 7198e2e6-b1c3-0003-aae7-a998e679b2be [Candidate]" term=2
testLoggerAdapter verbose: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.313Z [INFO]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: election won: tally=2
    testing.go:149: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.313Z [INFO]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: election won: tally=2
testLoggerAdapter verbose: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.313Z [INFO]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: entering leader state: leader="Node at 7198e2e6-b1c3-0003-aae7-a998e679b2be [Leader]"
    testing.go:149: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.313Z [INFO]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: entering leader state: leader="Node at 7198e2e6-b1c3-0003-aae7-a998e679b2be [Leader]"
testLoggerAdapter verbose: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.313Z [INFO]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: added peer, starting replication: peer=server-04f12023-517d-7f2b-f1cf-06545fbcde04
    testing.go:149: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.313Z [INFO]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: added peer, starting replication: peer=server-04f12023-517d-7f2b-f1cf-06545fbcde04
testLoggerAdapter verbose: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.313Z [INFO]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: pipelining replication: peer="{Voter server-04f12023-517d-7f2b-f1cf-06545fbcde04 04f12023-517d-7f2b-f1cf-06545fbcde04}"
    testing.go:149: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.313Z [INFO]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: pipelining replication: peer="{Voter server-04f12023-517d-7f2b-f1cf-06545fbcde04 04f12023-517d-7f2b-f1cf-06545fbcde04}"
testLoggerAdapter verbose: server-6a4b3372-729a-f122-9465-d18f6df32d65: 2024-01-08T00:31:01.316Z [WARN]  server-6a4b3372-729a-f122-9465-d18f6df32d65: no known peers, aborting election
    testing.go:149: server-6a4b3372-729a-f122-9465-d18f6df32d65: 2024-01-08T00:31:01.316Z [WARN]  server-6a4b3372-729a-f122-9465-d18f6df32d65: no known peers, aborting election
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:01.419Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:01.313450468 +0000 UTC m=+7.360339712 (1 nodes), 53.073445ms from start of poll, 53.575307ms from cluster start. Timeout at 2024-01-08 00:31:01.419476132 +0000 UTC m=+7.466365416, 106.025704ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:01.419Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:01.313450468 +0000 UTC m=+7.360339712 (1 nodes), 53.073445ms from start of poll, 53.575307ms from cluster start. Timeout at 2024-01-08 00:31:01.419476132 +0000 UTC m=+7.466365416, 106.025704ms after stability
testLoggerAdapter verbose: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.419Z [INFO]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: updating configuration: command=AddStaging server-id=server-6a4b3372-729a-f122-9465-d18f6df32d65 server-addr=6a4b3372-729a-f122-9465-d18f6df32d65 servers="[{Suffrage:Voter ID:server-7198e2e6-b1c3-0003-aae7-a998e679b2be Address:7198e2e6-b1c3-0003-aae7-a998e679b2be} {Suffrage:Voter ID:server-04f12023-517d-7f2b-f1cf-06545fbcde04 Address:04f12023-517d-7f2b-f1cf-06545fbcde04} {Suffrage:Voter ID:server-6a4b3372-729a-f122-9465-d18f6df32d65 Address:6a4b3372-729a-f122-9465-d18f6df32d65}]"
    testing.go:149: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.419Z [INFO]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: updating configuration: command=AddStaging server-id=server-6a4b3372-729a-f122-9465-d18f6df32d65 server-addr=6a4b3372-729a-f122-9465-d18f6df32d65 servers="[{Suffrage:Voter ID:server-7198e2e6-b1c3-0003-aae7-a998e679b2be Address:7198e2e6-b1c3-0003-aae7-a998e679b2be} {Suffrage:Voter ID:server-04f12023-517d-7f2b-f1cf-06545fbcde04 Address:04f12023-517d-7f2b-f1cf-06545fbcde04} {Suffrage:Voter ID:server-6a4b3372-729a-f122-9465-d18f6df32d65 Address:6a4b3372-729a-f122-9465-d18f6df32d65}]"
testLoggerAdapter verbose: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.419Z [INFO]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: added peer, starting replication: peer=server-6a4b3372-729a-f122-9465-d18f6df32d65
    testing.go:149: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.419Z [INFO]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: added peer, starting replication: peer=server-6a4b3372-729a-f122-9465-d18f6df32d65
testLoggerAdapter verbose: server-6a4b3372-729a-f122-9465-d18f6df32d65: 2024-01-08T00:31:01.419Z [WARN]  server-6a4b3372-729a-f122-9465-d18f6df32d65: failed to get previous log: previous-index=3 last-index=0 error="log not found"
    testing.go:149: server-6a4b3372-729a-f122-9465-d18f6df32d65: 2024-01-08T00:31:01.419Z [WARN]  server-6a4b3372-729a-f122-9465-d18f6df32d65: failed to get previous log: previous-index=3 last-index=0 error="log not found"
testLoggerAdapter verbose: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.419Z [WARN]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: appendEntries rejected, sending older logs: peer="{Voter server-6a4b3372-729a-f122-9465-d18f6df32d65 6a4b3372-729a-f122-9465-d18f6df32d65}" next=1
    testing.go:149: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.419Z [WARN]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: appendEntries rejected, sending older logs: peer="{Voter server-6a4b3372-729a-f122-9465-d18f6df32d65 6a4b3372-729a-f122-9465-d18f6df32d65}" next=1
testLoggerAdapter verbose: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.420Z [INFO]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: pipelining replication: peer="{Voter server-6a4b3372-729a-f122-9465-d18f6df32d65 6a4b3372-729a-f122-9465-d18f6df32d65}"
    testing.go:149: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.420Z [INFO]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: pipelining replication: peer="{Voter server-6a4b3372-729a-f122-9465-d18f6df32d65 6a4b3372-729a-f122-9465-d18f6df32d65}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:01.420Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:01.420Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:01.525Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:01.420274228 +0000 UTC m=+7.467163510 (1 nodes), 2.18µs from start of poll, 160.399105ms from cluster start. Timeout at 2024-01-08 00:31:01.525741828 +0000 UTC m=+7.572631066, 105.467556ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:01.525Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:01.420274228 +0000 UTC m=+7.467163510 (1 nodes), 2.18µs from start of poll, 160.399105ms from cluster start. Timeout at 2024-01-08 00:31:01.525741828 +0000 UTC m=+7.572631066, 105.467556ms after stability
testLoggerAdapter verbose: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.525Z [INFO]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: aborting pipeline replication: peer="{Voter server-6a4b3372-729a-f122-9465-d18f6df32d65 6a4b3372-729a-f122-9465-d18f6df32d65}"
    testing.go:149: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.525Z [INFO]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: aborting pipeline replication: peer="{Voter server-6a4b3372-729a-f122-9465-d18f6df32d65 6a4b3372-729a-f122-9465-d18f6df32d65}"
testLoggerAdapter verbose: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.526Z [INFO]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: aborting pipeline replication: peer="{Voter server-04f12023-517d-7f2b-f1cf-06545fbcde04 04f12023-517d-7f2b-f1cf-06545fbcde04}"
    testing.go:149: server-7198e2e6-b1c3-0003-aae7-a998e679b2be: 2024-01-08T00:31:01.526Z [INFO]  server-7198e2e6-b1c3-0003-aae7-a998e679b2be: aborting pipeline replication: peer="{Voter server-04f12023-517d-7f2b-f1cf-06545fbcde04 04f12023-517d-7f2b-f1cf-06545fbcde04}"
--- PASS: TestRaft_JoinNode (0.27s)
=== RUN   TestRaft_JoinNode_ConfigStore
testLoggerAdapter verbose: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.527Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-db8a847a-b3d9-9eb0-022b-79c71f04809d Address:db8a847a-b3d9-9eb0-022b-79c71f04809d}]"
    testing.go:149: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.527Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-db8a847a-b3d9-9eb0-022b-79c71f04809d Address:db8a847a-b3d9-9eb0-022b-79c71f04809d}]"
testLoggerAdapter verbose: server-f04316e6-db58-f69e-9ee1-2882377e6fff: 2024-01-08T00:31:01.527Z [INFO]  server-f04316e6-db58-f69e-9ee1-2882377e6fff: initial configuration: index=0 servers=[]
    testing.go:149: server-f04316e6-db58-f69e-9ee1-2882377e6fff: 2024-01-08T00:31:01.527Z [INFO]  server-f04316e6-db58-f69e-9ee1-2882377e6fff: initial configuration: index=0 servers=[]
testLoggerAdapter verbose: server-3994d176-a39a-d04f-3682-d70777352990: 2024-01-08T00:31:01.527Z [INFO]  server-3994d176-a39a-d04f-3682-d70777352990: initial configuration: index=0 servers=[]
    testing.go:149: server-3994d176-a39a-d04f-3682-d70777352990: 2024-01-08T00:31:01.527Z [INFO]  server-3994d176-a39a-d04f-3682-d70777352990: initial configuration: index=0 servers=[]
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:01.527Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:01.527Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.527Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: entering follower state: follower="Node at db8a847a-b3d9-9eb0-022b-79c71f04809d [Follower]" leader=
    testing.go:149: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.527Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: entering follower state: follower="Node at db8a847a-b3d9-9eb0-022b-79c71f04809d [Follower]" leader=
testLoggerAdapter verbose: server-f04316e6-db58-f69e-9ee1-2882377e6fff: 2024-01-08T00:31:01.527Z [INFO]  server-f04316e6-db58-f69e-9ee1-2882377e6fff: entering follower state: follower="Node at f04316e6-db58-f69e-9ee1-2882377e6fff [Follower]" leader=
    testing.go:149: server-f04316e6-db58-f69e-9ee1-2882377e6fff: 2024-01-08T00:31:01.527Z [INFO]  server-f04316e6-db58-f69e-9ee1-2882377e6fff: entering follower state: follower="Node at f04316e6-db58-f69e-9ee1-2882377e6fff [Follower]" leader=
testLoggerAdapter verbose: server-3994d176-a39a-d04f-3682-d70777352990: 2024-01-08T00:31:01.527Z [INFO]  server-3994d176-a39a-d04f-3682-d70777352990: entering follower state: follower="Node at 3994d176-a39a-d04f-3682-d70777352990 [Follower]" leader=
    testing.go:149: server-3994d176-a39a-d04f-3682-d70777352990: 2024-01-08T00:31:01.527Z [INFO]  server-3994d176-a39a-d04f-3682-d70777352990: entering follower state: follower="Node at 3994d176-a39a-d04f-3682-d70777352990 [Follower]" leader=
testLoggerAdapter verbose: server-f04316e6-db58-f69e-9ee1-2882377e6fff: 2024-01-08T00:31:01.588Z [WARN]  server-f04316e6-db58-f69e-9ee1-2882377e6fff: no known peers, aborting election
    testing.go:149: server-f04316e6-db58-f69e-9ee1-2882377e6fff: 2024-01-08T00:31:01.588Z [WARN]  server-f04316e6-db58-f69e-9ee1-2882377e6fff: no known peers, aborting election
testLoggerAdapter verbose: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.590Z [WARN]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.590Z [WARN]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.590Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: entering candidate state: node="Node at db8a847a-b3d9-9eb0-022b-79c71f04809d [Candidate]" term=2
    testing.go:149: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.590Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: entering candidate state: node="Node at db8a847a-b3d9-9eb0-022b-79c71f04809d [Candidate]" term=2
testLoggerAdapter verbose: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.590Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: election won: tally=1
    testing.go:149: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.590Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: election won: tally=1
testLoggerAdapter verbose: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.590Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: entering leader state: leader="Node at db8a847a-b3d9-9eb0-022b-79c71f04809d [Leader]"
    testing.go:149: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.590Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: entering leader state: leader="Node at db8a847a-b3d9-9eb0-022b-79c71f04809d [Leader]"
testLoggerAdapter verbose: server-3994d176-a39a-d04f-3682-d70777352990: 2024-01-08T00:31:01.597Z [WARN]  server-3994d176-a39a-d04f-3682-d70777352990: no known peers, aborting election
    testing.go:149: server-3994d176-a39a-d04f-3682-d70777352990: 2024-01-08T00:31:01.597Z [WARN]  server-3994d176-a39a-d04f-3682-d70777352990: no known peers, aborting election
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:01.695Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:01.59051652 +0000 UTC m=+7.637405624 (1 nodes), 62.814091ms from start of poll, 63.47673ms from cluster start. Timeout at 2024-01-08 00:31:01.695537593 +0000 UTC m=+7.742426818, 105.021194ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:01.695Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:01.59051652 +0000 UTC m=+7.637405624 (1 nodes), 62.814091ms from start of poll, 63.47673ms from cluster start. Timeout at 2024-01-08 00:31:01.695537593 +0000 UTC m=+7.742426818, 105.021194ms after stability
testLoggerAdapter verbose: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.695Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: updating configuration: command=AddStaging server-id=server-f04316e6-db58-f69e-9ee1-2882377e6fff server-addr=f04316e6-db58-f69e-9ee1-2882377e6fff servers="[{Suffrage:Voter ID:server-db8a847a-b3d9-9eb0-022b-79c71f04809d Address:db8a847a-b3d9-9eb0-022b-79c71f04809d} {Suffrage:Voter ID:server-f04316e6-db58-f69e-9ee1-2882377e6fff Address:f04316e6-db58-f69e-9ee1-2882377e6fff}]"
    testing.go:149: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.695Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: updating configuration: command=AddStaging server-id=server-f04316e6-db58-f69e-9ee1-2882377e6fff server-addr=f04316e6-db58-f69e-9ee1-2882377e6fff servers="[{Suffrage:Voter ID:server-db8a847a-b3d9-9eb0-022b-79c71f04809d Address:db8a847a-b3d9-9eb0-022b-79c71f04809d} {Suffrage:Voter ID:server-f04316e6-db58-f69e-9ee1-2882377e6fff Address:f04316e6-db58-f69e-9ee1-2882377e6fff}]"
testLoggerAdapter verbose: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.695Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: added peer, starting replication: peer=server-f04316e6-db58-f69e-9ee1-2882377e6fff
    testing.go:149: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.695Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: added peer, starting replication: peer=server-f04316e6-db58-f69e-9ee1-2882377e6fff
testLoggerAdapter verbose: server-f04316e6-db58-f69e-9ee1-2882377e6fff: 2024-01-08T00:31:01.695Z [WARN]  server-f04316e6-db58-f69e-9ee1-2882377e6fff: failed to get previous log: previous-index=3 last-index=0 error="log not found"
    testing.go:149: server-f04316e6-db58-f69e-9ee1-2882377e6fff: 2024-01-08T00:31:01.695Z [WARN]  server-f04316e6-db58-f69e-9ee1-2882377e6fff: failed to get previous log: previous-index=3 last-index=0 error="log not found"
testLoggerAdapter verbose: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.695Z [WARN]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: appendEntries rejected, sending older logs: peer="{Voter server-f04316e6-db58-f69e-9ee1-2882377e6fff f04316e6-db58-f69e-9ee1-2882377e6fff}" next=1
    testing.go:149: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.695Z [WARN]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: appendEntries rejected, sending older logs: peer="{Voter server-f04316e6-db58-f69e-9ee1-2882377e6fff f04316e6-db58-f69e-9ee1-2882377e6fff}" next=1
testLoggerAdapter verbose: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.696Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: pipelining replication: peer="{Voter server-f04316e6-db58-f69e-9ee1-2882377e6fff f04316e6-db58-f69e-9ee1-2882377e6fff}"
    testing.go:149: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.696Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: pipelining replication: peer="{Voter server-f04316e6-db58-f69e-9ee1-2882377e6fff f04316e6-db58-f69e-9ee1-2882377e6fff}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:01.696Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:01.696Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:01.801Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:01.696189828 +0000 UTC m=+7.743079006 (1 nodes), 2.274µs from start of poll, 169.150112ms from cluster start. Timeout at 2024-01-08 00:31:01.801544492 +0000 UTC m=+7.848433727, 105.354721ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:01.801Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:01.696189828 +0000 UTC m=+7.743079006 (1 nodes), 2.274µs from start of poll, 169.150112ms from cluster start. Timeout at 2024-01-08 00:31:01.801544492 +0000 UTC m=+7.848433727, 105.354721ms after stability
testLoggerAdapter verbose: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.801Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: updating configuration: command=AddStaging server-id=server-3994d176-a39a-d04f-3682-d70777352990 server-addr=3994d176-a39a-d04f-3682-d70777352990 servers="[{Suffrage:Voter ID:server-db8a847a-b3d9-9eb0-022b-79c71f04809d Address:db8a847a-b3d9-9eb0-022b-79c71f04809d} {Suffrage:Voter ID:server-f04316e6-db58-f69e-9ee1-2882377e6fff Address:f04316e6-db58-f69e-9ee1-2882377e6fff} {Suffrage:Voter ID:server-3994d176-a39a-d04f-3682-d70777352990 Address:3994d176-a39a-d04f-3682-d70777352990}]"
    testing.go:149: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.801Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: updating configuration: command=AddStaging server-id=server-3994d176-a39a-d04f-3682-d70777352990 server-addr=3994d176-a39a-d04f-3682-d70777352990 servers="[{Suffrage:Voter ID:server-db8a847a-b3d9-9eb0-022b-79c71f04809d Address:db8a847a-b3d9-9eb0-022b-79c71f04809d} {Suffrage:Voter ID:server-f04316e6-db58-f69e-9ee1-2882377e6fff Address:f04316e6-db58-f69e-9ee1-2882377e6fff} {Suffrage:Voter ID:server-3994d176-a39a-d04f-3682-d70777352990 Address:3994d176-a39a-d04f-3682-d70777352990}]"
testLoggerAdapter verbose: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.801Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: added peer, starting replication: peer=server-3994d176-a39a-d04f-3682-d70777352990
    testing.go:149: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.801Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: added peer, starting replication: peer=server-3994d176-a39a-d04f-3682-d70777352990
testLoggerAdapter verbose: server-3994d176-a39a-d04f-3682-d70777352990: 2024-01-08T00:31:01.801Z [WARN]  server-3994d176-a39a-d04f-3682-d70777352990: failed to get previous log: previous-index=4 last-index=0 error="log not found"
    testing.go:149: server-3994d176-a39a-d04f-3682-d70777352990: 2024-01-08T00:31:01.801Z [WARN]  server-3994d176-a39a-d04f-3682-d70777352990: failed to get previous log: previous-index=4 last-index=0 error="log not found"
testLoggerAdapter verbose: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.801Z [WARN]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: appendEntries rejected, sending older logs: peer="{Voter server-3994d176-a39a-d04f-3682-d70777352990 3994d176-a39a-d04f-3682-d70777352990}" next=1
    testing.go:149: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.801Z [WARN]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: appendEntries rejected, sending older logs: peer="{Voter server-3994d176-a39a-d04f-3682-d70777352990 3994d176-a39a-d04f-3682-d70777352990}" next=1
testLoggerAdapter verbose: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.802Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: pipelining replication: peer="{Voter server-3994d176-a39a-d04f-3682-d70777352990 3994d176-a39a-d04f-3682-d70777352990}"
    testing.go:149: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.802Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: pipelining replication: peer="{Voter server-3994d176-a39a-d04f-3682-d70777352990 3994d176-a39a-d04f-3682-d70777352990}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:01.802Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:01.802Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:01.907Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:01.802255276 +0000 UTC m=+7.849144360 (1 nodes), 2.054µs from start of poll, 275.215466ms from cluster start. Timeout at 2024-01-08 00:31:01.907771546 +0000 UTC m=+7.954660687, 105.516327ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:01.907Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:01.802255276 +0000 UTC m=+7.849144360 (1 nodes), 2.054µs from start of poll, 275.215466ms from cluster start. Timeout at 2024-01-08 00:31:01.907771546 +0000 UTC m=+7.954660687, 105.516327ms after stability
testLoggerAdapter verbose: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.907Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: aborting pipeline replication: peer="{Voter server-3994d176-a39a-d04f-3682-d70777352990 3994d176-a39a-d04f-3682-d70777352990}"
    testing.go:149: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.907Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: aborting pipeline replication: peer="{Voter server-3994d176-a39a-d04f-3682-d70777352990 3994d176-a39a-d04f-3682-d70777352990}"
testLoggerAdapter verbose: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.908Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: aborting pipeline replication: peer="{Voter server-f04316e6-db58-f69e-9ee1-2882377e6fff f04316e6-db58-f69e-9ee1-2882377e6fff}"
    testing.go:149: server-db8a847a-b3d9-9eb0-022b-79c71f04809d: 2024-01-08T00:31:01.908Z [INFO]  server-db8a847a-b3d9-9eb0-022b-79c71f04809d: aborting pipeline replication: peer="{Voter server-f04316e6-db58-f69e-9ee1-2882377e6fff f04316e6-db58-f69e-9ee1-2882377e6fff}"
--- PASS: TestRaft_JoinNode_ConfigStore (0.38s)
=== RUN   TestRaft_RemoveFollower
testLoggerAdapter verbose: server-3f9779dc-bb02-61c2-80be-2e80e1a434c5: 2024-01-08T00:31:01.909Z [INFO]  server-3f9779dc-bb02-61c2-80be-2e80e1a434c5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3f9779dc-bb02-61c2-80be-2e80e1a434c5 Address:3f9779dc-bb02-61c2-80be-2e80e1a434c5} {Suffrage:Voter ID:server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8 Address:8734b349-06b6-45f6-9f8f-b52a9e96fbd8} {Suffrage:Voter ID:server-3acca375-1934-dbdd-b9cd-bef661b10ce0 Address:3acca375-1934-dbdd-b9cd-bef661b10ce0}]"
    testing.go:149: server-3f9779dc-bb02-61c2-80be-2e80e1a434c5: 2024-01-08T00:31:01.909Z [INFO]  server-3f9779dc-bb02-61c2-80be-2e80e1a434c5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3f9779dc-bb02-61c2-80be-2e80e1a434c5 Address:3f9779dc-bb02-61c2-80be-2e80e1a434c5} {Suffrage:Voter ID:server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8 Address:8734b349-06b6-45f6-9f8f-b52a9e96fbd8} {Suffrage:Voter ID:server-3acca375-1934-dbdd-b9cd-bef661b10ce0 Address:3acca375-1934-dbdd-b9cd-bef661b10ce0}]"
testLoggerAdapter verbose: server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8: 2024-01-08T00:31:01.909Z [INFO]  server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3f9779dc-bb02-61c2-80be-2e80e1a434c5 Address:3f9779dc-bb02-61c2-80be-2e80e1a434c5} {Suffrage:Voter ID:server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8 Address:8734b349-06b6-45f6-9f8f-b52a9e96fbd8} {Suffrage:Voter ID:server-3acca375-1934-dbdd-b9cd-bef661b10ce0 Address:3acca375-1934-dbdd-b9cd-bef661b10ce0}]"
    testing.go:149: server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8: 2024-01-08T00:31:01.909Z [INFO]  server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3f9779dc-bb02-61c2-80be-2e80e1a434c5 Address:3f9779dc-bb02-61c2-80be-2e80e1a434c5} {Suffrage:Voter ID:server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8 Address:8734b349-06b6-45f6-9f8f-b52a9e96fbd8} {Suffrage:Voter ID:server-3acca375-1934-dbdd-b9cd-bef661b10ce0 Address:3acca375-1934-dbdd-b9cd-bef661b10ce0}]"
testLoggerAdapter verbose: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:01.909Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3f9779dc-bb02-61c2-80be-2e80e1a434c5 Address:3f9779dc-bb02-61c2-80be-2e80e1a434c5} {Suffrage:Voter ID:server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8 Address:8734b349-06b6-45f6-9f8f-b52a9e96fbd8} {Suffrage:Voter ID:server-3acca375-1934-dbdd-b9cd-bef661b10ce0 Address:3acca375-1934-dbdd-b9cd-bef661b10ce0}]"
    testing.go:149: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:01.909Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3f9779dc-bb02-61c2-80be-2e80e1a434c5 Address:3f9779dc-bb02-61c2-80be-2e80e1a434c5} {Suffrage:Voter ID:server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8 Address:8734b349-06b6-45f6-9f8f-b52a9e96fbd8} {Suffrage:Voter ID:server-3acca375-1934-dbdd-b9cd-bef661b10ce0 Address:3acca375-1934-dbdd-b9cd-bef661b10ce0}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:01.909Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:01.909Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-3f9779dc-bb02-61c2-80be-2e80e1a434c5: 2024-01-08T00:31:01.909Z [INFO]  server-3f9779dc-bb02-61c2-80be-2e80e1a434c5: entering follower state: follower="Node at 3f9779dc-bb02-61c2-80be-2e80e1a434c5 [Follower]" leader=
    testing.go:149: server-3f9779dc-bb02-61c2-80be-2e80e1a434c5: 2024-01-08T00:31:01.909Z [INFO]  server-3f9779dc-bb02-61c2-80be-2e80e1a434c5: entering follower state: follower="Node at 3f9779dc-bb02-61c2-80be-2e80e1a434c5 [Follower]" leader=
testLoggerAdapter verbose: server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8: 2024-01-08T00:31:01.910Z [INFO]  server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8: entering follower state: follower="Node at 8734b349-06b6-45f6-9f8f-b52a9e96fbd8 [Follower]" leader=
    testing.go:149: server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8: 2024-01-08T00:31:01.910Z [INFO]  server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8: entering follower state: follower="Node at 8734b349-06b6-45f6-9f8f-b52a9e96fbd8 [Follower]" leader=
testLoggerAdapter verbose: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:01.910Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: entering follower state: follower="Node at 3acca375-1934-dbdd-b9cd-bef661b10ce0 [Follower]" leader=
    testing.go:149: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:01.910Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: entering follower state: follower="Node at 3acca375-1934-dbdd-b9cd-bef661b10ce0 [Follower]" leader=
testLoggerAdapter verbose: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:01.970Z [WARN]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:01.970Z [WARN]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:01.970Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: entering candidate state: node="Node at 3acca375-1934-dbdd-b9cd-bef661b10ce0 [Candidate]" term=2
    testing.go:149: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:01.970Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: entering candidate state: node="Node at 3acca375-1934-dbdd-b9cd-bef661b10ce0 [Candidate]" term=2
testLoggerAdapter verbose: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:01.970Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: election won: tally=2
    testing.go:149: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:01.970Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: election won: tally=2
testLoggerAdapter verbose: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:01.970Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: entering leader state: leader="Node at 3acca375-1934-dbdd-b9cd-bef661b10ce0 [Leader]"
    testing.go:149: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:01.970Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: entering leader state: leader="Node at 3acca375-1934-dbdd-b9cd-bef661b10ce0 [Leader]"
testLoggerAdapter verbose: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:01.970Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: added peer, starting replication: peer=server-3f9779dc-bb02-61c2-80be-2e80e1a434c5
    testing.go:149: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:01.970Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: added peer, starting replication: peer=server-3f9779dc-bb02-61c2-80be-2e80e1a434c5
testLoggerAdapter verbose: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:01.971Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: added peer, starting replication: peer=server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8
    testing.go:149: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:01.971Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: added peer, starting replication: peer=server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8
testLoggerAdapter verbose: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:01.971Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: pipelining replication: peer="{Voter server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8 8734b349-06b6-45f6-9f8f-b52a9e96fbd8}"
    testing.go:149: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:01.971Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: pipelining replication: peer="{Voter server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8 8734b349-06b6-45f6-9f8f-b52a9e96fbd8}"
testLoggerAdapter verbose: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:01.971Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: pipelining replication: peer="{Voter server-3f9779dc-bb02-61c2-80be-2e80e1a434c5 3f9779dc-bb02-61c2-80be-2e80e1a434c5}"
    testing.go:149: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:01.971Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: pipelining replication: peer="{Voter server-3f9779dc-bb02-61c2-80be-2e80e1a434c5 3f9779dc-bb02-61c2-80be-2e80e1a434c5}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:02.076Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:01.971339231 +0000 UTC m=+8.018228321 (1 nodes), 61.361401ms from start of poll, 61.898196ms from cluster start. Timeout at 2024-01-08 00:31:02.076610168 +0000 UTC m=+8.123499302, 105.270981ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:02.076Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:01.971339231 +0000 UTC m=+8.018228321 (1 nodes), 61.361401ms from start of poll, 61.898196ms from cluster start. Timeout at 2024-01-08 00:31:02.076610168 +0000 UTC m=+8.123499302, 105.270981ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:02.083Z [INFO]  starting stability test: raft-state=Follower
    testing.go:149: cluster: 2024-01-08T00:31:02.083Z [INFO]  starting stability test: raft-state=Follower
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:02.188Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:02.083347634 +0000 UTC m=+8.130236728 (2 nodes), 2.457µs from start of poll, 173.906603ms from cluster start. Timeout at 2024-01-08 00:31:02.188657989 +0000 UTC m=+8.235547099, 105.310371ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:02.188Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:02.083347634 +0000 UTC m=+8.130236728 (2 nodes), 2.457µs from start of poll, 173.906603ms from cluster start. Timeout at 2024-01-08 00:31:02.188657989 +0000 UTC m=+8.235547099, 105.310371ms after stability
testLoggerAdapter verbose: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:02.188Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: updating configuration: command=RemoveServer server-id=server-3f9779dc-bb02-61c2-80be-2e80e1a434c5 server-addr= servers="[{Suffrage:Voter ID:server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8 Address:8734b349-06b6-45f6-9f8f-b52a9e96fbd8} {Suffrage:Voter ID:server-3acca375-1934-dbdd-b9cd-bef661b10ce0 Address:3acca375-1934-dbdd-b9cd-bef661b10ce0}]"
    testing.go:149: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:02.188Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: updating configuration: command=RemoveServer server-id=server-3f9779dc-bb02-61c2-80be-2e80e1a434c5 server-addr= servers="[{Suffrage:Voter ID:server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8 Address:8734b349-06b6-45f6-9f8f-b52a9e96fbd8} {Suffrage:Voter ID:server-3acca375-1934-dbdd-b9cd-bef661b10ce0 Address:3acca375-1934-dbdd-b9cd-bef661b10ce0}]"
testLoggerAdapter verbose: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:02.188Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: removed peer, stopping replication: peer=server-3f9779dc-bb02-61c2-80be-2e80e1a434c5 last-index=3
    testing.go:149: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:02.188Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: removed peer, stopping replication: peer=server-3f9779dc-bb02-61c2-80be-2e80e1a434c5 last-index=3
testLoggerAdapter verbose: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:02.188Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: aborting pipeline replication: peer="{Voter server-3f9779dc-bb02-61c2-80be-2e80e1a434c5 3f9779dc-bb02-61c2-80be-2e80e1a434c5}"
    testing.go:149: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:02.188Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: aborting pipeline replication: peer="{Voter server-3f9779dc-bb02-61c2-80be-2e80e1a434c5 3f9779dc-bb02-61c2-80be-2e80e1a434c5}"
testLoggerAdapter verbose: server-3f9779dc-bb02-61c2-80be-2e80e1a434c5: 2024-01-08T00:31:02.291Z [WARN]  server-3f9779dc-bb02-61c2-80be-2e80e1a434c5: heartbeat timeout reached, starting election: last-leader=3acca375-1934-dbdd-b9cd-bef661b10ce0
    testing.go:149: server-3f9779dc-bb02-61c2-80be-2e80e1a434c5: 2024-01-08T00:31:02.291Z [WARN]  server-3f9779dc-bb02-61c2-80be-2e80e1a434c5: heartbeat timeout reached, starting election: last-leader=3acca375-1934-dbdd-b9cd-bef661b10ce0
testLoggerAdapter verbose: server-3f9779dc-bb02-61c2-80be-2e80e1a434c5: 2024-01-08T00:31:02.291Z [INFO]  server-3f9779dc-bb02-61c2-80be-2e80e1a434c5: entering candidate state: node="Node at 3f9779dc-bb02-61c2-80be-2e80e1a434c5 [Candidate]" term=3
    testing.go:149: server-3f9779dc-bb02-61c2-80be-2e80e1a434c5: 2024-01-08T00:31:02.291Z [INFO]  server-3f9779dc-bb02-61c2-80be-2e80e1a434c5: entering candidate state: node="Node at 3f9779dc-bb02-61c2-80be-2e80e1a434c5 [Candidate]" term=3
testLoggerAdapter verbose: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:02.291Z [WARN]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: rejecting vote request since we have a leader: from=3f9779dc-bb02-61c2-80be-2e80e1a434c5 leader=3acca375-1934-dbdd-b9cd-bef661b10ce0
    testing.go:149: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:02.291Z [WARN]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: rejecting vote request since we have a leader: from=3f9779dc-bb02-61c2-80be-2e80e1a434c5 leader=3acca375-1934-dbdd-b9cd-bef661b10ce0
testLoggerAdapter verbose: server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8: 2024-01-08T00:31:02.292Z [WARN]  server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8: rejecting vote request since we have a leader: from=3f9779dc-bb02-61c2-80be-2e80e1a434c5 leader=3acca375-1934-dbdd-b9cd-bef661b10ce0
    testing.go:149: server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8: 2024-01-08T00:31:02.292Z [WARN]  server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8: rejecting vote request since we have a leader: from=3f9779dc-bb02-61c2-80be-2e80e1a434c5 leader=3acca375-1934-dbdd-b9cd-bef661b10ce0
testLoggerAdapter verbose: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:02.294Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: aborting pipeline replication: peer="{Voter server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8 8734b349-06b6-45f6-9f8f-b52a9e96fbd8}"
    testing.go:149: server-3acca375-1934-dbdd-b9cd-bef661b10ce0: 2024-01-08T00:31:02.294Z [INFO]  server-3acca375-1934-dbdd-b9cd-bef661b10ce0: aborting pipeline replication: peer="{Voter server-8734b349-06b6-45f6-9f8f-b52a9e96fbd8 8734b349-06b6-45f6-9f8f-b52a9e96fbd8}"
--- PASS: TestRaft_RemoveFollower (0.39s)
=== RUN   TestRaft_RemoveLeader
testLoggerAdapter verbose: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.295Z [INFO]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-31a1b4e1-a672-c703-b03b-4730ec8b4a57 Address:31a1b4e1-a672-c703-b03b-4730ec8b4a57} {Suffrage:Voter ID:server-8176d594-816f-b691-12da-1cf5ae6bae90 Address:8176d594-816f-b691-12da-1cf5ae6bae90} {Suffrage:Voter ID:server-13f348d1-8e65-864d-3378-178df005bf6d Address:13f348d1-8e65-864d-3378-178df005bf6d}]"
    testing.go:149: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.295Z [INFO]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-31a1b4e1-a672-c703-b03b-4730ec8b4a57 Address:31a1b4e1-a672-c703-b03b-4730ec8b4a57} {Suffrage:Voter ID:server-8176d594-816f-b691-12da-1cf5ae6bae90 Address:8176d594-816f-b691-12da-1cf5ae6bae90} {Suffrage:Voter ID:server-13f348d1-8e65-864d-3378-178df005bf6d Address:13f348d1-8e65-864d-3378-178df005bf6d}]"
testLoggerAdapter verbose: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.295Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-31a1b4e1-a672-c703-b03b-4730ec8b4a57 Address:31a1b4e1-a672-c703-b03b-4730ec8b4a57} {Suffrage:Voter ID:server-8176d594-816f-b691-12da-1cf5ae6bae90 Address:8176d594-816f-b691-12da-1cf5ae6bae90} {Suffrage:Voter ID:server-13f348d1-8e65-864d-3378-178df005bf6d Address:13f348d1-8e65-864d-3378-178df005bf6d}]"
    testing.go:149: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.295Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-31a1b4e1-a672-c703-b03b-4730ec8b4a57 Address:31a1b4e1-a672-c703-b03b-4730ec8b4a57} {Suffrage:Voter ID:server-8176d594-816f-b691-12da-1cf5ae6bae90 Address:8176d594-816f-b691-12da-1cf5ae6bae90} {Suffrage:Voter ID:server-13f348d1-8e65-864d-3378-178df005bf6d Address:13f348d1-8e65-864d-3378-178df005bf6d}]"
testLoggerAdapter verbose: server-13f348d1-8e65-864d-3378-178df005bf6d: 2024-01-08T00:31:02.295Z [INFO]  server-13f348d1-8e65-864d-3378-178df005bf6d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-31a1b4e1-a672-c703-b03b-4730ec8b4a57 Address:31a1b4e1-a672-c703-b03b-4730ec8b4a57} {Suffrage:Voter ID:server-8176d594-816f-b691-12da-1cf5ae6bae90 Address:8176d594-816f-b691-12da-1cf5ae6bae90} {Suffrage:Voter ID:server-13f348d1-8e65-864d-3378-178df005bf6d Address:13f348d1-8e65-864d-3378-178df005bf6d}]"
    testing.go:149: server-13f348d1-8e65-864d-3378-178df005bf6d: 2024-01-08T00:31:02.295Z [INFO]  server-13f348d1-8e65-864d-3378-178df005bf6d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-31a1b4e1-a672-c703-b03b-4730ec8b4a57 Address:31a1b4e1-a672-c703-b03b-4730ec8b4a57} {Suffrage:Voter ID:server-8176d594-816f-b691-12da-1cf5ae6bae90 Address:8176d594-816f-b691-12da-1cf5ae6bae90} {Suffrage:Voter ID:server-13f348d1-8e65-864d-3378-178df005bf6d Address:13f348d1-8e65-864d-3378-178df005bf6d}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:02.295Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:02.295Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.295Z [INFO]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: entering follower state: follower="Node at 31a1b4e1-a672-c703-b03b-4730ec8b4a57 [Follower]" leader=
    testing.go:149: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.295Z [INFO]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: entering follower state: follower="Node at 31a1b4e1-a672-c703-b03b-4730ec8b4a57 [Follower]" leader=
testLoggerAdapter verbose: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.295Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: entering follower state: follower="Node at 8176d594-816f-b691-12da-1cf5ae6bae90 [Follower]" leader=
    testing.go:149: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.295Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: entering follower state: follower="Node at 8176d594-816f-b691-12da-1cf5ae6bae90 [Follower]" leader=
testLoggerAdapter verbose: server-13f348d1-8e65-864d-3378-178df005bf6d: 2024-01-08T00:31:02.296Z [INFO]  server-13f348d1-8e65-864d-3378-178df005bf6d: entering follower state: follower="Node at 13f348d1-8e65-864d-3378-178df005bf6d [Follower]" leader=
    testing.go:149: server-13f348d1-8e65-864d-3378-178df005bf6d: 2024-01-08T00:31:02.296Z [INFO]  server-13f348d1-8e65-864d-3378-178df005bf6d: entering follower state: follower="Node at 13f348d1-8e65-864d-3378-178df005bf6d [Follower]" leader=
testLoggerAdapter verbose: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.349Z [WARN]  server-8176d594-816f-b691-12da-1cf5ae6bae90: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.349Z [WARN]  server-8176d594-816f-b691-12da-1cf5ae6bae90: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.349Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: entering candidate state: node="Node at 8176d594-816f-b691-12da-1cf5ae6bae90 [Candidate]" term=2
    testing.go:149: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.349Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: entering candidate state: node="Node at 8176d594-816f-b691-12da-1cf5ae6bae90 [Candidate]" term=2
testLoggerAdapter verbose: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.349Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: election won: tally=2
    testing.go:149: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.349Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: election won: tally=2
testLoggerAdapter verbose: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.349Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: entering leader state: leader="Node at 8176d594-816f-b691-12da-1cf5ae6bae90 [Leader]"
    testing.go:149: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.349Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: entering leader state: leader="Node at 8176d594-816f-b691-12da-1cf5ae6bae90 [Leader]"
testLoggerAdapter verbose: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.350Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: added peer, starting replication: peer=server-31a1b4e1-a672-c703-b03b-4730ec8b4a57
    testing.go:149: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.350Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: added peer, starting replication: peer=server-31a1b4e1-a672-c703-b03b-4730ec8b4a57
testLoggerAdapter verbose: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.350Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: added peer, starting replication: peer=server-13f348d1-8e65-864d-3378-178df005bf6d
    testing.go:149: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.350Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: added peer, starting replication: peer=server-13f348d1-8e65-864d-3378-178df005bf6d
testLoggerAdapter verbose: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.350Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: pipelining replication: peer="{Voter server-13f348d1-8e65-864d-3378-178df005bf6d 13f348d1-8e65-864d-3378-178df005bf6d}"
    testing.go:149: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.350Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: pipelining replication: peer="{Voter server-13f348d1-8e65-864d-3378-178df005bf6d 13f348d1-8e65-864d-3378-178df005bf6d}"
testLoggerAdapter verbose: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.350Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: pipelining replication: peer="{Voter server-31a1b4e1-a672-c703-b03b-4730ec8b4a57 31a1b4e1-a672-c703-b03b-4730ec8b4a57}"
    testing.go:149: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.350Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: pipelining replication: peer="{Voter server-31a1b4e1-a672-c703-b03b-4730ec8b4a57 31a1b4e1-a672-c703-b03b-4730ec8b4a57}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:02.455Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:02.350359518 +0000 UTC m=+8.397248609 (1 nodes), 54.471426ms from start of poll, 54.896596ms from cluster start. Timeout at 2024-01-08 00:31:02.455839037 +0000 UTC m=+8.502728160, 105.479551ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:02.455Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:02.350359518 +0000 UTC m=+8.397248609 (1 nodes), 54.471426ms from start of poll, 54.896596ms from cluster start. Timeout at 2024-01-08 00:31:02.455839037 +0000 UTC m=+8.502728160, 105.479551ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:02.461Z [INFO]  starting stability test: raft-state=Follower
    testing.go:149: cluster: 2024-01-08T00:31:02.461Z [INFO]  starting stability test: raft-state=Follower
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:02.566Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:02.461600292 +0000 UTC m=+8.508489380 (2 nodes), 2.473µs from start of poll, 166.137367ms from cluster start. Timeout at 2024-01-08 00:31:02.566645189 +0000 UTC m=+8.613534291, 105.044911ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:02.566Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:02.461600292 +0000 UTC m=+8.508489380 (2 nodes), 2.473µs from start of poll, 166.137367ms from cluster start. Timeout at 2024-01-08 00:31:02.566645189 +0000 UTC m=+8.613534291, 105.044911ms after stability
testLoggerAdapter verbose: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.566Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: updating configuration: command=RemoveServer server-id=server-8176d594-816f-b691-12da-1cf5ae6bae90 server-addr= servers="[{Suffrage:Voter ID:server-31a1b4e1-a672-c703-b03b-4730ec8b4a57 Address:31a1b4e1-a672-c703-b03b-4730ec8b4a57} {Suffrage:Voter ID:server-13f348d1-8e65-864d-3378-178df005bf6d Address:13f348d1-8e65-864d-3378-178df005bf6d}]"
    testing.go:149: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.566Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: updating configuration: command=RemoveServer server-id=server-8176d594-816f-b691-12da-1cf5ae6bae90 server-addr= servers="[{Suffrage:Voter ID:server-31a1b4e1-a672-c703-b03b-4730ec8b4a57 Address:31a1b4e1-a672-c703-b03b-4730ec8b4a57} {Suffrage:Voter ID:server-13f348d1-8e65-864d-3378-178df005bf6d Address:13f348d1-8e65-864d-3378-178df005bf6d}]"
testLoggerAdapter verbose: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.566Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: removed ourself, shutting down
    testing.go:149: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.566Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: removed ourself, shutting down
testLoggerAdapter verbose: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.567Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: aborting pipeline replication: peer="{Voter server-13f348d1-8e65-864d-3378-178df005bf6d 13f348d1-8e65-864d-3378-178df005bf6d}"
    testing.go:149: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.567Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: aborting pipeline replication: peer="{Voter server-13f348d1-8e65-864d-3378-178df005bf6d 13f348d1-8e65-864d-3378-178df005bf6d}"
testLoggerAdapter verbose: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.567Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: aborting pipeline replication: peer="{Voter server-31a1b4e1-a672-c703-b03b-4730ec8b4a57 31a1b4e1-a672-c703-b03b-4730ec8b4a57}"
    testing.go:149: server-8176d594-816f-b691-12da-1cf5ae6bae90: 2024-01-08T00:31:02.567Z [INFO]  server-8176d594-816f-b691-12da-1cf5ae6bae90: aborting pipeline replication: peer="{Voter server-31a1b4e1-a672-c703-b03b-4730ec8b4a57 31a1b4e1-a672-c703-b03b-4730ec8b4a57}"
testLoggerAdapter verbose: server-13f348d1-8e65-864d-3378-178df005bf6d: 2024-01-08T00:31:02.629Z [WARN]  server-13f348d1-8e65-864d-3378-178df005bf6d: heartbeat timeout reached, starting election: last-leader=8176d594-816f-b691-12da-1cf5ae6bae90
    testing.go:149: server-13f348d1-8e65-864d-3378-178df005bf6d: 2024-01-08T00:31:02.629Z [WARN]  server-13f348d1-8e65-864d-3378-178df005bf6d: heartbeat timeout reached, starting election: last-leader=8176d594-816f-b691-12da-1cf5ae6bae90
testLoggerAdapter verbose: server-13f348d1-8e65-864d-3378-178df005bf6d: 2024-01-08T00:31:02.629Z [INFO]  server-13f348d1-8e65-864d-3378-178df005bf6d: entering candidate state: node="Node at 13f348d1-8e65-864d-3378-178df005bf6d [Candidate]" term=3
    testing.go:149: server-13f348d1-8e65-864d-3378-178df005bf6d: 2024-01-08T00:31:02.629Z [INFO]  server-13f348d1-8e65-864d-3378-178df005bf6d: entering candidate state: node="Node at 13f348d1-8e65-864d-3378-178df005bf6d [Candidate]" term=3
testLoggerAdapter verbose: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.629Z [WARN]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: rejecting vote request since we have a leader: from=13f348d1-8e65-864d-3378-178df005bf6d leader=8176d594-816f-b691-12da-1cf5ae6bae90
    testing.go:149: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.629Z [WARN]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: rejecting vote request since we have a leader: from=13f348d1-8e65-864d-3378-178df005bf6d leader=8176d594-816f-b691-12da-1cf5ae6bae90
testLoggerAdapter verbose: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.657Z [WARN]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: heartbeat timeout reached, starting election: last-leader=8176d594-816f-b691-12da-1cf5ae6bae90
    testing.go:149: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.657Z [WARN]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: heartbeat timeout reached, starting election: last-leader=8176d594-816f-b691-12da-1cf5ae6bae90
testLoggerAdapter verbose: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.657Z [INFO]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: entering candidate state: node="Node at 31a1b4e1-a672-c703-b03b-4730ec8b4a57 [Candidate]" term=3
    testing.go:149: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.657Z [INFO]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: entering candidate state: node="Node at 31a1b4e1-a672-c703-b03b-4730ec8b4a57 [Candidate]" term=3
testLoggerAdapter verbose: server-13f348d1-8e65-864d-3378-178df005bf6d: 2024-01-08T00:31:02.657Z [INFO]  server-13f348d1-8e65-864d-3378-178df005bf6d: duplicate requestVote for same term: term=3
    testing.go:149: server-13f348d1-8e65-864d-3378-178df005bf6d: 2024-01-08T00:31:02.657Z [INFO]  server-13f348d1-8e65-864d-3378-178df005bf6d: duplicate requestVote for same term: term=3
testLoggerAdapter verbose: server-13f348d1-8e65-864d-3378-178df005bf6d: 2024-01-08T00:31:02.728Z [WARN]  server-13f348d1-8e65-864d-3378-178df005bf6d: Election timeout reached, restarting election
    testing.go:149: server-13f348d1-8e65-864d-3378-178df005bf6d: 2024-01-08T00:31:02.728Z [WARN]  server-13f348d1-8e65-864d-3378-178df005bf6d: Election timeout reached, restarting election
testLoggerAdapter verbose: server-13f348d1-8e65-864d-3378-178df005bf6d: 2024-01-08T00:31:02.728Z [INFO]  server-13f348d1-8e65-864d-3378-178df005bf6d: entering candidate state: node="Node at 13f348d1-8e65-864d-3378-178df005bf6d [Candidate]" term=4
    testing.go:149: server-13f348d1-8e65-864d-3378-178df005bf6d: 2024-01-08T00:31:02.728Z [INFO]  server-13f348d1-8e65-864d-3378-178df005bf6d: entering candidate state: node="Node at 13f348d1-8e65-864d-3378-178df005bf6d [Candidate]" term=4
testLoggerAdapter verbose: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.728Z [WARN]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: Election timeout reached, restarting election
    testing.go:149: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.728Z [WARN]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: Election timeout reached, restarting election
testLoggerAdapter verbose: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.729Z [INFO]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: entering candidate state: node="Node at 31a1b4e1-a672-c703-b03b-4730ec8b4a57 [Candidate]" term=4
    testing.go:149: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.729Z [INFO]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: entering candidate state: node="Node at 31a1b4e1-a672-c703-b03b-4730ec8b4a57 [Candidate]" term=4
testLoggerAdapter verbose: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.729Z [INFO]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: duplicate requestVote for same term: term=4
    testing.go:149: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.729Z [INFO]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: duplicate requestVote for same term: term=4
testLoggerAdapter verbose: server-13f348d1-8e65-864d-3378-178df005bf6d: 2024-01-08T00:31:02.729Z [INFO]  server-13f348d1-8e65-864d-3378-178df005bf6d: duplicate requestVote for same term: term=4
    testing.go:149: server-13f348d1-8e65-864d-3378-178df005bf6d: 2024-01-08T00:31:02.729Z [INFO]  server-13f348d1-8e65-864d-3378-178df005bf6d: duplicate requestVote for same term: term=4
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:02.778Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:02.778Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.804Z [WARN]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: Election timeout reached, restarting election
    testing.go:149: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.804Z [WARN]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: Election timeout reached, restarting election
testLoggerAdapter verbose: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.804Z [INFO]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: entering candidate state: node="Node at 31a1b4e1-a672-c703-b03b-4730ec8b4a57 [Candidate]" term=5
    testing.go:149: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.804Z [INFO]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: entering candidate state: node="Node at 31a1b4e1-a672-c703-b03b-4730ec8b4a57 [Candidate]" term=5
testLoggerAdapter verbose: server-13f348d1-8e65-864d-3378-178df005bf6d: 2024-01-08T00:31:02.804Z [INFO]  server-13f348d1-8e65-864d-3378-178df005bf6d: entering follower state: follower="Node at 13f348d1-8e65-864d-3378-178df005bf6d [Follower]" leader=
    testing.go:149: server-13f348d1-8e65-864d-3378-178df005bf6d: 2024-01-08T00:31:02.804Z [INFO]  server-13f348d1-8e65-864d-3378-178df005bf6d: entering follower state: follower="Node at 13f348d1-8e65-864d-3378-178df005bf6d [Follower]" leader=
testLoggerAdapter verbose: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.804Z [INFO]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: election won: tally=2
    testing.go:149: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.804Z [INFO]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: election won: tally=2
testLoggerAdapter verbose: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.804Z [INFO]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: entering leader state: leader="Node at 31a1b4e1-a672-c703-b03b-4730ec8b4a57 [Leader]"
    testing.go:149: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.804Z [INFO]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: entering leader state: leader="Node at 31a1b4e1-a672-c703-b03b-4730ec8b4a57 [Leader]"
testLoggerAdapter verbose: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.804Z [INFO]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: added peer, starting replication: peer=server-13f348d1-8e65-864d-3378-178df005bf6d
    testing.go:149: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.804Z [INFO]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: added peer, starting replication: peer=server-13f348d1-8e65-864d-3378-178df005bf6d
testLoggerAdapter verbose: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.804Z [INFO]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: pipelining replication: peer="{Voter server-13f348d1-8e65-864d-3378-178df005bf6d 13f348d1-8e65-864d-3378-178df005bf6d}"
    testing.go:149: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.804Z [INFO]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: pipelining replication: peer="{Voter server-13f348d1-8e65-864d-3378-178df005bf6d 13f348d1-8e65-864d-3378-178df005bf6d}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:02.909Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:02.804549042 +0000 UTC m=+8.851438128 (1 nodes), 25.935905ms from start of poll, 509.086115ms from cluster start. Timeout at 2024-01-08 00:31:02.909691093 +0000 UTC m=+8.956580165, 105.142037ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:02.909Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:02.804549042 +0000 UTC m=+8.851438128 (1 nodes), 25.935905ms from start of poll, 509.086115ms from cluster start. Timeout at 2024-01-08 00:31:02.909691093 +0000 UTC m=+8.956580165, 105.142037ms after stability
testLoggerAdapter verbose: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.909Z [INFO]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: aborting pipeline replication: peer="{Voter server-13f348d1-8e65-864d-3378-178df005bf6d 13f348d1-8e65-864d-3378-178df005bf6d}"
    testing.go:149: server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: 2024-01-08T00:31:02.909Z [INFO]  server-31a1b4e1-a672-c703-b03b-4730ec8b4a57: aborting pipeline replication: peer="{Voter server-13f348d1-8e65-864d-3378-178df005bf6d 13f348d1-8e65-864d-3378-178df005bf6d}"
--- PASS: TestRaft_RemoveLeader (0.62s)
=== RUN   TestRaft_RemoveLeader_NoShutdown
testLoggerAdapter verbose: server-c8916d4d-c692-c08c-342f-b49dac6d77de: 2024-01-08T00:31:02.911Z [INFO]  server-c8916d4d-c692-c08c-342f-b49dac6d77de: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c8916d4d-c692-c08c-342f-b49dac6d77de Address:c8916d4d-c692-c08c-342f-b49dac6d77de} {Suffrage:Voter ID:server-ba02d1da-3554-f86d-33cd-a66d1e967adb Address:ba02d1da-3554-f86d-33cd-a66d1e967adb} {Suffrage:Voter ID:server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61 Address:5ea0aa5b-efe8-e78c-befc-caf4b98dfd61}]"
    testing.go:149: server-c8916d4d-c692-c08c-342f-b49dac6d77de: 2024-01-08T00:31:02.911Z [INFO]  server-c8916d4d-c692-c08c-342f-b49dac6d77de: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c8916d4d-c692-c08c-342f-b49dac6d77de Address:c8916d4d-c692-c08c-342f-b49dac6d77de} {Suffrage:Voter ID:server-ba02d1da-3554-f86d-33cd-a66d1e967adb Address:ba02d1da-3554-f86d-33cd-a66d1e967adb} {Suffrage:Voter ID:server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61 Address:5ea0aa5b-efe8-e78c-befc-caf4b98dfd61}]"
testLoggerAdapter verbose: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:02.911Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c8916d4d-c692-c08c-342f-b49dac6d77de Address:c8916d4d-c692-c08c-342f-b49dac6d77de} {Suffrage:Voter ID:server-ba02d1da-3554-f86d-33cd-a66d1e967adb Address:ba02d1da-3554-f86d-33cd-a66d1e967adb} {Suffrage:Voter ID:server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61 Address:5ea0aa5b-efe8-e78c-befc-caf4b98dfd61}]"
    testing.go:149: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:02.911Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c8916d4d-c692-c08c-342f-b49dac6d77de Address:c8916d4d-c692-c08c-342f-b49dac6d77de} {Suffrage:Voter ID:server-ba02d1da-3554-f86d-33cd-a66d1e967adb Address:ba02d1da-3554-f86d-33cd-a66d1e967adb} {Suffrage:Voter ID:server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61 Address:5ea0aa5b-efe8-e78c-befc-caf4b98dfd61}]"
testLoggerAdapter verbose: server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: 2024-01-08T00:31:02.911Z [INFO]  server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c8916d4d-c692-c08c-342f-b49dac6d77de Address:c8916d4d-c692-c08c-342f-b49dac6d77de} {Suffrage:Voter ID:server-ba02d1da-3554-f86d-33cd-a66d1e967adb Address:ba02d1da-3554-f86d-33cd-a66d1e967adb} {Suffrage:Voter ID:server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61 Address:5ea0aa5b-efe8-e78c-befc-caf4b98dfd61}]"
    testing.go:149: server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: 2024-01-08T00:31:02.911Z [INFO]  server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-c8916d4d-c692-c08c-342f-b49dac6d77de Address:c8916d4d-c692-c08c-342f-b49dac6d77de} {Suffrage:Voter ID:server-ba02d1da-3554-f86d-33cd-a66d1e967adb Address:ba02d1da-3554-f86d-33cd-a66d1e967adb} {Suffrage:Voter ID:server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61 Address:5ea0aa5b-efe8-e78c-befc-caf4b98dfd61}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:02.911Z [INFO]  starting stability test: raft-state=Follower
    testing.go:149: cluster: 2024-01-08T00:31:02.911Z [INFO]  starting stability test: raft-state=Follower
testLoggerAdapter verbose: server-c8916d4d-c692-c08c-342f-b49dac6d77de: 2024-01-08T00:31:02.911Z [INFO]  server-c8916d4d-c692-c08c-342f-b49dac6d77de: entering follower state: follower="Node at c8916d4d-c692-c08c-342f-b49dac6d77de [Follower]" leader=
    testing.go:149: server-c8916d4d-c692-c08c-342f-b49dac6d77de: 2024-01-08T00:31:02.911Z [INFO]  server-c8916d4d-c692-c08c-342f-b49dac6d77de: entering follower state: follower="Node at c8916d4d-c692-c08c-342f-b49dac6d77de [Follower]" leader=
testLoggerAdapter verbose: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:02.911Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: entering follower state: follower="Node at ba02d1da-3554-f86d-33cd-a66d1e967adb [Follower]" leader=
    testing.go:149: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:02.911Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: entering follower state: follower="Node at ba02d1da-3554-f86d-33cd-a66d1e967adb [Follower]" leader=
testLoggerAdapter verbose: server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: 2024-01-08T00:31:02.911Z [INFO]  server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: entering follower state: follower="Node at 5ea0aa5b-efe8-e78c-befc-caf4b98dfd61 [Follower]" leader=
    testing.go:149: server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: 2024-01-08T00:31:02.911Z [INFO]  server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: entering follower state: follower="Node at 5ea0aa5b-efe8-e78c-befc-caf4b98dfd61 [Follower]" leader=
testLoggerAdapter verbose: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:02.979Z [WARN]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:02.979Z [WARN]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:02.979Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: entering candidate state: node="Node at ba02d1da-3554-f86d-33cd-a66d1e967adb [Candidate]" term=2
    testing.go:149: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:02.979Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: entering candidate state: node="Node at ba02d1da-3554-f86d-33cd-a66d1e967adb [Candidate]" term=2
testLoggerAdapter verbose: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:02.979Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: election won: tally=2
    testing.go:149: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:02.979Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: election won: tally=2
testLoggerAdapter verbose: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:02.979Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: entering leader state: leader="Node at ba02d1da-3554-f86d-33cd-a66d1e967adb [Leader]"
    testing.go:149: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:02.979Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: entering leader state: leader="Node at ba02d1da-3554-f86d-33cd-a66d1e967adb [Leader]"
testLoggerAdapter verbose: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:02.979Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: added peer, starting replication: peer=server-c8916d4d-c692-c08c-342f-b49dac6d77de
    testing.go:149: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:02.979Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: added peer, starting replication: peer=server-c8916d4d-c692-c08c-342f-b49dac6d77de
testLoggerAdapter verbose: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:02.979Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: added peer, starting replication: peer=server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61
    testing.go:149: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:02.979Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: added peer, starting replication: peer=server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61
testLoggerAdapter verbose: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:02.979Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: pipelining replication: peer="{Voter server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61 5ea0aa5b-efe8-e78c-befc-caf4b98dfd61}"
    testing.go:149: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:02.979Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: pipelining replication: peer="{Voter server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61 5ea0aa5b-efe8-e78c-befc-caf4b98dfd61}"
testLoggerAdapter verbose: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:02.979Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: pipelining replication: peer="{Voter server-c8916d4d-c692-c08c-342f-b49dac6d77de c8916d4d-c692-c08c-342f-b49dac6d77de}"
    testing.go:149: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:02.979Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: pipelining replication: peer="{Voter server-c8916d4d-c692-c08c-342f-b49dac6d77de c8916d4d-c692-c08c-342f-b49dac6d77de}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:03.085Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:02.979678301 +0000 UTC m=+9.026567346 (2 nodes), 68.395986ms from start of poll, 68.767979ms from cluster start. Timeout at 2024-01-08 00:31:03.085207872 +0000 UTC m=+9.132096948, 105.529602ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:03.085Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:02.979678301 +0000 UTC m=+9.026567346 (2 nodes), 68.395986ms from start of poll, 68.767979ms from cluster start. Timeout at 2024-01-08 00:31:03.085207872 +0000 UTC m=+9.132096948, 105.529602ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:03.085Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:03.085Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:03.191Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:03.085325084 +0000 UTC m=+9.132214121 (1 nodes), 2.169µs from start of poll, 174.414754ms from cluster start. Timeout at 2024-01-08 00:31:03.19126752 +0000 UTC m=+9.238156588, 105.942467ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:03.191Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:03.085325084 +0000 UTC m=+9.132214121 (1 nodes), 2.169µs from start of poll, 174.414754ms from cluster start. Timeout at 2024-01-08 00:31:03.19126752 +0000 UTC m=+9.238156588, 105.942467ms after stability
testLoggerAdapter verbose: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:03.194Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: updating configuration: command=RemoveServer server-id=server-ba02d1da-3554-f86d-33cd-a66d1e967adb server-addr= servers="[{Suffrage:Voter ID:server-c8916d4d-c692-c08c-342f-b49dac6d77de Address:c8916d4d-c692-c08c-342f-b49dac6d77de} {Suffrage:Voter ID:server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61 Address:5ea0aa5b-efe8-e78c-befc-caf4b98dfd61}]"
    testing.go:149: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:03.194Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: updating configuration: command=RemoveServer server-id=server-ba02d1da-3554-f86d-33cd-a66d1e967adb server-addr= servers="[{Suffrage:Voter ID:server-c8916d4d-c692-c08c-342f-b49dac6d77de Address:c8916d4d-c692-c08c-342f-b49dac6d77de} {Suffrage:Voter ID:server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61 Address:5ea0aa5b-efe8-e78c-befc-caf4b98dfd61}]"
testLoggerAdapter verbose: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:03.194Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: removed ourself, transitioning to follower
    testing.go:149: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:03.194Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: removed ourself, transitioning to follower
testLoggerAdapter verbose: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:03.194Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: entering follower state: follower="Node at ba02d1da-3554-f86d-33cd-a66d1e967adb [Follower]" leader=
    testing.go:149: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:03.194Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: entering follower state: follower="Node at ba02d1da-3554-f86d-33cd-a66d1e967adb [Follower]" leader=
testLoggerAdapter verbose: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:03.194Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: aborting pipeline replication: peer="{Voter server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61 5ea0aa5b-efe8-e78c-befc-caf4b98dfd61}"
    testing.go:149: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:03.194Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: aborting pipeline replication: peer="{Voter server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61 5ea0aa5b-efe8-e78c-befc-caf4b98dfd61}"
testLoggerAdapter verbose: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:03.194Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: aborting pipeline replication: peer="{Voter server-c8916d4d-c692-c08c-342f-b49dac6d77de c8916d4d-c692-c08c-342f-b49dac6d77de}"
    testing.go:149: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:03.194Z [INFO]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: aborting pipeline replication: peer="{Voter server-c8916d4d-c692-c08c-342f-b49dac6d77de c8916d4d-c692-c08c-342f-b49dac6d77de}"
testLoggerAdapter verbose: server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: 2024-01-08T00:31:03.246Z [WARN]  server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: heartbeat timeout reached, starting election: last-leader=ba02d1da-3554-f86d-33cd-a66d1e967adb
    testing.go:149: server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: 2024-01-08T00:31:03.246Z [WARN]  server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: heartbeat timeout reached, starting election: last-leader=ba02d1da-3554-f86d-33cd-a66d1e967adb
testLoggerAdapter verbose: server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: 2024-01-08T00:31:03.246Z [INFO]  server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: entering candidate state: node="Node at 5ea0aa5b-efe8-e78c-befc-caf4b98dfd61 [Candidate]" term=3
    testing.go:149: server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: 2024-01-08T00:31:03.246Z [INFO]  server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: entering candidate state: node="Node at 5ea0aa5b-efe8-e78c-befc-caf4b98dfd61 [Candidate]" term=3
testLoggerAdapter verbose: server-c8916d4d-c692-c08c-342f-b49dac6d77de: 2024-01-08T00:31:03.246Z [WARN]  server-c8916d4d-c692-c08c-342f-b49dac6d77de: rejecting vote request since we have a leader: from=5ea0aa5b-efe8-e78c-befc-caf4b98dfd61 leader=ba02d1da-3554-f86d-33cd-a66d1e967adb
    testing.go:149: server-c8916d4d-c692-c08c-342f-b49dac6d77de: 2024-01-08T00:31:03.246Z [WARN]  server-c8916d4d-c692-c08c-342f-b49dac6d77de: rejecting vote request since we have a leader: from=5ea0aa5b-efe8-e78c-befc-caf4b98dfd61 leader=ba02d1da-3554-f86d-33cd-a66d1e967adb
testLoggerAdapter verbose: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:03.279Z [WARN]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: not part of stable configuration, aborting election
    testing.go:149: server-ba02d1da-3554-f86d-33cd-a66d1e967adb: 2024-01-08T00:31:03.279Z [WARN]  server-ba02d1da-3554-f86d-33cd-a66d1e967adb: not part of stable configuration, aborting election
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:03.300Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:03.300Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-c8916d4d-c692-c08c-342f-b49dac6d77de: 2024-01-08T00:31:03.311Z [WARN]  server-c8916d4d-c692-c08c-342f-b49dac6d77de: heartbeat timeout reached, starting election: last-leader=ba02d1da-3554-f86d-33cd-a66d1e967adb
    testing.go:149: server-c8916d4d-c692-c08c-342f-b49dac6d77de: 2024-01-08T00:31:03.311Z [WARN]  server-c8916d4d-c692-c08c-342f-b49dac6d77de: heartbeat timeout reached, starting election: last-leader=ba02d1da-3554-f86d-33cd-a66d1e967adb
testLoggerAdapter verbose: server-c8916d4d-c692-c08c-342f-b49dac6d77de: 2024-01-08T00:31:03.311Z [INFO]  server-c8916d4d-c692-c08c-342f-b49dac6d77de: entering candidate state: node="Node at c8916d4d-c692-c08c-342f-b49dac6d77de [Candidate]" term=3
    testing.go:149: server-c8916d4d-c692-c08c-342f-b49dac6d77de: 2024-01-08T00:31:03.311Z [INFO]  server-c8916d4d-c692-c08c-342f-b49dac6d77de: entering candidate state: node="Node at c8916d4d-c692-c08c-342f-b49dac6d77de [Candidate]" term=3
testLoggerAdapter verbose: server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: 2024-01-08T00:31:03.311Z [INFO]  server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: duplicate requestVote for same term: term=3
    testing.go:149: server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: 2024-01-08T00:31:03.311Z [INFO]  server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: duplicate requestVote for same term: term=3
testLoggerAdapter verbose: server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: 2024-01-08T00:31:03.333Z [WARN]  server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: Election timeout reached, restarting election
    testing.go:149: server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: 2024-01-08T00:31:03.333Z [WARN]  server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: Election timeout reached, restarting election
testLoggerAdapter verbose: server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: 2024-01-08T00:31:03.333Z [INFO]  server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: entering candidate state: node="Node at 5ea0aa5b-efe8-e78c-befc-caf4b98dfd61 [Candidate]" term=4
    testing.go:149: server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: 2024-01-08T00:31:03.333Z [INFO]  server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: entering candidate state: node="Node at 5ea0aa5b-efe8-e78c-befc-caf4b98dfd61 [Candidate]" term=4
testLoggerAdapter verbose: server-c8916d4d-c692-c08c-342f-b49dac6d77de: 2024-01-08T00:31:03.333Z [INFO]  server-c8916d4d-c692-c08c-342f-b49dac6d77de: entering follower state: follower="Node at c8916d4d-c692-c08c-342f-b49dac6d77de [Follower]" leader=
    testing.go:149: server-c8916d4d-c692-c08c-342f-b49dac6d77de: 2024-01-08T00:31:03.333Z [INFO]  server-c8916d4d-c692-c08c-342f-b49dac6d77de: entering follower state: follower="Node at c8916d4d-c692-c08c-342f-b49dac6d77de [Follower]" leader=
testLoggerAdapter verbose: server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: 2024-01-08T00:31:03.333Z [INFO]  server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: election won: tally=2
    testing.go:149: server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: 2024-01-08T00:31:03.333Z [INFO]  server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: election won: tally=2
testLoggerAdapter verbose: server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: 2024-01-08T00:31:03.333Z [INFO]  server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: entering leader state: leader="Node at 5ea0aa5b-efe8-e78c-befc-caf4b98dfd61 [Leader]"
    testing.go:149: server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: 2024-01-08T00:31:03.333Z [INFO]  server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: entering leader state: leader="Node at 5ea0aa5b-efe8-e78c-befc-caf4b98dfd61 [Leader]"
testLoggerAdapter verbose: server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: 2024-01-08T00:31:03.333Z [INFO]  server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: added peer, starting replication: peer=server-c8916d4d-c692-c08c-342f-b49dac6d77de
    testing.go:149: server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: 2024-01-08T00:31:03.333Z [INFO]  server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: added peer, starting replication: peer=server-c8916d4d-c692-c08c-342f-b49dac6d77de
testLoggerAdapter verbose: server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: 2024-01-08T00:31:03.333Z [INFO]  server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: pipelining replication: peer="{Voter server-c8916d4d-c692-c08c-342f-b49dac6d77de c8916d4d-c692-c08c-342f-b49dac6d77de}"
    testing.go:149: server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: 2024-01-08T00:31:03.333Z [INFO]  server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: pipelining replication: peer="{Voter server-c8916d4d-c692-c08c-342f-b49dac6d77de c8916d4d-c692-c08c-342f-b49dac6d77de}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:03.439Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:03.333891048 +0000 UTC m=+9.380780083 (1 nodes), 33.229502ms from start of poll, 422.980716ms from cluster start. Timeout at 2024-01-08 00:31:03.43926069 +0000 UTC m=+9.486149748, 105.369665ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:03.439Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:03.333891048 +0000 UTC m=+9.380780083 (1 nodes), 33.229502ms from start of poll, 422.980716ms from cluster start. Timeout at 2024-01-08 00:31:03.43926069 +0000 UTC m=+9.486149748, 105.369665ms after stability
testLoggerAdapter verbose: server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: 2024-01-08T00:31:03.545Z [INFO]  server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: aborting pipeline replication: peer="{Voter server-c8916d4d-c692-c08c-342f-b49dac6d77de c8916d4d-c692-c08c-342f-b49dac6d77de}"
    testing.go:149: server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: 2024-01-08T00:31:03.545Z [INFO]  server-5ea0aa5b-efe8-e78c-befc-caf4b98dfd61: aborting pipeline replication: peer="{Voter server-c8916d4d-c692-c08c-342f-b49dac6d77de c8916d4d-c692-c08c-342f-b49dac6d77de}"
--- PASS: TestRaft_RemoveLeader_NoShutdown (0.64s)
=== RUN   TestRaft_RemoveFollower_SplitCluster
testLoggerAdapter verbose: server-1a498f52-a6b7-3194-596f-5b8daeed77c9: 2024-01-08T00:31:03.546Z [INFO]  server-1a498f52-a6b7-3194-596f-5b8daeed77c9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1a498f52-a6b7-3194-596f-5b8daeed77c9 Address:1a498f52-a6b7-3194-596f-5b8daeed77c9} {Suffrage:Voter ID:server-04abdece-8234-ac33-4ead-679b17f7c194 Address:04abdece-8234-ac33-4ead-679b17f7c194} {Suffrage:Voter ID:server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826 Address:e5cb02de-0e10-6cc3-f95e-83e39bfbe826} {Suffrage:Voter ID:server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e Address:4d7088ed-c8b2-7193-7741-0cfc60f25d7e}]"
    testing.go:149: server-1a498f52-a6b7-3194-596f-5b8daeed77c9: 2024-01-08T00:31:03.546Z [INFO]  server-1a498f52-a6b7-3194-596f-5b8daeed77c9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1a498f52-a6b7-3194-596f-5b8daeed77c9 Address:1a498f52-a6b7-3194-596f-5b8daeed77c9} {Suffrage:Voter ID:server-04abdece-8234-ac33-4ead-679b17f7c194 Address:04abdece-8234-ac33-4ead-679b17f7c194} {Suffrage:Voter ID:server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826 Address:e5cb02de-0e10-6cc3-f95e-83e39bfbe826} {Suffrage:Voter ID:server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e Address:4d7088ed-c8b2-7193-7741-0cfc60f25d7e}]"
testLoggerAdapter verbose: server-04abdece-8234-ac33-4ead-679b17f7c194: 2024-01-08T00:31:03.546Z [INFO]  server-04abdece-8234-ac33-4ead-679b17f7c194: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1a498f52-a6b7-3194-596f-5b8daeed77c9 Address:1a498f52-a6b7-3194-596f-5b8daeed77c9} {Suffrage:Voter ID:server-04abdece-8234-ac33-4ead-679b17f7c194 Address:04abdece-8234-ac33-4ead-679b17f7c194} {Suffrage:Voter ID:server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826 Address:e5cb02de-0e10-6cc3-f95e-83e39bfbe826} {Suffrage:Voter ID:server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e Address:4d7088ed-c8b2-7193-7741-0cfc60f25d7e}]"
    testing.go:149: server-04abdece-8234-ac33-4ead-679b17f7c194: 2024-01-08T00:31:03.546Z [INFO]  server-04abdece-8234-ac33-4ead-679b17f7c194: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1a498f52-a6b7-3194-596f-5b8daeed77c9 Address:1a498f52-a6b7-3194-596f-5b8daeed77c9} {Suffrage:Voter ID:server-04abdece-8234-ac33-4ead-679b17f7c194 Address:04abdece-8234-ac33-4ead-679b17f7c194} {Suffrage:Voter ID:server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826 Address:e5cb02de-0e10-6cc3-f95e-83e39bfbe826} {Suffrage:Voter ID:server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e Address:4d7088ed-c8b2-7193-7741-0cfc60f25d7e}]"
testLoggerAdapter verbose: server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826: 2024-01-08T00:31:03.546Z [INFO]  server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1a498f52-a6b7-3194-596f-5b8daeed77c9 Address:1a498f52-a6b7-3194-596f-5b8daeed77c9} {Suffrage:Voter ID:server-04abdece-8234-ac33-4ead-679b17f7c194 Address:04abdece-8234-ac33-4ead-679b17f7c194} {Suffrage:Voter ID:server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826 Address:e5cb02de-0e10-6cc3-f95e-83e39bfbe826} {Suffrage:Voter ID:server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e Address:4d7088ed-c8b2-7193-7741-0cfc60f25d7e}]"
    testing.go:149: server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826: 2024-01-08T00:31:03.546Z [INFO]  server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1a498f52-a6b7-3194-596f-5b8daeed77c9 Address:1a498f52-a6b7-3194-596f-5b8daeed77c9} {Suffrage:Voter ID:server-04abdece-8234-ac33-4ead-679b17f7c194 Address:04abdece-8234-ac33-4ead-679b17f7c194} {Suffrage:Voter ID:server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826 Address:e5cb02de-0e10-6cc3-f95e-83e39bfbe826} {Suffrage:Voter ID:server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e Address:4d7088ed-c8b2-7193-7741-0cfc60f25d7e}]"
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.546Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1a498f52-a6b7-3194-596f-5b8daeed77c9 Address:1a498f52-a6b7-3194-596f-5b8daeed77c9} {Suffrage:Voter ID:server-04abdece-8234-ac33-4ead-679b17f7c194 Address:04abdece-8234-ac33-4ead-679b17f7c194} {Suffrage:Voter ID:server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826 Address:e5cb02de-0e10-6cc3-f95e-83e39bfbe826} {Suffrage:Voter ID:server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e Address:4d7088ed-c8b2-7193-7741-0cfc60f25d7e}]"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.546Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-1a498f52-a6b7-3194-596f-5b8daeed77c9 Address:1a498f52-a6b7-3194-596f-5b8daeed77c9} {Suffrage:Voter ID:server-04abdece-8234-ac33-4ead-679b17f7c194 Address:04abdece-8234-ac33-4ead-679b17f7c194} {Suffrage:Voter ID:server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826 Address:e5cb02de-0e10-6cc3-f95e-83e39bfbe826} {Suffrage:Voter ID:server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e Address:4d7088ed-c8b2-7193-7741-0cfc60f25d7e}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:03.546Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:03.546Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-1a498f52-a6b7-3194-596f-5b8daeed77c9: 2024-01-08T00:31:03.547Z [INFO]  server-1a498f52-a6b7-3194-596f-5b8daeed77c9: entering follower state: follower="Node at 1a498f52-a6b7-3194-596f-5b8daeed77c9 [Follower]" leader=
    testing.go:149: server-1a498f52-a6b7-3194-596f-5b8daeed77c9: 2024-01-08T00:31:03.547Z [INFO]  server-1a498f52-a6b7-3194-596f-5b8daeed77c9: entering follower state: follower="Node at 1a498f52-a6b7-3194-596f-5b8daeed77c9 [Follower]" leader=
testLoggerAdapter verbose: server-04abdece-8234-ac33-4ead-679b17f7c194: 2024-01-08T00:31:03.547Z [INFO]  server-04abdece-8234-ac33-4ead-679b17f7c194: entering follower state: follower="Node at 04abdece-8234-ac33-4ead-679b17f7c194 [Follower]" leader=
    testing.go:149: server-04abdece-8234-ac33-4ead-679b17f7c194: 2024-01-08T00:31:03.547Z [INFO]  server-04abdece-8234-ac33-4ead-679b17f7c194: entering follower state: follower="Node at 04abdece-8234-ac33-4ead-679b17f7c194 [Follower]" leader=
testLoggerAdapter verbose: server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826: 2024-01-08T00:31:03.547Z [INFO]  server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826: entering follower state: follower="Node at e5cb02de-0e10-6cc3-f95e-83e39bfbe826 [Follower]" leader=
    testing.go:149: server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826: 2024-01-08T00:31:03.547Z [INFO]  server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826: entering follower state: follower="Node at e5cb02de-0e10-6cc3-f95e-83e39bfbe826 [Follower]" leader=
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.547Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: entering follower state: follower="Node at 4d7088ed-c8b2-7193-7741-0cfc60f25d7e [Follower]" leader=
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.547Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: entering follower state: follower="Node at 4d7088ed-c8b2-7193-7741-0cfc60f25d7e [Follower]" leader=
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.599Z [WARN]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.599Z [WARN]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.599Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: entering candidate state: node="Node at 4d7088ed-c8b2-7193-7741-0cfc60f25d7e [Candidate]" term=2
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.599Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: entering candidate state: node="Node at 4d7088ed-c8b2-7193-7741-0cfc60f25d7e [Candidate]" term=2
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.599Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: election won: tally=3
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.599Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: election won: tally=3
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.599Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: entering leader state: leader="Node at 4d7088ed-c8b2-7193-7741-0cfc60f25d7e [Leader]"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.599Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: entering leader state: leader="Node at 4d7088ed-c8b2-7193-7741-0cfc60f25d7e [Leader]"
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.599Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: added peer, starting replication: peer=server-1a498f52-a6b7-3194-596f-5b8daeed77c9
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.599Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: added peer, starting replication: peer=server-1a498f52-a6b7-3194-596f-5b8daeed77c9
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.599Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: added peer, starting replication: peer=server-04abdece-8234-ac33-4ead-679b17f7c194
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.599Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: added peer, starting replication: peer=server-04abdece-8234-ac33-4ead-679b17f7c194
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.599Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: added peer, starting replication: peer=server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.599Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: added peer, starting replication: peer=server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.599Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: pipelining replication: peer="{Voter server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826 e5cb02de-0e10-6cc3-f95e-83e39bfbe826}"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.599Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: pipelining replication: peer="{Voter server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826 e5cb02de-0e10-6cc3-f95e-83e39bfbe826}"
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.600Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: pipelining replication: peer="{Voter server-1a498f52-a6b7-3194-596f-5b8daeed77c9 1a498f52-a6b7-3194-596f-5b8daeed77c9}"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.600Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: pipelining replication: peer="{Voter server-1a498f52-a6b7-3194-596f-5b8daeed77c9 1a498f52-a6b7-3194-596f-5b8daeed77c9}"
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.600Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: pipelining replication: peer="{Voter server-04abdece-8234-ac33-4ead-679b17f7c194 04abdece-8234-ac33-4ead-679b17f7c194}"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.600Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: pipelining replication: peer="{Voter server-04abdece-8234-ac33-4ead-679b17f7c194 04abdece-8234-ac33-4ead-679b17f7c194}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:03.705Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:03.600051891 +0000 UTC m=+9.646940930 (1 nodes), 53.052013ms from start of poll, 53.554535ms from cluster start. Timeout at 2024-01-08 00:31:03.705269581 +0000 UTC m=+9.752158732, 105.217802ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:03.705Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:03.600051891 +0000 UTC m=+9.646940930 (1 nodes), 53.052013ms from start of poll, 53.554535ms from cluster start. Timeout at 2024-01-08 00:31:03.705269581 +0000 UTC m=+9.752158732, 105.217802ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:03.811Z [INFO]  starting stability test: raft-state=Follower
    testing.go:149: cluster: 2024-01-08T00:31:03.811Z [INFO]  starting stability test: raft-state=Follower
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:03.917Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:03.811250756 +0000 UTC m=+9.858139854 (3 nodes), 2.563µs from start of poll, 264.753459ms from cluster start. Timeout at 2024-01-08 00:31:03.916979389 +0000 UTC m=+9.963868510, 105.728656ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:03.917Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:03.811250756 +0000 UTC m=+9.858139854 (3 nodes), 2.563µs from start of poll, 264.753459ms from cluster start. Timeout at 2024-01-08 00:31:03.916979389 +0000 UTC m=+9.963868510, 105.728656ms after stability
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.917Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: updating configuration: command=RemoveServer server-id=server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826 server-addr= servers="[{Suffrage:Voter ID:server-1a498f52-a6b7-3194-596f-5b8daeed77c9 Address:1a498f52-a6b7-3194-596f-5b8daeed77c9} {Suffrage:Voter ID:server-04abdece-8234-ac33-4ead-679b17f7c194 Address:04abdece-8234-ac33-4ead-679b17f7c194} {Suffrage:Voter ID:server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e Address:4d7088ed-c8b2-7193-7741-0cfc60f25d7e}]"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.917Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: updating configuration: command=RemoveServer server-id=server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826 server-addr= servers="[{Suffrage:Voter ID:server-1a498f52-a6b7-3194-596f-5b8daeed77c9 Address:1a498f52-a6b7-3194-596f-5b8daeed77c9} {Suffrage:Voter ID:server-04abdece-8234-ac33-4ead-679b17f7c194 Address:04abdece-8234-ac33-4ead-679b17f7c194} {Suffrage:Voter ID:server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e Address:4d7088ed-c8b2-7193-7741-0cfc60f25d7e}]"
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.917Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: removed peer, stopping replication: peer=server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826 last-index=3
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.917Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: removed peer, stopping replication: peer=server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826 last-index=3
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.917Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: aborting pipeline replication: peer="{Voter server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826 e5cb02de-0e10-6cc3-f95e-83e39bfbe826}"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.917Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: aborting pipeline replication: peer="{Voter server-e5cb02de-0e10-6cc3-f95e-83e39bfbe826 e5cb02de-0e10-6cc3-f95e-83e39bfbe826}"
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.917Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to pipeline appendEntries: peer="{Voter server-1a498f52-a6b7-3194-596f-5b8daeed77c9 1a498f52-a6b7-3194-596f-5b8daeed77c9}" error="append pipeline closed"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.917Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to pipeline appendEntries: peer="{Voter server-1a498f52-a6b7-3194-596f-5b8daeed77c9 1a498f52-a6b7-3194-596f-5b8daeed77c9}" error="append pipeline closed"
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.917Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: aborting pipeline replication: peer="{Voter server-1a498f52-a6b7-3194-596f-5b8daeed77c9 1a498f52-a6b7-3194-596f-5b8daeed77c9}"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.917Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: aborting pipeline replication: peer="{Voter server-1a498f52-a6b7-3194-596f-5b8daeed77c9 1a498f52-a6b7-3194-596f-5b8daeed77c9}"
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.917Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to pipeline appendEntries: peer="{Voter server-04abdece-8234-ac33-4ead-679b17f7c194 04abdece-8234-ac33-4ead-679b17f7c194}" error="append pipeline closed"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.917Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to pipeline appendEntries: peer="{Voter server-04abdece-8234-ac33-4ead-679b17f7c194 04abdece-8234-ac33-4ead-679b17f7c194}" error="append pipeline closed"
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.917Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: aborting pipeline replication: peer="{Voter server-04abdece-8234-ac33-4ead-679b17f7c194 04abdece-8234-ac33-4ead-679b17f7c194}"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.917Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: aborting pipeline replication: peer="{Voter server-04abdece-8234-ac33-4ead-679b17f7c194 04abdece-8234-ac33-4ead-679b17f7c194}"
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.917Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to heartbeat to: peer=1a498f52-a6b7-3194-596f-5b8daeed77c9 error="failed to connect to peer: 1a498f52-a6b7-3194-596f-5b8daeed77c9"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.917Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to heartbeat to: peer=1a498f52-a6b7-3194-596f-5b8daeed77c9 error="failed to connect to peer: 1a498f52-a6b7-3194-596f-5b8daeed77c9"
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.923Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to appendEntries to: peer="{Voter server-04abdece-8234-ac33-4ead-679b17f7c194 04abdece-8234-ac33-4ead-679b17f7c194}" error="failed to connect to peer: 04abdece-8234-ac33-4ead-679b17f7c194"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.923Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to appendEntries to: peer="{Voter server-04abdece-8234-ac33-4ead-679b17f7c194 04abdece-8234-ac33-4ead-679b17f7c194}" error="failed to connect to peer: 04abdece-8234-ac33-4ead-679b17f7c194"
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.924Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to appendEntries to: peer="{Voter server-1a498f52-a6b7-3194-596f-5b8daeed77c9 1a498f52-a6b7-3194-596f-5b8daeed77c9}" error="failed to connect to peer: 1a498f52-a6b7-3194-596f-5b8daeed77c9"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.924Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to appendEntries to: peer="{Voter server-1a498f52-a6b7-3194-596f-5b8daeed77c9 1a498f52-a6b7-3194-596f-5b8daeed77c9}" error="failed to connect to peer: 1a498f52-a6b7-3194-596f-5b8daeed77c9"
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.924Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to heartbeat to: peer=04abdece-8234-ac33-4ead-679b17f7c194 error="failed to connect to peer: 04abdece-8234-ac33-4ead-679b17f7c194"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.924Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to heartbeat to: peer=04abdece-8234-ac33-4ead-679b17f7c194 error="failed to connect to peer: 04abdece-8234-ac33-4ead-679b17f7c194"
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.936Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to heartbeat to: peer=1a498f52-a6b7-3194-596f-5b8daeed77c9 error="failed to connect to peer: 1a498f52-a6b7-3194-596f-5b8daeed77c9"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.936Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to heartbeat to: peer=1a498f52-a6b7-3194-596f-5b8daeed77c9 error="failed to connect to peer: 1a498f52-a6b7-3194-596f-5b8daeed77c9"
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.942Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to appendEntries to: peer="{Voter server-1a498f52-a6b7-3194-596f-5b8daeed77c9 1a498f52-a6b7-3194-596f-5b8daeed77c9}" error="failed to connect to peer: 1a498f52-a6b7-3194-596f-5b8daeed77c9"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.942Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to appendEntries to: peer="{Voter server-1a498f52-a6b7-3194-596f-5b8daeed77c9 1a498f52-a6b7-3194-596f-5b8daeed77c9}" error="failed to connect to peer: 1a498f52-a6b7-3194-596f-5b8daeed77c9"
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.942Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to heartbeat to: peer=04abdece-8234-ac33-4ead-679b17f7c194 error="failed to connect to peer: 04abdece-8234-ac33-4ead-679b17f7c194"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.942Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to heartbeat to: peer=04abdece-8234-ac33-4ead-679b17f7c194 error="failed to connect to peer: 04abdece-8234-ac33-4ead-679b17f7c194"
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.942Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to appendEntries to: peer="{Voter server-04abdece-8234-ac33-4ead-679b17f7c194 04abdece-8234-ac33-4ead-679b17f7c194}" error="failed to connect to peer: 04abdece-8234-ac33-4ead-679b17f7c194"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.942Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to appendEntries to: peer="{Voter server-04abdece-8234-ac33-4ead-679b17f7c194 04abdece-8234-ac33-4ead-679b17f7c194}" error="failed to connect to peer: 04abdece-8234-ac33-4ead-679b17f7c194"
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.954Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to heartbeat to: peer=1a498f52-a6b7-3194-596f-5b8daeed77c9 error="failed to connect to peer: 1a498f52-a6b7-3194-596f-5b8daeed77c9"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.954Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to heartbeat to: peer=1a498f52-a6b7-3194-596f-5b8daeed77c9 error="failed to connect to peer: 1a498f52-a6b7-3194-596f-5b8daeed77c9"
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.958Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to heartbeat to: peer=04abdece-8234-ac33-4ead-679b17f7c194 error="failed to connect to peer: 04abdece-8234-ac33-4ead-679b17f7c194"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.958Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to heartbeat to: peer=04abdece-8234-ac33-4ead-679b17f7c194 error="failed to connect to peer: 04abdece-8234-ac33-4ead-679b17f7c194"
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.960Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to appendEntries to: peer="{Voter server-1a498f52-a6b7-3194-596f-5b8daeed77c9 1a498f52-a6b7-3194-596f-5b8daeed77c9}" error="failed to connect to peer: 1a498f52-a6b7-3194-596f-5b8daeed77c9"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.960Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to appendEntries to: peer="{Voter server-1a498f52-a6b7-3194-596f-5b8daeed77c9 1a498f52-a6b7-3194-596f-5b8daeed77c9}" error="failed to connect to peer: 1a498f52-a6b7-3194-596f-5b8daeed77c9"
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.961Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to appendEntries to: peer="{Voter server-04abdece-8234-ac33-4ead-679b17f7c194 04abdece-8234-ac33-4ead-679b17f7c194}" error="failed to connect to peer: 04abdece-8234-ac33-4ead-679b17f7c194"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.961Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to appendEntries to: peer="{Voter server-04abdece-8234-ac33-4ead-679b17f7c194 04abdece-8234-ac33-4ead-679b17f7c194}" error="failed to connect to peer: 04abdece-8234-ac33-4ead-679b17f7c194"
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.962Z [WARN]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to contact: server-id=server-1a498f52-a6b7-3194-596f-5b8daeed77c9 time=50.938364ms
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.962Z [WARN]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to contact: server-id=server-1a498f52-a6b7-3194-596f-5b8daeed77c9 time=50.938364ms
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.973Z [WARN]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to contact: server-id=server-1a498f52-a6b7-3194-596f-5b8daeed77c9 time=61.87283ms
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.973Z [WARN]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to contact: server-id=server-1a498f52-a6b7-3194-596f-5b8daeed77c9 time=61.87283ms
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.973Z [WARN]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to contact: server-id=server-04abdece-8234-ac33-4ead-679b17f7c194 time=59.624733ms
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.973Z [WARN]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to contact: server-id=server-04abdece-8234-ac33-4ead-679b17f7c194 time=59.624733ms
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.973Z [WARN]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to contact quorum of nodes, stepping down
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.973Z [WARN]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to contact quorum of nodes, stepping down
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.973Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: entering follower state: follower="Node at 4d7088ed-c8b2-7193-7741-0cfc60f25d7e [Follower]" leader=
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.973Z [INFO]  server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: entering follower state: follower="Node at 4d7088ed-c8b2-7193-7741-0cfc60f25d7e [Follower]" leader=
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.973Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to appendEntries to: peer="{Voter server-04abdece-8234-ac33-4ead-679b17f7c194 04abdece-8234-ac33-4ead-679b17f7c194}" error="failed to connect to peer: 04abdece-8234-ac33-4ead-679b17f7c194"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.973Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to appendEntries to: peer="{Voter server-04abdece-8234-ac33-4ead-679b17f7c194 04abdece-8234-ac33-4ead-679b17f7c194}" error="failed to connect to peer: 04abdece-8234-ac33-4ead-679b17f7c194"
testLoggerAdapter verbose: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.973Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to appendEntries to: peer="{Voter server-1a498f52-a6b7-3194-596f-5b8daeed77c9 1a498f52-a6b7-3194-596f-5b8daeed77c9}" error="failed to connect to peer: 1a498f52-a6b7-3194-596f-5b8daeed77c9"
    testing.go:149: server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: 2024-01-08T00:31:03.973Z [ERROR] server-4d7088ed-c8b2-7193-7741-0cfc60f25d7e: failed to appendEntries to: peer="{Voter server-1a498f52-a6b7-3194-596f-5b8daeed77c9 1a498f52-a6b7-3194-596f-5b8daeed77c9}" error="failed to connect to peer: 1a498f52-a6b7-3194-596f-5b8daeed77c9"
--- PASS: TestRaft_RemoveFollower_SplitCluster (0.43s)
=== RUN   TestRaft_AddKnownPeer
testLoggerAdapter verbose: server-99fa282e-be8b-8862-f95a-b2ea3590ec5a: 2024-01-08T00:31:03.974Z [INFO]  server-99fa282e-be8b-8862-f95a-b2ea3590ec5a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-99fa282e-be8b-8862-f95a-b2ea3590ec5a Address:99fa282e-be8b-8862-f95a-b2ea3590ec5a} {Suffrage:Voter ID:server-a89018d7-5e6b-df40-8a5f-33cd3d806e91 Address:a89018d7-5e6b-df40-8a5f-33cd3d806e91} {Suffrage:Voter ID:server-5cb4b95b-05a4-f504-2037-19709c616ee1 Address:5cb4b95b-05a4-f504-2037-19709c616ee1}]"
    testing.go:149: server-99fa282e-be8b-8862-f95a-b2ea3590ec5a: 2024-01-08T00:31:03.974Z [INFO]  server-99fa282e-be8b-8862-f95a-b2ea3590ec5a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-99fa282e-be8b-8862-f95a-b2ea3590ec5a Address:99fa282e-be8b-8862-f95a-b2ea3590ec5a} {Suffrage:Voter ID:server-a89018d7-5e6b-df40-8a5f-33cd3d806e91 Address:a89018d7-5e6b-df40-8a5f-33cd3d806e91} {Suffrage:Voter ID:server-5cb4b95b-05a4-f504-2037-19709c616ee1 Address:5cb4b95b-05a4-f504-2037-19709c616ee1}]"
testLoggerAdapter verbose: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:03.975Z [INFO]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-99fa282e-be8b-8862-f95a-b2ea3590ec5a Address:99fa282e-be8b-8862-f95a-b2ea3590ec5a} {Suffrage:Voter ID:server-a89018d7-5e6b-df40-8a5f-33cd3d806e91 Address:a89018d7-5e6b-df40-8a5f-33cd3d806e91} {Suffrage:Voter ID:server-5cb4b95b-05a4-f504-2037-19709c616ee1 Address:5cb4b95b-05a4-f504-2037-19709c616ee1}]"
    testing.go:149: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:03.975Z [INFO]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-99fa282e-be8b-8862-f95a-b2ea3590ec5a Address:99fa282e-be8b-8862-f95a-b2ea3590ec5a} {Suffrage:Voter ID:server-a89018d7-5e6b-df40-8a5f-33cd3d806e91 Address:a89018d7-5e6b-df40-8a5f-33cd3d806e91} {Suffrage:Voter ID:server-5cb4b95b-05a4-f504-2037-19709c616ee1 Address:5cb4b95b-05a4-f504-2037-19709c616ee1}]"
testLoggerAdapter verbose: server-5cb4b95b-05a4-f504-2037-19709c616ee1: 2024-01-08T00:31:03.975Z [INFO]  server-5cb4b95b-05a4-f504-2037-19709c616ee1: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-99fa282e-be8b-8862-f95a-b2ea3590ec5a Address:99fa282e-be8b-8862-f95a-b2ea3590ec5a} {Suffrage:Voter ID:server-a89018d7-5e6b-df40-8a5f-33cd3d806e91 Address:a89018d7-5e6b-df40-8a5f-33cd3d806e91} {Suffrage:Voter ID:server-5cb4b95b-05a4-f504-2037-19709c616ee1 Address:5cb4b95b-05a4-f504-2037-19709c616ee1}]"
    testing.go:149: server-5cb4b95b-05a4-f504-2037-19709c616ee1: 2024-01-08T00:31:03.975Z [INFO]  server-5cb4b95b-05a4-f504-2037-19709c616ee1: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-99fa282e-be8b-8862-f95a-b2ea3590ec5a Address:99fa282e-be8b-8862-f95a-b2ea3590ec5a} {Suffrage:Voter ID:server-a89018d7-5e6b-df40-8a5f-33cd3d806e91 Address:a89018d7-5e6b-df40-8a5f-33cd3d806e91} {Suffrage:Voter ID:server-5cb4b95b-05a4-f504-2037-19709c616ee1 Address:5cb4b95b-05a4-f504-2037-19709c616ee1}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:03.975Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:03.975Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-99fa282e-be8b-8862-f95a-b2ea3590ec5a: 2024-01-08T00:31:03.975Z [INFO]  server-99fa282e-be8b-8862-f95a-b2ea3590ec5a: entering follower state: follower="Node at 99fa282e-be8b-8862-f95a-b2ea3590ec5a [Follower]" leader=
    testing.go:149: server-99fa282e-be8b-8862-f95a-b2ea3590ec5a: 2024-01-08T00:31:03.975Z [INFO]  server-99fa282e-be8b-8862-f95a-b2ea3590ec5a: entering follower state: follower="Node at 99fa282e-be8b-8862-f95a-b2ea3590ec5a [Follower]" leader=
testLoggerAdapter verbose: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:03.975Z [INFO]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: entering follower state: follower="Node at a89018d7-5e6b-df40-8a5f-33cd3d806e91 [Follower]" leader=
    testing.go:149: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:03.975Z [INFO]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: entering follower state: follower="Node at a89018d7-5e6b-df40-8a5f-33cd3d806e91 [Follower]" leader=
testLoggerAdapter verbose: server-5cb4b95b-05a4-f504-2037-19709c616ee1: 2024-01-08T00:31:03.975Z [INFO]  server-5cb4b95b-05a4-f504-2037-19709c616ee1: entering follower state: follower="Node at 5cb4b95b-05a4-f504-2037-19709c616ee1 [Follower]" leader=
    testing.go:149: server-5cb4b95b-05a4-f504-2037-19709c616ee1: 2024-01-08T00:31:03.975Z [INFO]  server-5cb4b95b-05a4-f504-2037-19709c616ee1: entering follower state: follower="Node at 5cb4b95b-05a4-f504-2037-19709c616ee1 [Follower]" leader=
testLoggerAdapter verbose: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:04.029Z [WARN]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:04.029Z [WARN]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:04.029Z [INFO]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: entering candidate state: node="Node at a89018d7-5e6b-df40-8a5f-33cd3d806e91 [Candidate]" term=2
    testing.go:149: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:04.029Z [INFO]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: entering candidate state: node="Node at a89018d7-5e6b-df40-8a5f-33cd3d806e91 [Candidate]" term=2
testLoggerAdapter verbose: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:04.029Z [INFO]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: election won: tally=2
    testing.go:149: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:04.029Z [INFO]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: election won: tally=2
testLoggerAdapter verbose: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:04.029Z [INFO]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: entering leader state: leader="Node at a89018d7-5e6b-df40-8a5f-33cd3d806e91 [Leader]"
    testing.go:149: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:04.029Z [INFO]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: entering leader state: leader="Node at a89018d7-5e6b-df40-8a5f-33cd3d806e91 [Leader]"
testLoggerAdapter verbose: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:04.029Z [INFO]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: added peer, starting replication: peer=server-99fa282e-be8b-8862-f95a-b2ea3590ec5a
    testing.go:149: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:04.029Z [INFO]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: added peer, starting replication: peer=server-99fa282e-be8b-8862-f95a-b2ea3590ec5a
testLoggerAdapter verbose: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:04.029Z [INFO]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: added peer, starting replication: peer=server-5cb4b95b-05a4-f504-2037-19709c616ee1
    testing.go:149: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:04.029Z [INFO]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: added peer, starting replication: peer=server-5cb4b95b-05a4-f504-2037-19709c616ee1
testLoggerAdapter verbose: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:04.030Z [INFO]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: pipelining replication: peer="{Voter server-5cb4b95b-05a4-f504-2037-19709c616ee1 5cb4b95b-05a4-f504-2037-19709c616ee1}"
    testing.go:149: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:04.030Z [INFO]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: pipelining replication: peer="{Voter server-5cb4b95b-05a4-f504-2037-19709c616ee1 5cb4b95b-05a4-f504-2037-19709c616ee1}"
testLoggerAdapter verbose: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:04.030Z [INFO]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: pipelining replication: peer="{Voter server-99fa282e-be8b-8862-f95a-b2ea3590ec5a 99fa282e-be8b-8862-f95a-b2ea3590ec5a}"
    testing.go:149: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:04.030Z [INFO]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: pipelining replication: peer="{Voter server-99fa282e-be8b-8862-f95a-b2ea3590ec5a 99fa282e-be8b-8862-f95a-b2ea3590ec5a}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:04.135Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:04.030255164 +0000 UTC m=+10.077144288 (1 nodes), 54.98204ms from start of poll, 55.413551ms from cluster start. Timeout at 2024-01-08 00:31:04.135434641 +0000 UTC m=+10.182323731, 105.179443ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:04.135Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:04.030255164 +0000 UTC m=+10.077144288 (1 nodes), 54.98204ms from start of poll, 55.413551ms from cluster start. Timeout at 2024-01-08 00:31:04.135434641 +0000 UTC m=+10.182323731, 105.179443ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:04.135Z [INFO]  starting stability test: raft-state=Follower
    testing.go:149: cluster: 2024-01-08T00:31:04.135Z [INFO]  starting stability test: raft-state=Follower
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:04.240Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:04.135590954 +0000 UTC m=+10.182480089 (2 nodes), 2.439µs from start of poll, 160.749352ms from cluster start. Timeout at 2024-01-08 00:31:04.24059557 +0000 UTC m=+10.287484669, 105.00458ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:04.240Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:04.135590954 +0000 UTC m=+10.182480089 (2 nodes), 2.439µs from start of poll, 160.749352ms from cluster start. Timeout at 2024-01-08 00:31:04.24059557 +0000 UTC m=+10.287484669, 105.00458ms after stability
testLoggerAdapter verbose: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:04.240Z [INFO]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: updating configuration: command=AddStaging server-id=server-99fa282e-be8b-8862-f95a-b2ea3590ec5a server-addr=99fa282e-be8b-8862-f95a-b2ea3590ec5a servers="[{Suffrage:Voter ID:server-99fa282e-be8b-8862-f95a-b2ea3590ec5a Address:99fa282e-be8b-8862-f95a-b2ea3590ec5a} {Suffrage:Voter ID:server-a89018d7-5e6b-df40-8a5f-33cd3d806e91 Address:a89018d7-5e6b-df40-8a5f-33cd3d806e91} {Suffrage:Voter ID:server-5cb4b95b-05a4-f504-2037-19709c616ee1 Address:5cb4b95b-05a4-f504-2037-19709c616ee1}]"
    testing.go:149: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:04.240Z [INFO]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: updating configuration: command=AddStaging server-id=server-99fa282e-be8b-8862-f95a-b2ea3590ec5a server-addr=99fa282e-be8b-8862-f95a-b2ea3590ec5a servers="[{Suffrage:Voter ID:server-99fa282e-be8b-8862-f95a-b2ea3590ec5a Address:99fa282e-be8b-8862-f95a-b2ea3590ec5a} {Suffrage:Voter ID:server-a89018d7-5e6b-df40-8a5f-33cd3d806e91 Address:a89018d7-5e6b-df40-8a5f-33cd3d806e91} {Suffrage:Voter ID:server-5cb4b95b-05a4-f504-2037-19709c616ee1 Address:5cb4b95b-05a4-f504-2037-19709c616ee1}]"
testLoggerAdapter verbose: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:04.241Z [INFO]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: aborting pipeline replication: peer="{Voter server-5cb4b95b-05a4-f504-2037-19709c616ee1 5cb4b95b-05a4-f504-2037-19709c616ee1}"
    testing.go:149: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:04.241Z [INFO]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: aborting pipeline replication: peer="{Voter server-5cb4b95b-05a4-f504-2037-19709c616ee1 5cb4b95b-05a4-f504-2037-19709c616ee1}"
testLoggerAdapter verbose: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:04.241Z [INFO]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: aborting pipeline replication: peer="{Voter server-99fa282e-be8b-8862-f95a-b2ea3590ec5a 99fa282e-be8b-8862-f95a-b2ea3590ec5a}"
    testing.go:149: server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: 2024-01-08T00:31:04.241Z [INFO]  server-a89018d7-5e6b-df40-8a5f-33cd3d806e91: aborting pipeline replication: peer="{Voter server-99fa282e-be8b-8862-f95a-b2ea3590ec5a 99fa282e-be8b-8862-f95a-b2ea3590ec5a}"
--- PASS: TestRaft_AddKnownPeer (0.27s)
=== RUN   TestRaft_RemoveUnknownPeer
testLoggerAdapter verbose: server-bef9a972-9d46-e167-f0ac-cbc19d7339d5: 2024-01-08T00:31:04.242Z [INFO]  server-bef9a972-9d46-e167-f0ac-cbc19d7339d5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-bef9a972-9d46-e167-f0ac-cbc19d7339d5 Address:bef9a972-9d46-e167-f0ac-cbc19d7339d5} {Suffrage:Voter ID:server-62fd3dc2-c7f3-be3a-03fe-86bea0637862 Address:62fd3dc2-c7f3-be3a-03fe-86bea0637862} {Suffrage:Voter ID:server-b3355308-2239-2be5-e7fc-63f633d00e06 Address:b3355308-2239-2be5-e7fc-63f633d00e06}]"
    testing.go:149: server-bef9a972-9d46-e167-f0ac-cbc19d7339d5: 2024-01-08T00:31:04.242Z [INFO]  server-bef9a972-9d46-e167-f0ac-cbc19d7339d5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-bef9a972-9d46-e167-f0ac-cbc19d7339d5 Address:bef9a972-9d46-e167-f0ac-cbc19d7339d5} {Suffrage:Voter ID:server-62fd3dc2-c7f3-be3a-03fe-86bea0637862 Address:62fd3dc2-c7f3-be3a-03fe-86bea0637862} {Suffrage:Voter ID:server-b3355308-2239-2be5-e7fc-63f633d00e06 Address:b3355308-2239-2be5-e7fc-63f633d00e06}]"
testLoggerAdapter verbose: server-62fd3dc2-c7f3-be3a-03fe-86bea0637862: 2024-01-08T00:31:04.242Z [INFO]  server-62fd3dc2-c7f3-be3a-03fe-86bea0637862: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-bef9a972-9d46-e167-f0ac-cbc19d7339d5 Address:bef9a972-9d46-e167-f0ac-cbc19d7339d5} {Suffrage:Voter ID:server-62fd3dc2-c7f3-be3a-03fe-86bea0637862 Address:62fd3dc2-c7f3-be3a-03fe-86bea0637862} {Suffrage:Voter ID:server-b3355308-2239-2be5-e7fc-63f633d00e06 Address:b3355308-2239-2be5-e7fc-63f633d00e06}]"
    testing.go:149: server-62fd3dc2-c7f3-be3a-03fe-86bea0637862: 2024-01-08T00:31:04.242Z [INFO]  server-62fd3dc2-c7f3-be3a-03fe-86bea0637862: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-bef9a972-9d46-e167-f0ac-cbc19d7339d5 Address:bef9a972-9d46-e167-f0ac-cbc19d7339d5} {Suffrage:Voter ID:server-62fd3dc2-c7f3-be3a-03fe-86bea0637862 Address:62fd3dc2-c7f3-be3a-03fe-86bea0637862} {Suffrage:Voter ID:server-b3355308-2239-2be5-e7fc-63f633d00e06 Address:b3355308-2239-2be5-e7fc-63f633d00e06}]"
testLoggerAdapter verbose: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.242Z [INFO]  server-b3355308-2239-2be5-e7fc-63f633d00e06: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-bef9a972-9d46-e167-f0ac-cbc19d7339d5 Address:bef9a972-9d46-e167-f0ac-cbc19d7339d5} {Suffrage:Voter ID:server-62fd3dc2-c7f3-be3a-03fe-86bea0637862 Address:62fd3dc2-c7f3-be3a-03fe-86bea0637862} {Suffrage:Voter ID:server-b3355308-2239-2be5-e7fc-63f633d00e06 Address:b3355308-2239-2be5-e7fc-63f633d00e06}]"
    testing.go:149: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.242Z [INFO]  server-b3355308-2239-2be5-e7fc-63f633d00e06: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-bef9a972-9d46-e167-f0ac-cbc19d7339d5 Address:bef9a972-9d46-e167-f0ac-cbc19d7339d5} {Suffrage:Voter ID:server-62fd3dc2-c7f3-be3a-03fe-86bea0637862 Address:62fd3dc2-c7f3-be3a-03fe-86bea0637862} {Suffrage:Voter ID:server-b3355308-2239-2be5-e7fc-63f633d00e06 Address:b3355308-2239-2be5-e7fc-63f633d00e06}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:04.242Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:04.242Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-bef9a972-9d46-e167-f0ac-cbc19d7339d5: 2024-01-08T00:31:04.242Z [INFO]  server-bef9a972-9d46-e167-f0ac-cbc19d7339d5: entering follower state: follower="Node at bef9a972-9d46-e167-f0ac-cbc19d7339d5 [Follower]" leader=
    testing.go:149: server-bef9a972-9d46-e167-f0ac-cbc19d7339d5: 2024-01-08T00:31:04.242Z [INFO]  server-bef9a972-9d46-e167-f0ac-cbc19d7339d5: entering follower state: follower="Node at bef9a972-9d46-e167-f0ac-cbc19d7339d5 [Follower]" leader=
testLoggerAdapter verbose: server-62fd3dc2-c7f3-be3a-03fe-86bea0637862: 2024-01-08T00:31:04.242Z [INFO]  server-62fd3dc2-c7f3-be3a-03fe-86bea0637862: entering follower state: follower="Node at 62fd3dc2-c7f3-be3a-03fe-86bea0637862 [Follower]" leader=
    testing.go:149: server-62fd3dc2-c7f3-be3a-03fe-86bea0637862: 2024-01-08T00:31:04.242Z [INFO]  server-62fd3dc2-c7f3-be3a-03fe-86bea0637862: entering follower state: follower="Node at 62fd3dc2-c7f3-be3a-03fe-86bea0637862 [Follower]" leader=
testLoggerAdapter verbose: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.242Z [INFO]  server-b3355308-2239-2be5-e7fc-63f633d00e06: entering follower state: follower="Node at b3355308-2239-2be5-e7fc-63f633d00e06 [Follower]" leader=
    testing.go:149: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.242Z [INFO]  server-b3355308-2239-2be5-e7fc-63f633d00e06: entering follower state: follower="Node at b3355308-2239-2be5-e7fc-63f633d00e06 [Follower]" leader=
testLoggerAdapter verbose: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.318Z [WARN]  server-b3355308-2239-2be5-e7fc-63f633d00e06: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.318Z [WARN]  server-b3355308-2239-2be5-e7fc-63f633d00e06: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.318Z [INFO]  server-b3355308-2239-2be5-e7fc-63f633d00e06: entering candidate state: node="Node at b3355308-2239-2be5-e7fc-63f633d00e06 [Candidate]" term=2
    testing.go:149: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.318Z [INFO]  server-b3355308-2239-2be5-e7fc-63f633d00e06: entering candidate state: node="Node at b3355308-2239-2be5-e7fc-63f633d00e06 [Candidate]" term=2
testLoggerAdapter verbose: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.318Z [INFO]  server-b3355308-2239-2be5-e7fc-63f633d00e06: election won: tally=2
    testing.go:149: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.318Z [INFO]  server-b3355308-2239-2be5-e7fc-63f633d00e06: election won: tally=2
testLoggerAdapter verbose: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.318Z [INFO]  server-b3355308-2239-2be5-e7fc-63f633d00e06: entering leader state: leader="Node at b3355308-2239-2be5-e7fc-63f633d00e06 [Leader]"
    testing.go:149: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.318Z [INFO]  server-b3355308-2239-2be5-e7fc-63f633d00e06: entering leader state: leader="Node at b3355308-2239-2be5-e7fc-63f633d00e06 [Leader]"
testLoggerAdapter verbose: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.318Z [INFO]  server-b3355308-2239-2be5-e7fc-63f633d00e06: added peer, starting replication: peer=server-bef9a972-9d46-e167-f0ac-cbc19d7339d5
    testing.go:149: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.318Z [INFO]  server-b3355308-2239-2be5-e7fc-63f633d00e06: added peer, starting replication: peer=server-bef9a972-9d46-e167-f0ac-cbc19d7339d5
testLoggerAdapter verbose: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.318Z [INFO]  server-b3355308-2239-2be5-e7fc-63f633d00e06: added peer, starting replication: peer=server-62fd3dc2-c7f3-be3a-03fe-86bea0637862
    testing.go:149: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.318Z [INFO]  server-b3355308-2239-2be5-e7fc-63f633d00e06: added peer, starting replication: peer=server-62fd3dc2-c7f3-be3a-03fe-86bea0637862
testLoggerAdapter verbose: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.318Z [INFO]  server-b3355308-2239-2be5-e7fc-63f633d00e06: pipelining replication: peer="{Voter server-62fd3dc2-c7f3-be3a-03fe-86bea0637862 62fd3dc2-c7f3-be3a-03fe-86bea0637862}"
    testing.go:149: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.318Z [INFO]  server-b3355308-2239-2be5-e7fc-63f633d00e06: pipelining replication: peer="{Voter server-62fd3dc2-c7f3-be3a-03fe-86bea0637862 62fd3dc2-c7f3-be3a-03fe-86bea0637862}"
testLoggerAdapter verbose: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.318Z [INFO]  server-b3355308-2239-2be5-e7fc-63f633d00e06: pipelining replication: peer="{Voter server-bef9a972-9d46-e167-f0ac-cbc19d7339d5 bef9a972-9d46-e167-f0ac-cbc19d7339d5}"
    testing.go:149: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.318Z [INFO]  server-b3355308-2239-2be5-e7fc-63f633d00e06: pipelining replication: peer="{Voter server-bef9a972-9d46-e167-f0ac-cbc19d7339d5 bef9a972-9d46-e167-f0ac-cbc19d7339d5}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:04.424Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:04.318848665 +0000 UTC m=+10.365737779 (1 nodes), 76.134566ms from start of poll, 76.587108ms from cluster start. Timeout at 2024-01-08 00:31:04.424290039 +0000 UTC m=+10.471179086, 105.441307ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:04.424Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:04.318848665 +0000 UTC m=+10.365737779 (1 nodes), 76.134566ms from start of poll, 76.587108ms from cluster start. Timeout at 2024-01-08 00:31:04.424290039 +0000 UTC m=+10.471179086, 105.441307ms after stability
testLoggerAdapter verbose: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.424Z [INFO]  server-b3355308-2239-2be5-e7fc-63f633d00e06: updating configuration: command=RemoveServer server-id=99ae7ab9-15e9-f264-01bf-7a61134dc8c0 server-addr= servers="[{Suffrage:Voter ID:server-bef9a972-9d46-e167-f0ac-cbc19d7339d5 Address:bef9a972-9d46-e167-f0ac-cbc19d7339d5} {Suffrage:Voter ID:server-62fd3dc2-c7f3-be3a-03fe-86bea0637862 Address:62fd3dc2-c7f3-be3a-03fe-86bea0637862} {Suffrage:Voter ID:server-b3355308-2239-2be5-e7fc-63f633d00e06 Address:b3355308-2239-2be5-e7fc-63f633d00e06}]"
    testing.go:149: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.424Z [INFO]  server-b3355308-2239-2be5-e7fc-63f633d00e06: updating configuration: command=RemoveServer server-id=99ae7ab9-15e9-f264-01bf-7a61134dc8c0 server-addr= servers="[{Suffrage:Voter ID:server-bef9a972-9d46-e167-f0ac-cbc19d7339d5 Address:bef9a972-9d46-e167-f0ac-cbc19d7339d5} {Suffrage:Voter ID:server-62fd3dc2-c7f3-be3a-03fe-86bea0637862 Address:62fd3dc2-c7f3-be3a-03fe-86bea0637862} {Suffrage:Voter ID:server-b3355308-2239-2be5-e7fc-63f633d00e06 Address:b3355308-2239-2be5-e7fc-63f633d00e06}]"
testLoggerAdapter verbose: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.424Z [INFO]  server-b3355308-2239-2be5-e7fc-63f633d00e06: aborting pipeline replication: peer="{Voter server-62fd3dc2-c7f3-be3a-03fe-86bea0637862 62fd3dc2-c7f3-be3a-03fe-86bea0637862}"
    testing.go:149: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.424Z [INFO]  server-b3355308-2239-2be5-e7fc-63f633d00e06: aborting pipeline replication: peer="{Voter server-62fd3dc2-c7f3-be3a-03fe-86bea0637862 62fd3dc2-c7f3-be3a-03fe-86bea0637862}"
testLoggerAdapter verbose: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.424Z [INFO]  server-b3355308-2239-2be5-e7fc-63f633d00e06: aborting pipeline replication: peer="{Voter server-bef9a972-9d46-e167-f0ac-cbc19d7339d5 bef9a972-9d46-e167-f0ac-cbc19d7339d5}"
    testing.go:149: server-b3355308-2239-2be5-e7fc-63f633d00e06: 2024-01-08T00:31:04.424Z [INFO]  server-b3355308-2239-2be5-e7fc-63f633d00e06: aborting pipeline replication: peer="{Voter server-bef9a972-9d46-e167-f0ac-cbc19d7339d5 bef9a972-9d46-e167-f0ac-cbc19d7339d5}"
--- PASS: TestRaft_RemoveUnknownPeer (0.18s)
=== RUN   TestRaft_SnapshotRestore
testLoggerAdapter verbose: server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: 2024-01-08T00:31:04.425Z [INFO]  server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e3d918a9-adc4-4e25-5a00-a928c10fd24c Address:e3d918a9-adc4-4e25-5a00-a928c10fd24c}]"
    testing.go:149: server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: 2024-01-08T00:31:04.425Z [INFO]  server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e3d918a9-adc4-4e25-5a00-a928c10fd24c Address:e3d918a9-adc4-4e25-5a00-a928c10fd24c}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:04.425Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:04.425Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: 2024-01-08T00:31:04.425Z [INFO]  server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: entering follower state: follower="Node at e3d918a9-adc4-4e25-5a00-a928c10fd24c [Follower]" leader=
    testing.go:149: server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: 2024-01-08T00:31:04.425Z [INFO]  server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: entering follower state: follower="Node at e3d918a9-adc4-4e25-5a00-a928c10fd24c [Follower]" leader=
testLoggerAdapter verbose: server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: 2024-01-08T00:31:04.498Z [WARN]  server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: 2024-01-08T00:31:04.498Z [WARN]  server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: 2024-01-08T00:31:04.498Z [INFO]  server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: entering candidate state: node="Node at e3d918a9-adc4-4e25-5a00-a928c10fd24c [Candidate]" term=2
    testing.go:149: server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: 2024-01-08T00:31:04.498Z [INFO]  server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: entering candidate state: node="Node at e3d918a9-adc4-4e25-5a00-a928c10fd24c [Candidate]" term=2
testLoggerAdapter verbose: server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: 2024-01-08T00:31:04.498Z [INFO]  server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: election won: tally=1
    testing.go:149: server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: 2024-01-08T00:31:04.498Z [INFO]  server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: election won: tally=1
testLoggerAdapter verbose: server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: 2024-01-08T00:31:04.498Z [INFO]  server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: entering leader state: leader="Node at e3d918a9-adc4-4e25-5a00-a928c10fd24c [Leader]"
    testing.go:149: server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: 2024-01-08T00:31:04.498Z [INFO]  server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: entering leader state: leader="Node at e3d918a9-adc4-4e25-5a00-a928c10fd24c [Leader]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:04.604Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:04.498633221 +0000 UTC m=+10.545522260 (1 nodes), 73.069697ms from start of poll, 73.2095ms from cluster start. Timeout at 2024-01-08 00:31:04.604377403 +0000 UTC m=+10.651266477, 105.744217ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:04.604Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:04.498633221 +0000 UTC m=+10.545522260 (1 nodes), 73.069697ms from start of poll, 73.2095ms from cluster start. Timeout at 2024-01-08 00:31:04.604377403 +0000 UTC m=+10.651266477, 105.744217ms after stability
testLoggerAdapter verbose: server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: 2024-01-08T00:31:04.605Z [INFO]  server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: starting snapshot up to: index=102
    testing.go:149: server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: 2024-01-08T00:31:04.605Z [INFO]  server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: starting snapshot up to: index=102
    testing.go:153: 2024-01-08T00:31:04.605Z [INFO]  creating new snapshot: path=/tmp/raft2694771695/snapshots/2-102-1704673864605.tmp
testLoggerAdapter verbose: server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: 2024-01-08T00:31:04.615Z [INFO]  server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: compacting logs: from=1 to=92
    testing.go:149: server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: 2024-01-08T00:31:04.615Z [INFO]  server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: compacting logs: from=1 to=92
testLoggerAdapter verbose: server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: 2024-01-08T00:31:04.615Z [INFO]  server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: snapshot complete up to: index=102
    testing.go:149: server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: 2024-01-08T00:31:04.615Z [INFO]  server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: snapshot complete up to: index=102
testLoggerAdapter verbose: server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: 2024-01-08T00:31:04.615Z [INFO]  server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: restored from snapshot: id=2-102-1704673864605
    testing.go:149: server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: 2024-01-08T00:31:04.615Z [INFO]  server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: restored from snapshot: id=2-102-1704673864605
testLoggerAdapter verbose: server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: 2024-01-08T00:31:04.615Z [INFO]  server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e3d918a9-adc4-4e25-5a00-a928c10fd24c Address:e3d918a9-adc4-4e25-5a00-a928c10fd24c}]"
    testing.go:149: server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: 2024-01-08T00:31:04.615Z [INFO]  server-e3d918a9-adc4-4e25-5a00-a928c10fd24c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e3d918a9-adc4-4e25-5a00-a928c10fd24c Address:e3d918a9-adc4-4e25-5a00-a928c10fd24c}]"
--- PASS: TestRaft_SnapshotRestore (0.19s)
=== RUN   TestRaft_NoRestoreOnStart
testLoggerAdapter verbose: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.616Z [INFO]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-46420970-2970-2c8c-2ac8-09bd60779bd4 Address:46420970-2970-2c8c-2ac8-09bd60779bd4}]"
    testing.go:149: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.616Z [INFO]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-46420970-2970-2c8c-2ac8-09bd60779bd4 Address:46420970-2970-2c8c-2ac8-09bd60779bd4}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:04.616Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:04.616Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.616Z [INFO]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: entering follower state: follower="Node at 46420970-2970-2c8c-2ac8-09bd60779bd4 [Follower]" leader=
    testing.go:149: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.616Z [INFO]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: entering follower state: follower="Node at 46420970-2970-2c8c-2ac8-09bd60779bd4 [Follower]" leader=
testLoggerAdapter verbose: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.692Z [WARN]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.692Z [WARN]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.692Z [INFO]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: entering candidate state: node="Node at 46420970-2970-2c8c-2ac8-09bd60779bd4 [Candidate]" term=2
    testing.go:149: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.692Z [INFO]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: entering candidate state: node="Node at 46420970-2970-2c8c-2ac8-09bd60779bd4 [Candidate]" term=2
testLoggerAdapter verbose: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.692Z [INFO]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: election won: tally=1
    testing.go:149: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.692Z [INFO]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: election won: tally=1
testLoggerAdapter verbose: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.692Z [INFO]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: entering leader state: leader="Node at 46420970-2970-2c8c-2ac8-09bd60779bd4 [Leader]"
    testing.go:149: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.692Z [INFO]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: entering leader state: leader="Node at 46420970-2970-2c8c-2ac8-09bd60779bd4 [Leader]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:04.798Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:04.692822191 +0000 UTC m=+10.739711250 (1 nodes), 76.679261ms from start of poll, 76.80049ms from cluster start. Timeout at 2024-01-08 00:31:04.798355828 +0000 UTC m=+10.845244931, 105.533681ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:04.798Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:04.692822191 +0000 UTC m=+10.739711250 (1 nodes), 76.679261ms from start of poll, 76.80049ms from cluster start. Timeout at 2024-01-08 00:31:04.798355828 +0000 UTC m=+10.845244931, 105.533681ms after stability
testLoggerAdapter verbose: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.799Z [INFO]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: starting snapshot up to: index=102
    testing.go:149: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.799Z [INFO]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: starting snapshot up to: index=102
    testing.go:153: 2024-01-08T00:31:04.799Z [INFO]  creating new snapshot: path=/tmp/raft763335452/snapshots/2-102-1704673864799.tmp
testLoggerAdapter verbose: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.808Z [INFO]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: compacting logs: from=1 to=92
    testing.go:149: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.808Z [INFO]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: compacting logs: from=1 to=92
testLoggerAdapter verbose: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.808Z [INFO]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: snapshot complete up to: index=102
    testing.go:149: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.808Z [INFO]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: snapshot complete up to: index=102
testLoggerAdapter verbose: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.808Z [INFO]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-46420970-2970-2c8c-2ac8-09bd60779bd4 Address:46420970-2970-2c8c-2ac8-09bd60779bd4}]"
    testing.go:149: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.808Z [INFO]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-46420970-2970-2c8c-2ac8-09bd60779bd4 Address:46420970-2970-2c8c-2ac8-09bd60779bd4}]"
--- PASS: TestRaft_NoRestoreOnStart (0.19s)
=== RUN   TestRaft_SnapshotRestore_PeerChange
    raft_test.go:1026: Disabled unreliable test
--- SKIP: TestRaft_SnapshotRestore_PeerChange (0.00s)
=== RUN   TestRaft_AutoSnapshot
    raft_test.go:1132: Disabled unreliable test
--- SKIP: TestRaft_AutoSnapshot (0.00s)
=== RUN   TestRaft_UserSnapshot
testLoggerAdapter verbose: server-399e17c6-176c-5779-808a-568b0affb669: 2024-01-08T00:31:04.808Z [INFO]  server-399e17c6-176c-5779-808a-568b0affb669: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-399e17c6-176c-5779-808a-568b0affb669 Address:399e17c6-176c-5779-808a-568b0affb669}]"
    testing.go:149: server-399e17c6-176c-5779-808a-568b0affb669: 2024-01-08T00:31:04.808Z [INFO]  server-399e17c6-176c-5779-808a-568b0affb669: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-399e17c6-176c-5779-808a-568b0affb669 Address:399e17c6-176c-5779-808a-568b0affb669}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:04.808Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:04.808Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.808Z [INFO]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: entering follower state: follower="Node at 46420970-2970-2c8c-2ac8-09bd60779bd4 [Follower]" leader=
testLoggerAdapter verbose: server-399e17c6-176c-5779-808a-568b0affb669: 2024-01-08T00:31:04.808Z [INFO]  server-399e17c6-176c-5779-808a-568b0affb669: entering follower state: follower="Node at 399e17c6-176c-5779-808a-568b0affb669 [Follower]" leader=
    testing.go:149: server-399e17c6-176c-5779-808a-568b0affb669: 2024-01-08T00:31:04.808Z [INFO]  server-399e17c6-176c-5779-808a-568b0affb669: entering follower state: follower="Node at 399e17c6-176c-5779-808a-568b0affb669 [Follower]" leader=
testLoggerAdapter verbose: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.860Z [WARN]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.860Z [INFO]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: entering candidate state: node="Node at 46420970-2970-2c8c-2ac8-09bd60779bd4 [Candidate]" term=3
testLoggerAdapter verbose: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.860Z [INFO]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: election won: tally=1
testLoggerAdapter verbose: server-46420970-2970-2c8c-2ac8-09bd60779bd4: 2024-01-08T00:31:04.860Z [INFO]  server-46420970-2970-2c8c-2ac8-09bd60779bd4: entering leader state: leader="Node at 46420970-2970-2c8c-2ac8-09bd60779bd4 [Leader]"
testLoggerAdapter verbose: server-399e17c6-176c-5779-808a-568b0affb669: 2024-01-08T00:31:04.902Z [WARN]  server-399e17c6-176c-5779-808a-568b0affb669: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-399e17c6-176c-5779-808a-568b0affb669: 2024-01-08T00:31:04.902Z [WARN]  server-399e17c6-176c-5779-808a-568b0affb669: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-399e17c6-176c-5779-808a-568b0affb669: 2024-01-08T00:31:04.902Z [INFO]  server-399e17c6-176c-5779-808a-568b0affb669: entering candidate state: node="Node at 399e17c6-176c-5779-808a-568b0affb669 [Candidate]" term=2
    testing.go:149: server-399e17c6-176c-5779-808a-568b0affb669: 2024-01-08T00:31:04.902Z [INFO]  server-399e17c6-176c-5779-808a-568b0affb669: entering candidate state: node="Node at 399e17c6-176c-5779-808a-568b0affb669 [Candidate]" term=2
testLoggerAdapter verbose: server-399e17c6-176c-5779-808a-568b0affb669: 2024-01-08T00:31:04.902Z [INFO]  server-399e17c6-176c-5779-808a-568b0affb669: election won: tally=1
    testing.go:149: server-399e17c6-176c-5779-808a-568b0affb669: 2024-01-08T00:31:04.902Z [INFO]  server-399e17c6-176c-5779-808a-568b0affb669: election won: tally=1
testLoggerAdapter verbose: server-399e17c6-176c-5779-808a-568b0affb669: 2024-01-08T00:31:04.902Z [INFO]  server-399e17c6-176c-5779-808a-568b0affb669: entering leader state: leader="Node at 399e17c6-176c-5779-808a-568b0affb669 [Leader]"
    testing.go:149: server-399e17c6-176c-5779-808a-568b0affb669: 2024-01-08T00:31:04.902Z [INFO]  server-399e17c6-176c-5779-808a-568b0affb669: entering leader state: leader="Node at 399e17c6-176c-5779-808a-568b0affb669 [Leader]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:05.007Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:04.902449631 +0000 UTC m=+10.949338685 (1 nodes), 93.726439ms from start of poll, 93.8615ms from cluster start. Timeout at 2024-01-08 00:31:05.007491179 +0000 UTC m=+11.054380275, 105.04159ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:05.007Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:04.902449631 +0000 UTC m=+10.949338685 (1 nodes), 93.726439ms from start of poll, 93.8615ms from cluster start. Timeout at 2024-01-08 00:31:05.007491179 +0000 UTC m=+11.054380275, 105.04159ms after stability
testLoggerAdapter verbose: server-399e17c6-176c-5779-808a-568b0affb669: 2024-01-08T00:31:05.007Z [ERROR] server-399e17c6-176c-5779-808a-568b0affb669: failed to take snapshot: error="nothing new to snapshot"
    testing.go:149: server-399e17c6-176c-5779-808a-568b0affb669: 2024-01-08T00:31:05.007Z [ERROR] server-399e17c6-176c-5779-808a-568b0affb669: failed to take snapshot: error="nothing new to snapshot"
testLoggerAdapter verbose: server-399e17c6-176c-5779-808a-568b0affb669: 2024-01-08T00:31:05.007Z [INFO]  server-399e17c6-176c-5779-808a-568b0affb669: starting snapshot up to: index=12
    testing.go:149: server-399e17c6-176c-5779-808a-568b0affb669: 2024-01-08T00:31:05.007Z [INFO]  server-399e17c6-176c-5779-808a-568b0affb669: starting snapshot up to: index=12
    testing.go:153: 2024-01-08T00:31:05.007Z [INFO]  creating new snapshot: path=/tmp/raft3724544202/snapshots/2-12-1704673865007.tmp
testLoggerAdapter verbose: server-399e17c6-176c-5779-808a-568b0affb669: 2024-01-08T00:31:05.017Z [INFO]  server-399e17c6-176c-5779-808a-568b0affb669: compacting logs: from=1 to=2
    testing.go:149: server-399e17c6-176c-5779-808a-568b0affb669: 2024-01-08T00:31:05.017Z [INFO]  server-399e17c6-176c-5779-808a-568b0affb669: compacting logs: from=1 to=2
testLoggerAdapter verbose: server-399e17c6-176c-5779-808a-568b0affb669: 2024-01-08T00:31:05.017Z [INFO]  server-399e17c6-176c-5779-808a-568b0affb669: snapshot complete up to: index=12
    testing.go:149: server-399e17c6-176c-5779-808a-568b0affb669: 2024-01-08T00:31:05.017Z [INFO]  server-399e17c6-176c-5779-808a-568b0affb669: snapshot complete up to: index=12
--- PASS: TestRaft_UserSnapshot (0.21s)
=== RUN   TestRaft_UserRestore
testLoggerAdapter verbose: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.018Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7050e622-91b0-d318-c4fb-e32d1e4573e9 Address:7050e622-91b0-d318-c4fb-e32d1e4573e9} {Suffrage:Voter ID:server-00eb74b9-fc78-e96d-b62f-9811c7956233 Address:00eb74b9-fc78-e96d-b62f-9811c7956233} {Suffrage:Voter ID:server-73c1b83f-4a4c-e160-fb5a-091047822f4b Address:73c1b83f-4a4c-e160-fb5a-091047822f4b}]"
    testing.go:149: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.018Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7050e622-91b0-d318-c4fb-e32d1e4573e9 Address:7050e622-91b0-d318-c4fb-e32d1e4573e9} {Suffrage:Voter ID:server-00eb74b9-fc78-e96d-b62f-9811c7956233 Address:00eb74b9-fc78-e96d-b62f-9811c7956233} {Suffrage:Voter ID:server-73c1b83f-4a4c-e160-fb5a-091047822f4b Address:73c1b83f-4a4c-e160-fb5a-091047822f4b}]"
testLoggerAdapter verbose: server-00eb74b9-fc78-e96d-b62f-9811c7956233: 2024-01-08T00:31:05.018Z [INFO]  server-00eb74b9-fc78-e96d-b62f-9811c7956233: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7050e622-91b0-d318-c4fb-e32d1e4573e9 Address:7050e622-91b0-d318-c4fb-e32d1e4573e9} {Suffrage:Voter ID:server-00eb74b9-fc78-e96d-b62f-9811c7956233 Address:00eb74b9-fc78-e96d-b62f-9811c7956233} {Suffrage:Voter ID:server-73c1b83f-4a4c-e160-fb5a-091047822f4b Address:73c1b83f-4a4c-e160-fb5a-091047822f4b}]"
    testing.go:149: server-00eb74b9-fc78-e96d-b62f-9811c7956233: 2024-01-08T00:31:05.018Z [INFO]  server-00eb74b9-fc78-e96d-b62f-9811c7956233: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7050e622-91b0-d318-c4fb-e32d1e4573e9 Address:7050e622-91b0-d318-c4fb-e32d1e4573e9} {Suffrage:Voter ID:server-00eb74b9-fc78-e96d-b62f-9811c7956233 Address:00eb74b9-fc78-e96d-b62f-9811c7956233} {Suffrage:Voter ID:server-73c1b83f-4a4c-e160-fb5a-091047822f4b Address:73c1b83f-4a4c-e160-fb5a-091047822f4b}]"
testLoggerAdapter verbose: server-73c1b83f-4a4c-e160-fb5a-091047822f4b: 2024-01-08T00:31:05.018Z [INFO]  server-73c1b83f-4a4c-e160-fb5a-091047822f4b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7050e622-91b0-d318-c4fb-e32d1e4573e9 Address:7050e622-91b0-d318-c4fb-e32d1e4573e9} {Suffrage:Voter ID:server-00eb74b9-fc78-e96d-b62f-9811c7956233 Address:00eb74b9-fc78-e96d-b62f-9811c7956233} {Suffrage:Voter ID:server-73c1b83f-4a4c-e160-fb5a-091047822f4b Address:73c1b83f-4a4c-e160-fb5a-091047822f4b}]"
    testing.go:149: server-73c1b83f-4a4c-e160-fb5a-091047822f4b: 2024-01-08T00:31:05.018Z [INFO]  server-73c1b83f-4a4c-e160-fb5a-091047822f4b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7050e622-91b0-d318-c4fb-e32d1e4573e9 Address:7050e622-91b0-d318-c4fb-e32d1e4573e9} {Suffrage:Voter ID:server-00eb74b9-fc78-e96d-b62f-9811c7956233 Address:00eb74b9-fc78-e96d-b62f-9811c7956233} {Suffrage:Voter ID:server-73c1b83f-4a4c-e160-fb5a-091047822f4b Address:73c1b83f-4a4c-e160-fb5a-091047822f4b}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:05.018Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:05.018Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.018Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: entering follower state: follower="Node at 7050e622-91b0-d318-c4fb-e32d1e4573e9 [Follower]" leader=
    testing.go:149: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.018Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: entering follower state: follower="Node at 7050e622-91b0-d318-c4fb-e32d1e4573e9 [Follower]" leader=
testLoggerAdapter verbose: server-00eb74b9-fc78-e96d-b62f-9811c7956233: 2024-01-08T00:31:05.018Z [INFO]  server-00eb74b9-fc78-e96d-b62f-9811c7956233: entering follower state: follower="Node at 00eb74b9-fc78-e96d-b62f-9811c7956233 [Follower]" leader=
    testing.go:149: server-00eb74b9-fc78-e96d-b62f-9811c7956233: 2024-01-08T00:31:05.018Z [INFO]  server-00eb74b9-fc78-e96d-b62f-9811c7956233: entering follower state: follower="Node at 00eb74b9-fc78-e96d-b62f-9811c7956233 [Follower]" leader=
testLoggerAdapter verbose: server-73c1b83f-4a4c-e160-fb5a-091047822f4b: 2024-01-08T00:31:05.019Z [INFO]  server-73c1b83f-4a4c-e160-fb5a-091047822f4b: entering follower state: follower="Node at 73c1b83f-4a4c-e160-fb5a-091047822f4b [Follower]" leader=
    testing.go:149: server-73c1b83f-4a4c-e160-fb5a-091047822f4b: 2024-01-08T00:31:05.019Z [INFO]  server-73c1b83f-4a4c-e160-fb5a-091047822f4b: entering follower state: follower="Node at 73c1b83f-4a4c-e160-fb5a-091047822f4b [Follower]" leader=
testLoggerAdapter verbose: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.083Z [WARN]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.083Z [WARN]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.083Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: entering candidate state: node="Node at 7050e622-91b0-d318-c4fb-e32d1e4573e9 [Candidate]" term=2
    testing.go:149: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.083Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: entering candidate state: node="Node at 7050e622-91b0-d318-c4fb-e32d1e4573e9 [Candidate]" term=2
testLoggerAdapter verbose: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.083Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: election won: tally=2
    testing.go:149: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.083Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: election won: tally=2
testLoggerAdapter verbose: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.083Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: entering leader state: leader="Node at 7050e622-91b0-d318-c4fb-e32d1e4573e9 [Leader]"
    testing.go:149: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.083Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: entering leader state: leader="Node at 7050e622-91b0-d318-c4fb-e32d1e4573e9 [Leader]"
testLoggerAdapter verbose: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.083Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: added peer, starting replication: peer=server-00eb74b9-fc78-e96d-b62f-9811c7956233
    testing.go:149: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.083Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: added peer, starting replication: peer=server-00eb74b9-fc78-e96d-b62f-9811c7956233
testLoggerAdapter verbose: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.083Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: added peer, starting replication: peer=server-73c1b83f-4a4c-e160-fb5a-091047822f4b
    testing.go:149: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.083Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: added peer, starting replication: peer=server-73c1b83f-4a4c-e160-fb5a-091047822f4b
testLoggerAdapter verbose: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.083Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: pipelining replication: peer="{Voter server-73c1b83f-4a4c-e160-fb5a-091047822f4b 73c1b83f-4a4c-e160-fb5a-091047822f4b}"
    testing.go:149: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.083Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: pipelining replication: peer="{Voter server-73c1b83f-4a4c-e160-fb5a-091047822f4b 73c1b83f-4a4c-e160-fb5a-091047822f4b}"
testLoggerAdapter verbose: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.083Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: pipelining replication: peer="{Voter server-00eb74b9-fc78-e96d-b62f-9811c7956233 00eb74b9-fc78-e96d-b62f-9811c7956233}"
    testing.go:149: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.083Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: pipelining replication: peer="{Voter server-00eb74b9-fc78-e96d-b62f-9811c7956233 00eb74b9-fc78-e96d-b62f-9811c7956233}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:05.189Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:05.083807014 +0000 UTC m=+11.130696055 (1 nodes), 64.881517ms from start of poll, 65.237087ms from cluster start. Timeout at 2024-01-08 00:31:05.189486328 +0000 UTC m=+11.236375413, 105.679358ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:05.189Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:05.083807014 +0000 UTC m=+11.130696055 (1 nodes), 64.881517ms from start of poll, 65.237087ms from cluster start. Timeout at 2024-01-08 00:31:05.189486328 +0000 UTC m=+11.236375413, 105.679358ms after stability
testLoggerAdapter verbose: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.190Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: starting snapshot up to: index=12
    testing.go:149: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.190Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: starting snapshot up to: index=12
    testing.go:153: 2024-01-08T00:31:05.190Z [INFO]  creating new snapshot: path=/tmp/raft3821454391/snapshots/2-12-1704673865190.tmp
testLoggerAdapter verbose: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.198Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: snapshot complete up to: index=12
    testing.go:149: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.198Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: snapshot complete up to: index=12
    testing.go:153: 2024-01-08T00:31:05.198Z [INFO]  creating new snapshot: path=/tmp/raft3821454391/snapshots/2-23-1704673865198.tmp
testLoggerAdapter verbose: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.206Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: copied to local snapshot: bytes=71
    testing.go:149: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.206Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: copied to local snapshot: bytes=71
testLoggerAdapter verbose: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.207Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: restored user snapshot: index=1
    testing.go:149: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.207Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: restored user snapshot: index=1
testLoggerAdapter verbose: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.207Z [ERROR] server-7050e622-91b0-d318-c4fb-e32d1e4573e9: failed to get log: index=23 error="log not found"
    testing.go:149: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.207Z [ERROR] server-7050e622-91b0-d318-c4fb-e32d1e4573e9: failed to get log: index=23 error="log not found"
testLoggerAdapter verbose: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.207Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: aborting pipeline replication: peer="{Voter server-73c1b83f-4a4c-e160-fb5a-091047822f4b 73c1b83f-4a4c-e160-fb5a-091047822f4b}"
    testing.go:149: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.207Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: aborting pipeline replication: peer="{Voter server-73c1b83f-4a4c-e160-fb5a-091047822f4b 73c1b83f-4a4c-e160-fb5a-091047822f4b}"
testLoggerAdapter verbose: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.207Z [ERROR] server-7050e622-91b0-d318-c4fb-e32d1e4573e9: failed to get log: index=23 error="log not found"
    testing.go:149: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.207Z [ERROR] server-7050e622-91b0-d318-c4fb-e32d1e4573e9: failed to get log: index=23 error="log not found"
testLoggerAdapter verbose: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.207Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: aborting pipeline replication: peer="{Voter server-00eb74b9-fc78-e96d-b62f-9811c7956233 00eb74b9-fc78-e96d-b62f-9811c7956233}"
    testing.go:149: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.207Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: aborting pipeline replication: peer="{Voter server-00eb74b9-fc78-e96d-b62f-9811c7956233 00eb74b9-fc78-e96d-b62f-9811c7956233}"
testLoggerAdapter verbose: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.213Z [ERROR] server-7050e622-91b0-d318-c4fb-e32d1e4573e9: failed to get log: index=23 error="log not found"
    testing.go:149: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.213Z [ERROR] server-7050e622-91b0-d318-c4fb-e32d1e4573e9: failed to get log: index=23 error="log not found"
    testing.go:153: 2024-01-08T00:31:05.214Z [INFO]  creating new snapshot: path=/tmp/raft2450759202/snapshots/2-23-1704673865214.tmp
testLoggerAdapter verbose: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.215Z [ERROR] server-7050e622-91b0-d318-c4fb-e32d1e4573e9: failed to get log: index=23 error="log not found"
    testing.go:149: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.215Z [ERROR] server-7050e622-91b0-d318-c4fb-e32d1e4573e9: failed to get log: index=23 error="log not found"
    testing.go:153: 2024-01-08T00:31:05.216Z [INFO]  creating new snapshot: path=/tmp/raft2422308845/snapshots/2-23-1704673865216.tmp
testLoggerAdapter verbose: server-00eb74b9-fc78-e96d-b62f-9811c7956233: 2024-01-08T00:31:05.226Z [INFO]  server-00eb74b9-fc78-e96d-b62f-9811c7956233: copied to local snapshot: bytes=71
    testing.go:149: server-00eb74b9-fc78-e96d-b62f-9811c7956233: 2024-01-08T00:31:05.226Z [INFO]  server-00eb74b9-fc78-e96d-b62f-9811c7956233: copied to local snapshot: bytes=71
testLoggerAdapter verbose: server-00eb74b9-fc78-e96d-b62f-9811c7956233: 2024-01-08T00:31:05.226Z [INFO]  server-00eb74b9-fc78-e96d-b62f-9811c7956233: Installed remote snapshot
    testing.go:149: server-00eb74b9-fc78-e96d-b62f-9811c7956233: 2024-01-08T00:31:05.226Z [INFO]  server-00eb74b9-fc78-e96d-b62f-9811c7956233: Installed remote snapshot
testLoggerAdapter verbose: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.226Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: pipelining replication: peer="{Voter server-00eb74b9-fc78-e96d-b62f-9811c7956233 00eb74b9-fc78-e96d-b62f-9811c7956233}"
    testing.go:149: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.226Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: pipelining replication: peer="{Voter server-00eb74b9-fc78-e96d-b62f-9811c7956233 00eb74b9-fc78-e96d-b62f-9811c7956233}"
testLoggerAdapter verbose: server-73c1b83f-4a4c-e160-fb5a-091047822f4b: 2024-01-08T00:31:05.226Z [INFO]  server-73c1b83f-4a4c-e160-fb5a-091047822f4b: copied to local snapshot: bytes=71
    testing.go:149: server-73c1b83f-4a4c-e160-fb5a-091047822f4b: 2024-01-08T00:31:05.226Z [INFO]  server-73c1b83f-4a4c-e160-fb5a-091047822f4b: copied to local snapshot: bytes=71
testLoggerAdapter verbose: server-73c1b83f-4a4c-e160-fb5a-091047822f4b: 2024-01-08T00:31:05.226Z [INFO]  server-73c1b83f-4a4c-e160-fb5a-091047822f4b: Installed remote snapshot
    testing.go:149: server-73c1b83f-4a4c-e160-fb5a-091047822f4b: 2024-01-08T00:31:05.226Z [INFO]  server-73c1b83f-4a4c-e160-fb5a-091047822f4b: Installed remote snapshot
testLoggerAdapter verbose: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.227Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: pipelining replication: peer="{Voter server-73c1b83f-4a4c-e160-fb5a-091047822f4b 73c1b83f-4a4c-e160-fb5a-091047822f4b}"
    testing.go:149: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.227Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: pipelining replication: peer="{Voter server-73c1b83f-4a4c-e160-fb5a-091047822f4b 73c1b83f-4a4c-e160-fb5a-091047822f4b}"
testLoggerAdapter verbose: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.244Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: aborting pipeline replication: peer="{Voter server-73c1b83f-4a4c-e160-fb5a-091047822f4b 73c1b83f-4a4c-e160-fb5a-091047822f4b}"
    testing.go:149: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.244Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: aborting pipeline replication: peer="{Voter server-73c1b83f-4a4c-e160-fb5a-091047822f4b 73c1b83f-4a4c-e160-fb5a-091047822f4b}"
testLoggerAdapter verbose: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.244Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: aborting pipeline replication: peer="{Voter server-00eb74b9-fc78-e96d-b62f-9811c7956233 00eb74b9-fc78-e96d-b62f-9811c7956233}"
    testing.go:149: server-7050e622-91b0-d318-c4fb-e32d1e4573e9: 2024-01-08T00:31:05.244Z [INFO]  server-7050e622-91b0-d318-c4fb-e32d1e4573e9: aborting pipeline replication: peer="{Voter server-00eb74b9-fc78-e96d-b62f-9811c7956233 00eb74b9-fc78-e96d-b62f-9811c7956233}"
testLoggerAdapter verbose: server-2af0776a-5a1b-5a54-d2d5-5626395e9402: 2024-01-08T00:31:05.246Z [INFO]  server-2af0776a-5a1b-5a54-d2d5-5626395e9402: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-2af0776a-5a1b-5a54-d2d5-5626395e9402 Address:2af0776a-5a1b-5a54-d2d5-5626395e9402} {Suffrage:Voter ID:server-37991d03-32aa-cfcf-1adb-12eaa3b69498 Address:37991d03-32aa-cfcf-1adb-12eaa3b69498} {Suffrage:Voter ID:server-792e1d30-30af-6ad5-b688-5c14ba5d38c9 Address:792e1d30-30af-6ad5-b688-5c14ba5d38c9}]"
    testing.go:149: server-2af0776a-5a1b-5a54-d2d5-5626395e9402: 2024-01-08T00:31:05.246Z [INFO]  server-2af0776a-5a1b-5a54-d2d5-5626395e9402: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-2af0776a-5a1b-5a54-d2d5-5626395e9402 Address:2af0776a-5a1b-5a54-d2d5-5626395e9402} {Suffrage:Voter ID:server-37991d03-32aa-cfcf-1adb-12eaa3b69498 Address:37991d03-32aa-cfcf-1adb-12eaa3b69498} {Suffrage:Voter ID:server-792e1d30-30af-6ad5-b688-5c14ba5d38c9 Address:792e1d30-30af-6ad5-b688-5c14ba5d38c9}]"
testLoggerAdapter verbose: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.246Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-2af0776a-5a1b-5a54-d2d5-5626395e9402 Address:2af0776a-5a1b-5a54-d2d5-5626395e9402} {Suffrage:Voter ID:server-37991d03-32aa-cfcf-1adb-12eaa3b69498 Address:37991d03-32aa-cfcf-1adb-12eaa3b69498} {Suffrage:Voter ID:server-792e1d30-30af-6ad5-b688-5c14ba5d38c9 Address:792e1d30-30af-6ad5-b688-5c14ba5d38c9}]"
    testing.go:149: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.246Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-2af0776a-5a1b-5a54-d2d5-5626395e9402 Address:2af0776a-5a1b-5a54-d2d5-5626395e9402} {Suffrage:Voter ID:server-37991d03-32aa-cfcf-1adb-12eaa3b69498 Address:37991d03-32aa-cfcf-1adb-12eaa3b69498} {Suffrage:Voter ID:server-792e1d30-30af-6ad5-b688-5c14ba5d38c9 Address:792e1d30-30af-6ad5-b688-5c14ba5d38c9}]"
testLoggerAdapter verbose: server-792e1d30-30af-6ad5-b688-5c14ba5d38c9: 2024-01-08T00:31:05.246Z [INFO]  server-792e1d30-30af-6ad5-b688-5c14ba5d38c9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-2af0776a-5a1b-5a54-d2d5-5626395e9402 Address:2af0776a-5a1b-5a54-d2d5-5626395e9402} {Suffrage:Voter ID:server-37991d03-32aa-cfcf-1adb-12eaa3b69498 Address:37991d03-32aa-cfcf-1adb-12eaa3b69498} {Suffrage:Voter ID:server-792e1d30-30af-6ad5-b688-5c14ba5d38c9 Address:792e1d30-30af-6ad5-b688-5c14ba5d38c9}]"
    testing.go:149: server-792e1d30-30af-6ad5-b688-5c14ba5d38c9: 2024-01-08T00:31:05.246Z [INFO]  server-792e1d30-30af-6ad5-b688-5c14ba5d38c9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-2af0776a-5a1b-5a54-d2d5-5626395e9402 Address:2af0776a-5a1b-5a54-d2d5-5626395e9402} {Suffrage:Voter ID:server-37991d03-32aa-cfcf-1adb-12eaa3b69498 Address:37991d03-32aa-cfcf-1adb-12eaa3b69498} {Suffrage:Voter ID:server-792e1d30-30af-6ad5-b688-5c14ba5d38c9 Address:792e1d30-30af-6ad5-b688-5c14ba5d38c9}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:05.247Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:05.247Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.247Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: entering follower state: follower="Node at 37991d03-32aa-cfcf-1adb-12eaa3b69498 [Follower]" leader=
    testing.go:149: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.247Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: entering follower state: follower="Node at 37991d03-32aa-cfcf-1adb-12eaa3b69498 [Follower]" leader=
testLoggerAdapter verbose: server-2af0776a-5a1b-5a54-d2d5-5626395e9402: 2024-01-08T00:31:05.248Z [INFO]  server-2af0776a-5a1b-5a54-d2d5-5626395e9402: entering follower state: follower="Node at 2af0776a-5a1b-5a54-d2d5-5626395e9402 [Follower]" leader=
    testing.go:149: server-2af0776a-5a1b-5a54-d2d5-5626395e9402: 2024-01-08T00:31:05.248Z [INFO]  server-2af0776a-5a1b-5a54-d2d5-5626395e9402: entering follower state: follower="Node at 2af0776a-5a1b-5a54-d2d5-5626395e9402 [Follower]" leader=
testLoggerAdapter verbose: server-792e1d30-30af-6ad5-b688-5c14ba5d38c9: 2024-01-08T00:31:05.248Z [INFO]  server-792e1d30-30af-6ad5-b688-5c14ba5d38c9: entering follower state: follower="Node at 792e1d30-30af-6ad5-b688-5c14ba5d38c9 [Follower]" leader=
    testing.go:149: server-792e1d30-30af-6ad5-b688-5c14ba5d38c9: 2024-01-08T00:31:05.248Z [INFO]  server-792e1d30-30af-6ad5-b688-5c14ba5d38c9: entering follower state: follower="Node at 792e1d30-30af-6ad5-b688-5c14ba5d38c9 [Follower]" leader=
testLoggerAdapter verbose: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.313Z [WARN]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.313Z [WARN]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.313Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: entering candidate state: node="Node at 37991d03-32aa-cfcf-1adb-12eaa3b69498 [Candidate]" term=2
    testing.go:149: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.313Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: entering candidate state: node="Node at 37991d03-32aa-cfcf-1adb-12eaa3b69498 [Candidate]" term=2
testLoggerAdapter verbose: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.313Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: election won: tally=2
    testing.go:149: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.313Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: election won: tally=2
testLoggerAdapter verbose: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.313Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: entering leader state: leader="Node at 37991d03-32aa-cfcf-1adb-12eaa3b69498 [Leader]"
    testing.go:149: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.313Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: entering leader state: leader="Node at 37991d03-32aa-cfcf-1adb-12eaa3b69498 [Leader]"
testLoggerAdapter verbose: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.314Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: added peer, starting replication: peer=server-2af0776a-5a1b-5a54-d2d5-5626395e9402
    testing.go:149: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.314Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: added peer, starting replication: peer=server-2af0776a-5a1b-5a54-d2d5-5626395e9402
testLoggerAdapter verbose: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.314Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: added peer, starting replication: peer=server-792e1d30-30af-6ad5-b688-5c14ba5d38c9
    testing.go:149: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.314Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: added peer, starting replication: peer=server-792e1d30-30af-6ad5-b688-5c14ba5d38c9
testLoggerAdapter verbose: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.314Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: pipelining replication: peer="{Voter server-792e1d30-30af-6ad5-b688-5c14ba5d38c9 792e1d30-30af-6ad5-b688-5c14ba5d38c9}"
    testing.go:149: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.314Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: pipelining replication: peer="{Voter server-792e1d30-30af-6ad5-b688-5c14ba5d38c9 792e1d30-30af-6ad5-b688-5c14ba5d38c9}"
testLoggerAdapter verbose: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.314Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: pipelining replication: peer="{Voter server-2af0776a-5a1b-5a54-d2d5-5626395e9402 2af0776a-5a1b-5a54-d2d5-5626395e9402}"
    testing.go:149: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.314Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: pipelining replication: peer="{Voter server-2af0776a-5a1b-5a54-d2d5-5626395e9402 2af0776a-5a1b-5a54-d2d5-5626395e9402}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:05.419Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:05.314299318 +0000 UTC m=+11.361188364 (1 nodes), 67.073214ms from start of poll, 67.906081ms from cluster start. Timeout at 2024-01-08 00:31:05.419573012 +0000 UTC m=+11.466462053, 105.273689ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:05.419Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:05.314299318 +0000 UTC m=+11.361188364 (1 nodes), 67.073214ms from start of poll, 67.906081ms from cluster start. Timeout at 2024-01-08 00:31:05.419573012 +0000 UTC m=+11.466462053, 105.273689ms after stability
testLoggerAdapter verbose: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.420Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: starting snapshot up to: index=12
    testing.go:149: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.420Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: starting snapshot up to: index=12
    testing.go:153: 2024-01-08T00:31:05.420Z [INFO]  creating new snapshot: path=/tmp/raft1640412050/snapshots/2-12-1704673865420.tmp
testLoggerAdapter verbose: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.428Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: snapshot complete up to: index=12
    testing.go:149: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.428Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: snapshot complete up to: index=12
    testing.go:153: 2024-01-08T00:31:05.429Z [INFO]  creating new snapshot: path=/tmp/raft1640412050/snapshots/2-23-1704673865429.tmp
testLoggerAdapter verbose: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.437Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: copied to local snapshot: bytes=71
    testing.go:149: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.437Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: copied to local snapshot: bytes=71
testLoggerAdapter verbose: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.437Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: restored user snapshot: index=1
    testing.go:149: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.437Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: restored user snapshot: index=1
testLoggerAdapter verbose: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.437Z [ERROR] server-37991d03-32aa-cfcf-1adb-12eaa3b69498: failed to get log: index=23 error="log not found"
    testing.go:149: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.437Z [ERROR] server-37991d03-32aa-cfcf-1adb-12eaa3b69498: failed to get log: index=23 error="log not found"
testLoggerAdapter verbose: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.437Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: aborting pipeline replication: peer="{Voter server-792e1d30-30af-6ad5-b688-5c14ba5d38c9 792e1d30-30af-6ad5-b688-5c14ba5d38c9}"
    testing.go:149: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.437Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: aborting pipeline replication: peer="{Voter server-792e1d30-30af-6ad5-b688-5c14ba5d38c9 792e1d30-30af-6ad5-b688-5c14ba5d38c9}"
testLoggerAdapter verbose: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.437Z [ERROR] server-37991d03-32aa-cfcf-1adb-12eaa3b69498: failed to get log: index=23 error="log not found"
    testing.go:149: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.437Z [ERROR] server-37991d03-32aa-cfcf-1adb-12eaa3b69498: failed to get log: index=23 error="log not found"
testLoggerAdapter verbose: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.437Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: aborting pipeline replication: peer="{Voter server-2af0776a-5a1b-5a54-d2d5-5626395e9402 2af0776a-5a1b-5a54-d2d5-5626395e9402}"
    testing.go:149: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.437Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: aborting pipeline replication: peer="{Voter server-2af0776a-5a1b-5a54-d2d5-5626395e9402 2af0776a-5a1b-5a54-d2d5-5626395e9402}"
testLoggerAdapter verbose: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.444Z [ERROR] server-37991d03-32aa-cfcf-1adb-12eaa3b69498: failed to get log: index=23 error="log not found"
    testing.go:149: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.444Z [ERROR] server-37991d03-32aa-cfcf-1adb-12eaa3b69498: failed to get log: index=23 error="log not found"
    testing.go:153: 2024-01-08T00:31:05.444Z [INFO]  creating new snapshot: path=/tmp/raft2790635236/snapshots/2-23-1704673865444.tmp
testLoggerAdapter verbose: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.447Z [ERROR] server-37991d03-32aa-cfcf-1adb-12eaa3b69498: failed to get log: index=23 error="log not found"
    testing.go:149: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.447Z [ERROR] server-37991d03-32aa-cfcf-1adb-12eaa3b69498: failed to get log: index=23 error="log not found"
    testing.go:153: 2024-01-08T00:31:05.447Z [INFO]  creating new snapshot: path=/tmp/raft889931139/snapshots/2-23-1704673865447.tmp
testLoggerAdapter verbose: server-2af0776a-5a1b-5a54-d2d5-5626395e9402: 2024-01-08T00:31:05.457Z [INFO]  server-2af0776a-5a1b-5a54-d2d5-5626395e9402: copied to local snapshot: bytes=71
    testing.go:149: server-2af0776a-5a1b-5a54-d2d5-5626395e9402: 2024-01-08T00:31:05.457Z [INFO]  server-2af0776a-5a1b-5a54-d2d5-5626395e9402: copied to local snapshot: bytes=71
testLoggerAdapter verbose: server-2af0776a-5a1b-5a54-d2d5-5626395e9402: 2024-01-08T00:31:05.457Z [INFO]  server-2af0776a-5a1b-5a54-d2d5-5626395e9402: Installed remote snapshot
    testing.go:149: server-2af0776a-5a1b-5a54-d2d5-5626395e9402: 2024-01-08T00:31:05.457Z [INFO]  server-2af0776a-5a1b-5a54-d2d5-5626395e9402: Installed remote snapshot
testLoggerAdapter verbose: server-2af0776a-5a1b-5a54-d2d5-5626395e9402: 2024-01-08T00:31:05.457Z [WARN]  server-2af0776a-5a1b-5a54-d2d5-5626395e9402: skipping application of old log: index=22
    testing.go:149: server-2af0776a-5a1b-5a54-d2d5-5626395e9402: 2024-01-08T00:31:05.457Z [WARN]  server-2af0776a-5a1b-5a54-d2d5-5626395e9402: skipping application of old log: index=22
testLoggerAdapter verbose: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.457Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: pipelining replication: peer="{Voter server-2af0776a-5a1b-5a54-d2d5-5626395e9402 2af0776a-5a1b-5a54-d2d5-5626395e9402}"
    testing.go:149: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.457Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: pipelining replication: peer="{Voter server-2af0776a-5a1b-5a54-d2d5-5626395e9402 2af0776a-5a1b-5a54-d2d5-5626395e9402}"
testLoggerAdapter verbose: server-792e1d30-30af-6ad5-b688-5c14ba5d38c9: 2024-01-08T00:31:05.459Z [INFO]  server-792e1d30-30af-6ad5-b688-5c14ba5d38c9: copied to local snapshot: bytes=71
    testing.go:149: server-792e1d30-30af-6ad5-b688-5c14ba5d38c9: 2024-01-08T00:31:05.459Z [INFO]  server-792e1d30-30af-6ad5-b688-5c14ba5d38c9: copied to local snapshot: bytes=71
testLoggerAdapter verbose: server-792e1d30-30af-6ad5-b688-5c14ba5d38c9: 2024-01-08T00:31:05.459Z [INFO]  server-792e1d30-30af-6ad5-b688-5c14ba5d38c9: Installed remote snapshot
    testing.go:149: server-792e1d30-30af-6ad5-b688-5c14ba5d38c9: 2024-01-08T00:31:05.459Z [INFO]  server-792e1d30-30af-6ad5-b688-5c14ba5d38c9: Installed remote snapshot
testLoggerAdapter verbose: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.459Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: pipelining replication: peer="{Voter server-792e1d30-30af-6ad5-b688-5c14ba5d38c9 792e1d30-30af-6ad5-b688-5c14ba5d38c9}"
    testing.go:149: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.459Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: pipelining replication: peer="{Voter server-792e1d30-30af-6ad5-b688-5c14ba5d38c9 792e1d30-30af-6ad5-b688-5c14ba5d38c9}"
testLoggerAdapter verbose: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.475Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: aborting pipeline replication: peer="{Voter server-792e1d30-30af-6ad5-b688-5c14ba5d38c9 792e1d30-30af-6ad5-b688-5c14ba5d38c9}"
    testing.go:149: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.475Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: aborting pipeline replication: peer="{Voter server-792e1d30-30af-6ad5-b688-5c14ba5d38c9 792e1d30-30af-6ad5-b688-5c14ba5d38c9}"
testLoggerAdapter verbose: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.475Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: aborting pipeline replication: peer="{Voter server-2af0776a-5a1b-5a54-d2d5-5626395e9402 2af0776a-5a1b-5a54-d2d5-5626395e9402}"
    testing.go:149: server-37991d03-32aa-cfcf-1adb-12eaa3b69498: 2024-01-08T00:31:05.475Z [INFO]  server-37991d03-32aa-cfcf-1adb-12eaa3b69498: aborting pipeline replication: peer="{Voter server-2af0776a-5a1b-5a54-d2d5-5626395e9402 2af0776a-5a1b-5a54-d2d5-5626395e9402}"
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.476Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9353476e-b9c8-84b8-dc54-c291e86420bf Address:9353476e-b9c8-84b8-dc54-c291e86420bf} {Suffrage:Voter ID:server-237d4f9a-e353-e960-cae3-f2ef9d843b0f Address:237d4f9a-e353-e960-cae3-f2ef9d843b0f} {Suffrage:Voter ID:server-ae651032-d12d-3749-0071-dc6b6cbe41b6 Address:ae651032-d12d-3749-0071-dc6b6cbe41b6}]"
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.476Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9353476e-b9c8-84b8-dc54-c291e86420bf Address:9353476e-b9c8-84b8-dc54-c291e86420bf} {Suffrage:Voter ID:server-237d4f9a-e353-e960-cae3-f2ef9d843b0f Address:237d4f9a-e353-e960-cae3-f2ef9d843b0f} {Suffrage:Voter ID:server-ae651032-d12d-3749-0071-dc6b6cbe41b6 Address:ae651032-d12d-3749-0071-dc6b6cbe41b6}]"
testLoggerAdapter verbose: server-237d4f9a-e353-e960-cae3-f2ef9d843b0f: 2024-01-08T00:31:05.476Z [INFO]  server-237d4f9a-e353-e960-cae3-f2ef9d843b0f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9353476e-b9c8-84b8-dc54-c291e86420bf Address:9353476e-b9c8-84b8-dc54-c291e86420bf} {Suffrage:Voter ID:server-237d4f9a-e353-e960-cae3-f2ef9d843b0f Address:237d4f9a-e353-e960-cae3-f2ef9d843b0f} {Suffrage:Voter ID:server-ae651032-d12d-3749-0071-dc6b6cbe41b6 Address:ae651032-d12d-3749-0071-dc6b6cbe41b6}]"
    testing.go:149: server-237d4f9a-e353-e960-cae3-f2ef9d843b0f: 2024-01-08T00:31:05.476Z [INFO]  server-237d4f9a-e353-e960-cae3-f2ef9d843b0f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9353476e-b9c8-84b8-dc54-c291e86420bf Address:9353476e-b9c8-84b8-dc54-c291e86420bf} {Suffrage:Voter ID:server-237d4f9a-e353-e960-cae3-f2ef9d843b0f Address:237d4f9a-e353-e960-cae3-f2ef9d843b0f} {Suffrage:Voter ID:server-ae651032-d12d-3749-0071-dc6b6cbe41b6 Address:ae651032-d12d-3749-0071-dc6b6cbe41b6}]"
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.476Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: entering follower state: follower="Node at 9353476e-b9c8-84b8-dc54-c291e86420bf [Follower]" leader=
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.476Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: entering follower state: follower="Node at 9353476e-b9c8-84b8-dc54-c291e86420bf [Follower]" leader=
testLoggerAdapter verbose: server-ae651032-d12d-3749-0071-dc6b6cbe41b6: 2024-01-08T00:31:05.476Z [INFO]  server-ae651032-d12d-3749-0071-dc6b6cbe41b6: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9353476e-b9c8-84b8-dc54-c291e86420bf Address:9353476e-b9c8-84b8-dc54-c291e86420bf} {Suffrage:Voter ID:server-237d4f9a-e353-e960-cae3-f2ef9d843b0f Address:237d4f9a-e353-e960-cae3-f2ef9d843b0f} {Suffrage:Voter ID:server-ae651032-d12d-3749-0071-dc6b6cbe41b6 Address:ae651032-d12d-3749-0071-dc6b6cbe41b6}]"
    testing.go:149: server-ae651032-d12d-3749-0071-dc6b6cbe41b6: 2024-01-08T00:31:05.476Z [INFO]  server-ae651032-d12d-3749-0071-dc6b6cbe41b6: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9353476e-b9c8-84b8-dc54-c291e86420bf Address:9353476e-b9c8-84b8-dc54-c291e86420bf} {Suffrage:Voter ID:server-237d4f9a-e353-e960-cae3-f2ef9d843b0f Address:237d4f9a-e353-e960-cae3-f2ef9d843b0f} {Suffrage:Voter ID:server-ae651032-d12d-3749-0071-dc6b6cbe41b6 Address:ae651032-d12d-3749-0071-dc6b6cbe41b6}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:05.477Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:05.477Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-237d4f9a-e353-e960-cae3-f2ef9d843b0f: 2024-01-08T00:31:05.476Z [INFO]  server-237d4f9a-e353-e960-cae3-f2ef9d843b0f: entering follower state: follower="Node at 237d4f9a-e353-e960-cae3-f2ef9d843b0f [Follower]" leader=
    testing.go:149: server-237d4f9a-e353-e960-cae3-f2ef9d843b0f: 2024-01-08T00:31:05.476Z [INFO]  server-237d4f9a-e353-e960-cae3-f2ef9d843b0f: entering follower state: follower="Node at 237d4f9a-e353-e960-cae3-f2ef9d843b0f [Follower]" leader=
testLoggerAdapter verbose: server-ae651032-d12d-3749-0071-dc6b6cbe41b6: 2024-01-08T00:31:05.477Z [INFO]  server-ae651032-d12d-3749-0071-dc6b6cbe41b6: entering follower state: follower="Node at ae651032-d12d-3749-0071-dc6b6cbe41b6 [Follower]" leader=
    testing.go:149: server-ae651032-d12d-3749-0071-dc6b6cbe41b6: 2024-01-08T00:31:05.477Z [INFO]  server-ae651032-d12d-3749-0071-dc6b6cbe41b6: entering follower state: follower="Node at ae651032-d12d-3749-0071-dc6b6cbe41b6 [Follower]" leader=
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.534Z [WARN]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.534Z [WARN]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.534Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: entering candidate state: node="Node at 9353476e-b9c8-84b8-dc54-c291e86420bf [Candidate]" term=2
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.534Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: entering candidate state: node="Node at 9353476e-b9c8-84b8-dc54-c291e86420bf [Candidate]" term=2
testLoggerAdapter verbose: server-ae651032-d12d-3749-0071-dc6b6cbe41b6: 2024-01-08T00:31:05.534Z [WARN]  server-ae651032-d12d-3749-0071-dc6b6cbe41b6: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-ae651032-d12d-3749-0071-dc6b6cbe41b6: 2024-01-08T00:31:05.534Z [WARN]  server-ae651032-d12d-3749-0071-dc6b6cbe41b6: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-ae651032-d12d-3749-0071-dc6b6cbe41b6: 2024-01-08T00:31:05.534Z [INFO]  server-ae651032-d12d-3749-0071-dc6b6cbe41b6: entering candidate state: node="Node at ae651032-d12d-3749-0071-dc6b6cbe41b6 [Candidate]" term=2
    testing.go:149: server-ae651032-d12d-3749-0071-dc6b6cbe41b6: 2024-01-08T00:31:05.534Z [INFO]  server-ae651032-d12d-3749-0071-dc6b6cbe41b6: entering candidate state: node="Node at ae651032-d12d-3749-0071-dc6b6cbe41b6 [Candidate]" term=2
testLoggerAdapter verbose: server-ae651032-d12d-3749-0071-dc6b6cbe41b6: 2024-01-08T00:31:05.534Z [INFO]  server-ae651032-d12d-3749-0071-dc6b6cbe41b6: duplicate requestVote for same term: term=2
    testing.go:149: server-ae651032-d12d-3749-0071-dc6b6cbe41b6: 2024-01-08T00:31:05.534Z [INFO]  server-ae651032-d12d-3749-0071-dc6b6cbe41b6: duplicate requestVote for same term: term=2
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.534Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: election won: tally=2
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.534Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: election won: tally=2
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.534Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: entering leader state: leader="Node at 9353476e-b9c8-84b8-dc54-c291e86420bf [Leader]"
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.534Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: entering leader state: leader="Node at 9353476e-b9c8-84b8-dc54-c291e86420bf [Leader]"
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.534Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: added peer, starting replication: peer=server-237d4f9a-e353-e960-cae3-f2ef9d843b0f
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.534Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: added peer, starting replication: peer=server-237d4f9a-e353-e960-cae3-f2ef9d843b0f
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.534Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: added peer, starting replication: peer=server-ae651032-d12d-3749-0071-dc6b6cbe41b6
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.534Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: added peer, starting replication: peer=server-ae651032-d12d-3749-0071-dc6b6cbe41b6
testLoggerAdapter verbose: server-ae651032-d12d-3749-0071-dc6b6cbe41b6: 2024-01-08T00:31:05.535Z [INFO]  server-ae651032-d12d-3749-0071-dc6b6cbe41b6: entering follower state: follower="Node at ae651032-d12d-3749-0071-dc6b6cbe41b6 [Follower]" leader=9353476e-b9c8-84b8-dc54-c291e86420bf
    testing.go:149: server-ae651032-d12d-3749-0071-dc6b6cbe41b6: 2024-01-08T00:31:05.535Z [INFO]  server-ae651032-d12d-3749-0071-dc6b6cbe41b6: entering follower state: follower="Node at ae651032-d12d-3749-0071-dc6b6cbe41b6 [Follower]" leader=9353476e-b9c8-84b8-dc54-c291e86420bf
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.535Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: pipelining replication: peer="{Voter server-ae651032-d12d-3749-0071-dc6b6cbe41b6 ae651032-d12d-3749-0071-dc6b6cbe41b6}"
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.535Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: pipelining replication: peer="{Voter server-ae651032-d12d-3749-0071-dc6b6cbe41b6 ae651032-d12d-3749-0071-dc6b6cbe41b6}"
testLoggerAdapter verbose: server-237d4f9a-e353-e960-cae3-f2ef9d843b0f: 2024-01-08T00:31:05.535Z [INFO]  server-237d4f9a-e353-e960-cae3-f2ef9d843b0f: duplicate requestVote for same term: term=2
    testing.go:149: server-237d4f9a-e353-e960-cae3-f2ef9d843b0f: 2024-01-08T00:31:05.535Z [INFO]  server-237d4f9a-e353-e960-cae3-f2ef9d843b0f: duplicate requestVote for same term: term=2
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.535Z [WARN]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: rejecting vote request since we have a leader: from=ae651032-d12d-3749-0071-dc6b6cbe41b6 leader=9353476e-b9c8-84b8-dc54-c291e86420bf
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.535Z [WARN]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: rejecting vote request since we have a leader: from=ae651032-d12d-3749-0071-dc6b6cbe41b6 leader=9353476e-b9c8-84b8-dc54-c291e86420bf
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.535Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: pipelining replication: peer="{Voter server-237d4f9a-e353-e960-cae3-f2ef9d843b0f 237d4f9a-e353-e960-cae3-f2ef9d843b0f}"
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.535Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: pipelining replication: peer="{Voter server-237d4f9a-e353-e960-cae3-f2ef9d843b0f 237d4f9a-e353-e960-cae3-f2ef9d843b0f}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:05.640Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:05.535218823 +0000 UTC m=+11.582107894 (1 nodes), 58.141283ms from start of poll, 58.746131ms from cluster start. Timeout at 2024-01-08 00:31:05.640702468 +0000 UTC m=+11.687591539, 105.483645ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:05.640Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:05.535218823 +0000 UTC m=+11.582107894 (1 nodes), 58.141283ms from start of poll, 58.746131ms from cluster start. Timeout at 2024-01-08 00:31:05.640702468 +0000 UTC m=+11.687591539, 105.483645ms after stability
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.641Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: starting snapshot up to: index=12
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.641Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: starting snapshot up to: index=12
    testing.go:153: 2024-01-08T00:31:05.641Z [INFO]  creating new snapshot: path=/tmp/raft511083705/snapshots/2-12-1704673865641.tmp
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.650Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: snapshot complete up to: index=12
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.650Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: snapshot complete up to: index=12
    testing.go:153: 2024-01-08T00:31:05.650Z [INFO]  creating new snapshot: path=/tmp/raft511083705/snapshots/2-23-1704673865650.tmp
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.662Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: copied to local snapshot: bytes=71
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.662Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: copied to local snapshot: bytes=71
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.663Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: restored user snapshot: index=1
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.663Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: restored user snapshot: index=1
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.663Z [ERROR] server-9353476e-b9c8-84b8-dc54-c291e86420bf: failed to get log: index=23 error="log not found"
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.663Z [ERROR] server-9353476e-b9c8-84b8-dc54-c291e86420bf: failed to get log: index=23 error="log not found"
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.663Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: aborting pipeline replication: peer="{Voter server-237d4f9a-e353-e960-cae3-f2ef9d843b0f 237d4f9a-e353-e960-cae3-f2ef9d843b0f}"
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.663Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: aborting pipeline replication: peer="{Voter server-237d4f9a-e353-e960-cae3-f2ef9d843b0f 237d4f9a-e353-e960-cae3-f2ef9d843b0f}"
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.663Z [ERROR] server-9353476e-b9c8-84b8-dc54-c291e86420bf: failed to get log: index=23 error="log not found"
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.663Z [ERROR] server-9353476e-b9c8-84b8-dc54-c291e86420bf: failed to get log: index=23 error="log not found"
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.663Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: aborting pipeline replication: peer="{Voter server-ae651032-d12d-3749-0071-dc6b6cbe41b6 ae651032-d12d-3749-0071-dc6b6cbe41b6}"
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.663Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: aborting pipeline replication: peer="{Voter server-ae651032-d12d-3749-0071-dc6b6cbe41b6 ae651032-d12d-3749-0071-dc6b6cbe41b6}"
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.669Z [ERROR] server-9353476e-b9c8-84b8-dc54-c291e86420bf: failed to get log: index=23 error="log not found"
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.669Z [ERROR] server-9353476e-b9c8-84b8-dc54-c291e86420bf: failed to get log: index=23 error="log not found"
    testing.go:153: 2024-01-08T00:31:05.669Z [INFO]  creating new snapshot: path=/tmp/raft3518562365/snapshots/2-23-1704673865669.tmp
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.672Z [ERROR] server-9353476e-b9c8-84b8-dc54-c291e86420bf: failed to get log: index=23 error="log not found"
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.672Z [ERROR] server-9353476e-b9c8-84b8-dc54-c291e86420bf: failed to get log: index=23 error="log not found"
    testing.go:153: 2024-01-08T00:31:05.673Z [INFO]  creating new snapshot: path=/tmp/raft1169431483/snapshots/2-23-1704673865673.tmp
testLoggerAdapter verbose: server-237d4f9a-e353-e960-cae3-f2ef9d843b0f: 2024-01-08T00:31:05.680Z [INFO]  server-237d4f9a-e353-e960-cae3-f2ef9d843b0f: copied to local snapshot: bytes=71
    testing.go:149: server-237d4f9a-e353-e960-cae3-f2ef9d843b0f: 2024-01-08T00:31:05.680Z [INFO]  server-237d4f9a-e353-e960-cae3-f2ef9d843b0f: copied to local snapshot: bytes=71
testLoggerAdapter verbose: server-237d4f9a-e353-e960-cae3-f2ef9d843b0f: 2024-01-08T00:31:05.681Z [INFO]  server-237d4f9a-e353-e960-cae3-f2ef9d843b0f: Installed remote snapshot
    testing.go:149: server-237d4f9a-e353-e960-cae3-f2ef9d843b0f: 2024-01-08T00:31:05.681Z [INFO]  server-237d4f9a-e353-e960-cae3-f2ef9d843b0f: Installed remote snapshot
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.681Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: pipelining replication: peer="{Voter server-237d4f9a-e353-e960-cae3-f2ef9d843b0f 237d4f9a-e353-e960-cae3-f2ef9d843b0f}"
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.681Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: pipelining replication: peer="{Voter server-237d4f9a-e353-e960-cae3-f2ef9d843b0f 237d4f9a-e353-e960-cae3-f2ef9d843b0f}"
testLoggerAdapter verbose: server-ae651032-d12d-3749-0071-dc6b6cbe41b6: 2024-01-08T00:31:05.683Z [INFO]  server-ae651032-d12d-3749-0071-dc6b6cbe41b6: copied to local snapshot: bytes=71
    testing.go:149: server-ae651032-d12d-3749-0071-dc6b6cbe41b6: 2024-01-08T00:31:05.683Z [INFO]  server-ae651032-d12d-3749-0071-dc6b6cbe41b6: copied to local snapshot: bytes=71
testLoggerAdapter verbose: server-ae651032-d12d-3749-0071-dc6b6cbe41b6: 2024-01-08T00:31:05.683Z [INFO]  server-ae651032-d12d-3749-0071-dc6b6cbe41b6: Installed remote snapshot
    testing.go:149: server-ae651032-d12d-3749-0071-dc6b6cbe41b6: 2024-01-08T00:31:05.683Z [INFO]  server-ae651032-d12d-3749-0071-dc6b6cbe41b6: Installed remote snapshot
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.683Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: pipelining replication: peer="{Voter server-ae651032-d12d-3749-0071-dc6b6cbe41b6 ae651032-d12d-3749-0071-dc6b6cbe41b6}"
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.683Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: pipelining replication: peer="{Voter server-ae651032-d12d-3749-0071-dc6b6cbe41b6 ae651032-d12d-3749-0071-dc6b6cbe41b6}"
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.697Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: aborting pipeline replication: peer="{Voter server-ae651032-d12d-3749-0071-dc6b6cbe41b6 ae651032-d12d-3749-0071-dc6b6cbe41b6}"
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.697Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: aborting pipeline replication: peer="{Voter server-ae651032-d12d-3749-0071-dc6b6cbe41b6 ae651032-d12d-3749-0071-dc6b6cbe41b6}"
testLoggerAdapter verbose: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.697Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: aborting pipeline replication: peer="{Voter server-237d4f9a-e353-e960-cae3-f2ef9d843b0f 237d4f9a-e353-e960-cae3-f2ef9d843b0f}"
    testing.go:149: server-9353476e-b9c8-84b8-dc54-c291e86420bf: 2024-01-08T00:31:05.697Z [INFO]  server-9353476e-b9c8-84b8-dc54-c291e86420bf: aborting pipeline replication: peer="{Voter server-237d4f9a-e353-e960-cae3-f2ef9d843b0f 237d4f9a-e353-e960-cae3-f2ef9d843b0f}"
testLoggerAdapter verbose: server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a: 2024-01-08T00:31:05.699Z [INFO]  server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a Address:42eaf54d-e01f-ab9a-3600-082b2aa9f99a} {Suffrage:Voter ID:server-8dbbcae6-dc88-aab9-703d-92afe3b4448c Address:8dbbcae6-dc88-aab9-703d-92afe3b4448c} {Suffrage:Voter ID:server-0b54f66c-220a-d219-88a5-afa3e4d794ab Address:0b54f66c-220a-d219-88a5-afa3e4d794ab}]"
    testing.go:149: server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a: 2024-01-08T00:31:05.699Z [INFO]  server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a Address:42eaf54d-e01f-ab9a-3600-082b2aa9f99a} {Suffrage:Voter ID:server-8dbbcae6-dc88-aab9-703d-92afe3b4448c Address:8dbbcae6-dc88-aab9-703d-92afe3b4448c} {Suffrage:Voter ID:server-0b54f66c-220a-d219-88a5-afa3e4d794ab Address:0b54f66c-220a-d219-88a5-afa3e4d794ab}]"
testLoggerAdapter verbose: server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: 2024-01-08T00:31:05.699Z [INFO]  server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a Address:42eaf54d-e01f-ab9a-3600-082b2aa9f99a} {Suffrage:Voter ID:server-8dbbcae6-dc88-aab9-703d-92afe3b4448c Address:8dbbcae6-dc88-aab9-703d-92afe3b4448c} {Suffrage:Voter ID:server-0b54f66c-220a-d219-88a5-afa3e4d794ab Address:0b54f66c-220a-d219-88a5-afa3e4d794ab}]"
    testing.go:149: server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: 2024-01-08T00:31:05.699Z [INFO]  server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a Address:42eaf54d-e01f-ab9a-3600-082b2aa9f99a} {Suffrage:Voter ID:server-8dbbcae6-dc88-aab9-703d-92afe3b4448c Address:8dbbcae6-dc88-aab9-703d-92afe3b4448c} {Suffrage:Voter ID:server-0b54f66c-220a-d219-88a5-afa3e4d794ab Address:0b54f66c-220a-d219-88a5-afa3e4d794ab}]"
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.699Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a Address:42eaf54d-e01f-ab9a-3600-082b2aa9f99a} {Suffrage:Voter ID:server-8dbbcae6-dc88-aab9-703d-92afe3b4448c Address:8dbbcae6-dc88-aab9-703d-92afe3b4448c} {Suffrage:Voter ID:server-0b54f66c-220a-d219-88a5-afa3e4d794ab Address:0b54f66c-220a-d219-88a5-afa3e4d794ab}]"
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.699Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a Address:42eaf54d-e01f-ab9a-3600-082b2aa9f99a} {Suffrage:Voter ID:server-8dbbcae6-dc88-aab9-703d-92afe3b4448c Address:8dbbcae6-dc88-aab9-703d-92afe3b4448c} {Suffrage:Voter ID:server-0b54f66c-220a-d219-88a5-afa3e4d794ab Address:0b54f66c-220a-d219-88a5-afa3e4d794ab}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:05.699Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:05.699Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a: 2024-01-08T00:31:05.699Z [INFO]  server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a: entering follower state: follower="Node at 42eaf54d-e01f-ab9a-3600-082b2aa9f99a [Follower]" leader=
    testing.go:149: server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a: 2024-01-08T00:31:05.699Z [INFO]  server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a: entering follower state: follower="Node at 42eaf54d-e01f-ab9a-3600-082b2aa9f99a [Follower]" leader=
testLoggerAdapter verbose: server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: 2024-01-08T00:31:05.699Z [INFO]  server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: entering follower state: follower="Node at 8dbbcae6-dc88-aab9-703d-92afe3b4448c [Follower]" leader=
    testing.go:149: server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: 2024-01-08T00:31:05.699Z [INFO]  server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: entering follower state: follower="Node at 8dbbcae6-dc88-aab9-703d-92afe3b4448c [Follower]" leader=
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.699Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: entering follower state: follower="Node at 0b54f66c-220a-d219-88a5-afa3e4d794ab [Follower]" leader=
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.699Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: entering follower state: follower="Node at 0b54f66c-220a-d219-88a5-afa3e4d794ab [Follower]" leader=
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.753Z [WARN]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.753Z [WARN]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.754Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: entering candidate state: node="Node at 0b54f66c-220a-d219-88a5-afa3e4d794ab [Candidate]" term=2
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.754Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: entering candidate state: node="Node at 0b54f66c-220a-d219-88a5-afa3e4d794ab [Candidate]" term=2
testLoggerAdapter verbose: server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: 2024-01-08T00:31:05.754Z [WARN]  server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: 2024-01-08T00:31:05.754Z [WARN]  server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: 2024-01-08T00:31:05.754Z [INFO]  server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: entering candidate state: node="Node at 8dbbcae6-dc88-aab9-703d-92afe3b4448c [Candidate]" term=2
    testing.go:149: server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: 2024-01-08T00:31:05.754Z [INFO]  server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: entering candidate state: node="Node at 8dbbcae6-dc88-aab9-703d-92afe3b4448c [Candidate]" term=2
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.754Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: election won: tally=2
testLoggerAdapter verbose: server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: 2024-01-08T00:31:05.754Z [INFO]  server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: duplicate requestVote for same term: term=2
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.754Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: election won: tally=2
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.754Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: entering leader state: leader="Node at 0b54f66c-220a-d219-88a5-afa3e4d794ab [Leader]"
testLoggerAdapter verbose: server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a: 2024-01-08T00:31:05.754Z [INFO]  server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a: duplicate requestVote for same term: term=2
    testing.go:149: server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: 2024-01-08T00:31:05.754Z [INFO]  server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: duplicate requestVote for same term: term=2
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.754Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: entering leader state: leader="Node at 0b54f66c-220a-d219-88a5-afa3e4d794ab [Leader]"
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.754Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: added peer, starting replication: peer=server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.754Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: added peer, starting replication: peer=server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.754Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: added peer, starting replication: peer=server-8dbbcae6-dc88-aab9-703d-92afe3b4448c
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.754Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: added peer, starting replication: peer=server-8dbbcae6-dc88-aab9-703d-92afe3b4448c
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.754Z [WARN]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: rejecting vote request since we have a leader: from=8dbbcae6-dc88-aab9-703d-92afe3b4448c leader=0b54f66c-220a-d219-88a5-afa3e4d794ab
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.754Z [WARN]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: rejecting vote request since we have a leader: from=8dbbcae6-dc88-aab9-703d-92afe3b4448c leader=0b54f66c-220a-d219-88a5-afa3e4d794ab
    testing.go:149: server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a: 2024-01-08T00:31:05.754Z [INFO]  server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a: duplicate requestVote for same term: term=2
testLoggerAdapter verbose: server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: 2024-01-08T00:31:05.754Z [INFO]  server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: entering follower state: follower="Node at 8dbbcae6-dc88-aab9-703d-92afe3b4448c [Follower]" leader=0b54f66c-220a-d219-88a5-afa3e4d794ab
    testing.go:149: server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: 2024-01-08T00:31:05.754Z [INFO]  server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: entering follower state: follower="Node at 8dbbcae6-dc88-aab9-703d-92afe3b4448c [Follower]" leader=0b54f66c-220a-d219-88a5-afa3e4d794ab
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.754Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: pipelining replication: peer="{Voter server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a 42eaf54d-e01f-ab9a-3600-082b2aa9f99a}"
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.754Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: pipelining replication: peer="{Voter server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a 42eaf54d-e01f-ab9a-3600-082b2aa9f99a}"
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.754Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: pipelining replication: peer="{Voter server-8dbbcae6-dc88-aab9-703d-92afe3b4448c 8dbbcae6-dc88-aab9-703d-92afe3b4448c}"
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.754Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: pipelining replication: peer="{Voter server-8dbbcae6-dc88-aab9-703d-92afe3b4448c 8dbbcae6-dc88-aab9-703d-92afe3b4448c}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:05.860Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:05.754598958 +0000 UTC m=+11.801488019 (1 nodes), 55.116941ms from start of poll, 55.538697ms from cluster start. Timeout at 2024-01-08 00:31:05.860045736 +0000 UTC m=+11.906934818, 105.446799ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:05.860Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:05.754598958 +0000 UTC m=+11.801488019 (1 nodes), 55.116941ms from start of poll, 55.538697ms from cluster start. Timeout at 2024-01-08 00:31:05.860045736 +0000 UTC m=+11.906934818, 105.446799ms after stability
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.860Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: starting snapshot up to: index=12
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.860Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: starting snapshot up to: index=12
    testing.go:153: 2024-01-08T00:31:05.860Z [INFO]  creating new snapshot: path=/tmp/raft1611914947/snapshots/2-12-1704673865860.tmp
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.869Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: snapshot complete up to: index=12
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.869Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: snapshot complete up to: index=12
    testing.go:153: 2024-01-08T00:31:05.869Z [INFO]  creating new snapshot: path=/tmp/raft1611914947/snapshots/2-113-1704673865869.tmp
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.878Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: copied to local snapshot: bytes=71
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.878Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: copied to local snapshot: bytes=71
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.878Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: restored user snapshot: index=1
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.878Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: restored user snapshot: index=1
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.878Z [ERROR] server-0b54f66c-220a-d219-88a5-afa3e4d794ab: failed to get log: index=23 error="log not found"
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.878Z [ERROR] server-0b54f66c-220a-d219-88a5-afa3e4d794ab: failed to get log: index=23 error="log not found"
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.878Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: aborting pipeline replication: peer="{Voter server-8dbbcae6-dc88-aab9-703d-92afe3b4448c 8dbbcae6-dc88-aab9-703d-92afe3b4448c}"
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.878Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: aborting pipeline replication: peer="{Voter server-8dbbcae6-dc88-aab9-703d-92afe3b4448c 8dbbcae6-dc88-aab9-703d-92afe3b4448c}"
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.878Z [ERROR] server-0b54f66c-220a-d219-88a5-afa3e4d794ab: failed to get log: index=23 error="log not found"
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.878Z [ERROR] server-0b54f66c-220a-d219-88a5-afa3e4d794ab: failed to get log: index=23 error="log not found"
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.878Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: aborting pipeline replication: peer="{Voter server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a 42eaf54d-e01f-ab9a-3600-082b2aa9f99a}"
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.878Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: aborting pipeline replication: peer="{Voter server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a 42eaf54d-e01f-ab9a-3600-082b2aa9f99a}"
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.884Z [ERROR] server-0b54f66c-220a-d219-88a5-afa3e4d794ab: failed to get log: index=23 error="log not found"
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.884Z [ERROR] server-0b54f66c-220a-d219-88a5-afa3e4d794ab: failed to get log: index=23 error="log not found"
    testing.go:153: 2024-01-08T00:31:05.885Z [INFO]  creating new snapshot: path=/tmp/raft3305911943/snapshots/2-113-1704673865885.tmp
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.886Z [ERROR] server-0b54f66c-220a-d219-88a5-afa3e4d794ab: failed to get log: index=23 error="log not found"
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.886Z [ERROR] server-0b54f66c-220a-d219-88a5-afa3e4d794ab: failed to get log: index=23 error="log not found"
    testing.go:153: 2024-01-08T00:31:05.887Z [INFO]  creating new snapshot: path=/tmp/raft2474434758/snapshots/2-113-1704673865887.tmp
testLoggerAdapter verbose: server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a: 2024-01-08T00:31:05.896Z [INFO]  server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a: copied to local snapshot: bytes=71
    testing.go:149: server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a: 2024-01-08T00:31:05.896Z [INFO]  server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a: copied to local snapshot: bytes=71
testLoggerAdapter verbose: server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a: 2024-01-08T00:31:05.896Z [INFO]  server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a: Installed remote snapshot
    testing.go:149: server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a: 2024-01-08T00:31:05.896Z [INFO]  server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a: Installed remote snapshot
testLoggerAdapter verbose: server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a: 2024-01-08T00:31:05.896Z [WARN]  server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a: skipping application of old log: index=113
    testing.go:149: server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a: 2024-01-08T00:31:05.896Z [WARN]  server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a: skipping application of old log: index=113
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.897Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: pipelining replication: peer="{Voter server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a 42eaf54d-e01f-ab9a-3600-082b2aa9f99a}"
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.897Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: pipelining replication: peer="{Voter server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a 42eaf54d-e01f-ab9a-3600-082b2aa9f99a}"
testLoggerAdapter verbose: server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: 2024-01-08T00:31:05.896Z [INFO]  server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: copied to local snapshot: bytes=71
    testing.go:149: server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: 2024-01-08T00:31:05.896Z [INFO]  server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: copied to local snapshot: bytes=71
testLoggerAdapter verbose: server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: 2024-01-08T00:31:05.897Z [INFO]  server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: Installed remote snapshot
    testing.go:149: server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: 2024-01-08T00:31:05.897Z [INFO]  server-8dbbcae6-dc88-aab9-703d-92afe3b4448c: Installed remote snapshot
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.897Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: pipelining replication: peer="{Voter server-8dbbcae6-dc88-aab9-703d-92afe3b4448c 8dbbcae6-dc88-aab9-703d-92afe3b4448c}"
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.897Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: pipelining replication: peer="{Voter server-8dbbcae6-dc88-aab9-703d-92afe3b4448c 8dbbcae6-dc88-aab9-703d-92afe3b4448c}"
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.909Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: aborting pipeline replication: peer="{Voter server-8dbbcae6-dc88-aab9-703d-92afe3b4448c 8dbbcae6-dc88-aab9-703d-92afe3b4448c}"
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.909Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: aborting pipeline replication: peer="{Voter server-8dbbcae6-dc88-aab9-703d-92afe3b4448c 8dbbcae6-dc88-aab9-703d-92afe3b4448c}"
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.909Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: aborting pipeline replication: peer="{Voter server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a 42eaf54d-e01f-ab9a-3600-082b2aa9f99a}"
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.909Z [INFO]  server-0b54f66c-220a-d219-88a5-afa3e4d794ab: aborting pipeline replication: peer="{Voter server-42eaf54d-e01f-ab9a-3600-082b2aa9f99a 42eaf54d-e01f-ab9a-3600-082b2aa9f99a}"
testLoggerAdapter verbose: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.960Z [ERROR] server-0b54f66c-220a-d219-88a5-afa3e4d794ab: failed to heartbeat to: peer=42eaf54d-e01f-ab9a-3600-082b2aa9f99a error="command timed out"
    testing.go:149: server-0b54f66c-220a-d219-88a5-afa3e4d794ab: 2024-01-08T00:31:05.960Z [ERROR] server-0b54f66c-220a-d219-88a5-afa3e4d794ab: failed to heartbeat to: peer=42eaf54d-e01f-ab9a-3600-082b2aa9f99a error="command timed out"
testLoggerAdapter verbose: server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84: 2024-01-08T00:31:05.962Z [INFO]  server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84 Address:3515b721-0e5e-91ac-2fa2-49f75a5b9a84} {Suffrage:Voter ID:server-240e00b6-e276-bafc-8be2-7e927c3b9b9c Address:240e00b6-e276-bafc-8be2-7e927c3b9b9c} {Suffrage:Voter ID:server-d904b220-5b5c-c279-16b3-99f3c5035f9a Address:d904b220-5b5c-c279-16b3-99f3c5035f9a}]"
    testing.go:149: server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84: 2024-01-08T00:31:05.962Z [INFO]  server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84 Address:3515b721-0e5e-91ac-2fa2-49f75a5b9a84} {Suffrage:Voter ID:server-240e00b6-e276-bafc-8be2-7e927c3b9b9c Address:240e00b6-e276-bafc-8be2-7e927c3b9b9c} {Suffrage:Voter ID:server-d904b220-5b5c-c279-16b3-99f3c5035f9a Address:d904b220-5b5c-c279-16b3-99f3c5035f9a}]"
testLoggerAdapter verbose: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:05.962Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84 Address:3515b721-0e5e-91ac-2fa2-49f75a5b9a84} {Suffrage:Voter ID:server-240e00b6-e276-bafc-8be2-7e927c3b9b9c Address:240e00b6-e276-bafc-8be2-7e927c3b9b9c} {Suffrage:Voter ID:server-d904b220-5b5c-c279-16b3-99f3c5035f9a Address:d904b220-5b5c-c279-16b3-99f3c5035f9a}]"
    testing.go:149: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:05.962Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84 Address:3515b721-0e5e-91ac-2fa2-49f75a5b9a84} {Suffrage:Voter ID:server-240e00b6-e276-bafc-8be2-7e927c3b9b9c Address:240e00b6-e276-bafc-8be2-7e927c3b9b9c} {Suffrage:Voter ID:server-d904b220-5b5c-c279-16b3-99f3c5035f9a Address:d904b220-5b5c-c279-16b3-99f3c5035f9a}]"
testLoggerAdapter verbose: server-d904b220-5b5c-c279-16b3-99f3c5035f9a: 2024-01-08T00:31:05.962Z [INFO]  server-d904b220-5b5c-c279-16b3-99f3c5035f9a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84 Address:3515b721-0e5e-91ac-2fa2-49f75a5b9a84} {Suffrage:Voter ID:server-240e00b6-e276-bafc-8be2-7e927c3b9b9c Address:240e00b6-e276-bafc-8be2-7e927c3b9b9c} {Suffrage:Voter ID:server-d904b220-5b5c-c279-16b3-99f3c5035f9a Address:d904b220-5b5c-c279-16b3-99f3c5035f9a}]"
    testing.go:149: server-d904b220-5b5c-c279-16b3-99f3c5035f9a: 2024-01-08T00:31:05.962Z [INFO]  server-d904b220-5b5c-c279-16b3-99f3c5035f9a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84 Address:3515b721-0e5e-91ac-2fa2-49f75a5b9a84} {Suffrage:Voter ID:server-240e00b6-e276-bafc-8be2-7e927c3b9b9c Address:240e00b6-e276-bafc-8be2-7e927c3b9b9c} {Suffrage:Voter ID:server-d904b220-5b5c-c279-16b3-99f3c5035f9a Address:d904b220-5b5c-c279-16b3-99f3c5035f9a}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:05.962Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:05.962Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84: 2024-01-08T00:31:05.962Z [INFO]  server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84: entering follower state: follower="Node at 3515b721-0e5e-91ac-2fa2-49f75a5b9a84 [Follower]" leader=
    testing.go:149: server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84: 2024-01-08T00:31:05.962Z [INFO]  server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84: entering follower state: follower="Node at 3515b721-0e5e-91ac-2fa2-49f75a5b9a84 [Follower]" leader=
testLoggerAdapter verbose: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:05.962Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: entering follower state: follower="Node at 240e00b6-e276-bafc-8be2-7e927c3b9b9c [Follower]" leader=
    testing.go:149: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:05.962Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: entering follower state: follower="Node at 240e00b6-e276-bafc-8be2-7e927c3b9b9c [Follower]" leader=
testLoggerAdapter verbose: server-d904b220-5b5c-c279-16b3-99f3c5035f9a: 2024-01-08T00:31:05.962Z [INFO]  server-d904b220-5b5c-c279-16b3-99f3c5035f9a: entering follower state: follower="Node at d904b220-5b5c-c279-16b3-99f3c5035f9a [Follower]" leader=
    testing.go:149: server-d904b220-5b5c-c279-16b3-99f3c5035f9a: 2024-01-08T00:31:05.962Z [INFO]  server-d904b220-5b5c-c279-16b3-99f3c5035f9a: entering follower state: follower="Node at d904b220-5b5c-c279-16b3-99f3c5035f9a [Follower]" leader=
testLoggerAdapter verbose: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.028Z [WARN]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.028Z [WARN]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.028Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: entering candidate state: node="Node at 240e00b6-e276-bafc-8be2-7e927c3b9b9c [Candidate]" term=2
    testing.go:149: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.028Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: entering candidate state: node="Node at 240e00b6-e276-bafc-8be2-7e927c3b9b9c [Candidate]" term=2
testLoggerAdapter verbose: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.028Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: election won: tally=2
    testing.go:149: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.028Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: election won: tally=2
testLoggerAdapter verbose: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.028Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: entering leader state: leader="Node at 240e00b6-e276-bafc-8be2-7e927c3b9b9c [Leader]"
    testing.go:149: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.028Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: entering leader state: leader="Node at 240e00b6-e276-bafc-8be2-7e927c3b9b9c [Leader]"
testLoggerAdapter verbose: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.028Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: added peer, starting replication: peer=server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84
    testing.go:149: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.028Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: added peer, starting replication: peer=server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84
testLoggerAdapter verbose: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.028Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: added peer, starting replication: peer=server-d904b220-5b5c-c279-16b3-99f3c5035f9a
    testing.go:149: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.028Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: added peer, starting replication: peer=server-d904b220-5b5c-c279-16b3-99f3c5035f9a
testLoggerAdapter verbose: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.028Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: pipelining replication: peer="{Voter server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84 3515b721-0e5e-91ac-2fa2-49f75a5b9a84}"
    testing.go:149: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.028Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: pipelining replication: peer="{Voter server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84 3515b721-0e5e-91ac-2fa2-49f75a5b9a84}"
testLoggerAdapter verbose: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.028Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: pipelining replication: peer="{Voter server-d904b220-5b5c-c279-16b3-99f3c5035f9a d904b220-5b5c-c279-16b3-99f3c5035f9a}"
    testing.go:149: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.028Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: pipelining replication: peer="{Voter server-d904b220-5b5c-c279-16b3-99f3c5035f9a d904b220-5b5c-c279-16b3-99f3c5035f9a}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:06.134Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:06.028513699 +0000 UTC m=+12.075402840 (1 nodes), 65.696728ms from start of poll, 66.135668ms from cluster start. Timeout at 2024-01-08 00:31:06.134408755 +0000 UTC m=+12.181297886, 105.895046ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:06.134Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:06.028513699 +0000 UTC m=+12.075402840 (1 nodes), 65.696728ms from start of poll, 66.135668ms from cluster start. Timeout at 2024-01-08 00:31:06.134408755 +0000 UTC m=+12.181297886, 105.895046ms after stability
testLoggerAdapter verbose: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.134Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: starting snapshot up to: index=12
    testing.go:149: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.134Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: starting snapshot up to: index=12
    testing.go:153: 2024-01-08T00:31:06.135Z [INFO]  creating new snapshot: path=/tmp/raft351508388/snapshots/2-12-1704673866135.tmp
testLoggerAdapter verbose: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.142Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: snapshot complete up to: index=12
    testing.go:149: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.142Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: snapshot complete up to: index=12
    testing.go:153: 2024-01-08T00:31:06.143Z [INFO]  creating new snapshot: path=/tmp/raft351508388/snapshots/2-1013-1704673866143.tmp
testLoggerAdapter verbose: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.150Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: copied to local snapshot: bytes=71
    testing.go:149: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.150Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: copied to local snapshot: bytes=71
testLoggerAdapter verbose: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.151Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: restored user snapshot: index=1
    testing.go:149: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.151Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: restored user snapshot: index=1
testLoggerAdapter verbose: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.151Z [ERROR] server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: failed to get log: index=23 error="log not found"
    testing.go:149: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.151Z [ERROR] server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: failed to get log: index=23 error="log not found"
testLoggerAdapter verbose: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.151Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: aborting pipeline replication: peer="{Voter server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84 3515b721-0e5e-91ac-2fa2-49f75a5b9a84}"
    testing.go:149: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.151Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: aborting pipeline replication: peer="{Voter server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84 3515b721-0e5e-91ac-2fa2-49f75a5b9a84}"
testLoggerAdapter verbose: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.151Z [ERROR] server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: failed to get log: index=23 error="log not found"
    testing.go:149: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.151Z [ERROR] server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: failed to get log: index=23 error="log not found"
testLoggerAdapter verbose: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.151Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: aborting pipeline replication: peer="{Voter server-d904b220-5b5c-c279-16b3-99f3c5035f9a d904b220-5b5c-c279-16b3-99f3c5035f9a}"
    testing.go:149: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.151Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: aborting pipeline replication: peer="{Voter server-d904b220-5b5c-c279-16b3-99f3c5035f9a d904b220-5b5c-c279-16b3-99f3c5035f9a}"
testLoggerAdapter verbose: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.157Z [ERROR] server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: failed to get log: index=23 error="log not found"
    testing.go:149: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.157Z [ERROR] server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: failed to get log: index=23 error="log not found"
    testing.go:153: 2024-01-08T00:31:06.158Z [INFO]  creating new snapshot: path=/tmp/raft3009075828/snapshots/2-1013-1704673866158.tmp
testLoggerAdapter verbose: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.161Z [ERROR] server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: failed to get log: index=23 error="log not found"
    testing.go:149: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.161Z [ERROR] server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: failed to get log: index=23 error="log not found"
    testing.go:153: 2024-01-08T00:31:06.161Z [INFO]  creating new snapshot: path=/tmp/raft2767649673/snapshots/2-1013-1704673866161.tmp
testLoggerAdapter verbose: server-d904b220-5b5c-c279-16b3-99f3c5035f9a: 2024-01-08T00:31:06.167Z [INFO]  server-d904b220-5b5c-c279-16b3-99f3c5035f9a: copied to local snapshot: bytes=71
    testing.go:149: server-d904b220-5b5c-c279-16b3-99f3c5035f9a: 2024-01-08T00:31:06.167Z [INFO]  server-d904b220-5b5c-c279-16b3-99f3c5035f9a: copied to local snapshot: bytes=71
testLoggerAdapter verbose: server-d904b220-5b5c-c279-16b3-99f3c5035f9a: 2024-01-08T00:31:06.168Z [INFO]  server-d904b220-5b5c-c279-16b3-99f3c5035f9a: Installed remote snapshot
    testing.go:149: server-d904b220-5b5c-c279-16b3-99f3c5035f9a: 2024-01-08T00:31:06.168Z [INFO]  server-d904b220-5b5c-c279-16b3-99f3c5035f9a: Installed remote snapshot
testLoggerAdapter verbose: server-d904b220-5b5c-c279-16b3-99f3c5035f9a: 2024-01-08T00:31:06.168Z [WARN]  server-d904b220-5b5c-c279-16b3-99f3c5035f9a: skipping application of old log: index=22
    testing.go:149: server-d904b220-5b5c-c279-16b3-99f3c5035f9a: 2024-01-08T00:31:06.168Z [WARN]  server-d904b220-5b5c-c279-16b3-99f3c5035f9a: skipping application of old log: index=22
testLoggerAdapter verbose: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.168Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: pipelining replication: peer="{Voter server-d904b220-5b5c-c279-16b3-99f3c5035f9a d904b220-5b5c-c279-16b3-99f3c5035f9a}"
    testing.go:149: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.168Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: pipelining replication: peer="{Voter server-d904b220-5b5c-c279-16b3-99f3c5035f9a d904b220-5b5c-c279-16b3-99f3c5035f9a}"
testLoggerAdapter verbose: server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84: 2024-01-08T00:31:06.170Z [INFO]  server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84: copied to local snapshot: bytes=71
    testing.go:149: server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84: 2024-01-08T00:31:06.170Z [INFO]  server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84: copied to local snapshot: bytes=71
testLoggerAdapter verbose: server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84: 2024-01-08T00:31:06.170Z [INFO]  server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84: Installed remote snapshot
    testing.go:149: server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84: 2024-01-08T00:31:06.170Z [INFO]  server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84: Installed remote snapshot
testLoggerAdapter verbose: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.171Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: pipelining replication: peer="{Voter server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84 3515b721-0e5e-91ac-2fa2-49f75a5b9a84}"
    testing.go:149: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.171Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: pipelining replication: peer="{Voter server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84 3515b721-0e5e-91ac-2fa2-49f75a5b9a84}"
testLoggerAdapter verbose: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.185Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: aborting pipeline replication: peer="{Voter server-d904b220-5b5c-c279-16b3-99f3c5035f9a d904b220-5b5c-c279-16b3-99f3c5035f9a}"
    testing.go:149: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.185Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: aborting pipeline replication: peer="{Voter server-d904b220-5b5c-c279-16b3-99f3c5035f9a d904b220-5b5c-c279-16b3-99f3c5035f9a}"
testLoggerAdapter verbose: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.185Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: aborting pipeline replication: peer="{Voter server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84 3515b721-0e5e-91ac-2fa2-49f75a5b9a84}"
    testing.go:149: server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: 2024-01-08T00:31:06.185Z [INFO]  server-240e00b6-e276-bafc-8be2-7e927c3b9b9c: aborting pipeline replication: peer="{Voter server-3515b721-0e5e-91ac-2fa2-49f75a5b9a84 3515b721-0e5e-91ac-2fa2-49f75a5b9a84}"
testLoggerAdapter verbose: server-ea94504c-b9dd-9d53-9131-959998be4670: 2024-01-08T00:31:06.187Z [INFO]  server-ea94504c-b9dd-9d53-9131-959998be4670: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ea94504c-b9dd-9d53-9131-959998be4670 Address:ea94504c-b9dd-9d53-9131-959998be4670} {Suffrage:Voter ID:server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0 Address:7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0} {Suffrage:Voter ID:server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0 Address:de0fcf05-17c2-a10a-c09c-a660e2b9f5f0}]"
    testing.go:149: server-ea94504c-b9dd-9d53-9131-959998be4670: 2024-01-08T00:31:06.187Z [INFO]  server-ea94504c-b9dd-9d53-9131-959998be4670: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ea94504c-b9dd-9d53-9131-959998be4670 Address:ea94504c-b9dd-9d53-9131-959998be4670} {Suffrage:Voter ID:server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0 Address:7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0} {Suffrage:Voter ID:server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0 Address:de0fcf05-17c2-a10a-c09c-a660e2b9f5f0}]"
testLoggerAdapter verbose: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.187Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ea94504c-b9dd-9d53-9131-959998be4670 Address:ea94504c-b9dd-9d53-9131-959998be4670} {Suffrage:Voter ID:server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0 Address:7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0} {Suffrage:Voter ID:server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0 Address:de0fcf05-17c2-a10a-c09c-a660e2b9f5f0}]"
    testing.go:149: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.187Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ea94504c-b9dd-9d53-9131-959998be4670 Address:ea94504c-b9dd-9d53-9131-959998be4670} {Suffrage:Voter ID:server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0 Address:7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0} {Suffrage:Voter ID:server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0 Address:de0fcf05-17c2-a10a-c09c-a660e2b9f5f0}]"
testLoggerAdapter verbose: server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0: 2024-01-08T00:31:06.187Z [INFO]  server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ea94504c-b9dd-9d53-9131-959998be4670 Address:ea94504c-b9dd-9d53-9131-959998be4670} {Suffrage:Voter ID:server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0 Address:7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0} {Suffrage:Voter ID:server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0 Address:de0fcf05-17c2-a10a-c09c-a660e2b9f5f0}]"
    testing.go:149: server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0: 2024-01-08T00:31:06.187Z [INFO]  server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-ea94504c-b9dd-9d53-9131-959998be4670 Address:ea94504c-b9dd-9d53-9131-959998be4670} {Suffrage:Voter ID:server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0 Address:7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0} {Suffrage:Voter ID:server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0 Address:de0fcf05-17c2-a10a-c09c-a660e2b9f5f0}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:06.187Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:06.187Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-ea94504c-b9dd-9d53-9131-959998be4670: 2024-01-08T00:31:06.187Z [INFO]  server-ea94504c-b9dd-9d53-9131-959998be4670: entering follower state: follower="Node at ea94504c-b9dd-9d53-9131-959998be4670 [Follower]" leader=
    testing.go:149: server-ea94504c-b9dd-9d53-9131-959998be4670: 2024-01-08T00:31:06.187Z [INFO]  server-ea94504c-b9dd-9d53-9131-959998be4670: entering follower state: follower="Node at ea94504c-b9dd-9d53-9131-959998be4670 [Follower]" leader=
testLoggerAdapter verbose: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.187Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: entering follower state: follower="Node at 7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0 [Follower]" leader=
    testing.go:149: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.187Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: entering follower state: follower="Node at 7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0 [Follower]" leader=
testLoggerAdapter verbose: server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0: 2024-01-08T00:31:06.187Z [INFO]  server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0: entering follower state: follower="Node at de0fcf05-17c2-a10a-c09c-a660e2b9f5f0 [Follower]" leader=
    testing.go:149: server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0: 2024-01-08T00:31:06.187Z [INFO]  server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0: entering follower state: follower="Node at de0fcf05-17c2-a10a-c09c-a660e2b9f5f0 [Follower]" leader=
testLoggerAdapter verbose: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.244Z [WARN]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.244Z [WARN]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.244Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: entering candidate state: node="Node at 7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0 [Candidate]" term=2
    testing.go:149: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.244Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: entering candidate state: node="Node at 7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0 [Candidate]" term=2
testLoggerAdapter verbose: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.244Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: election won: tally=2
    testing.go:149: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.244Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: election won: tally=2
testLoggerAdapter verbose: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.245Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: entering leader state: leader="Node at 7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0 [Leader]"
    testing.go:149: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.245Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: entering leader state: leader="Node at 7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0 [Leader]"
testLoggerAdapter verbose: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.245Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: added peer, starting replication: peer=server-ea94504c-b9dd-9d53-9131-959998be4670
    testing.go:149: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.245Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: added peer, starting replication: peer=server-ea94504c-b9dd-9d53-9131-959998be4670
testLoggerAdapter verbose: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.245Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: added peer, starting replication: peer=server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0
    testing.go:149: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.245Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: added peer, starting replication: peer=server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0
testLoggerAdapter verbose: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.245Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: pipelining replication: peer="{Voter server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0 de0fcf05-17c2-a10a-c09c-a660e2b9f5f0}"
    testing.go:149: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.245Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: pipelining replication: peer="{Voter server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0 de0fcf05-17c2-a10a-c09c-a660e2b9f5f0}"
testLoggerAdapter verbose: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.245Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: pipelining replication: peer="{Voter server-ea94504c-b9dd-9d53-9131-959998be4670 ea94504c-b9dd-9d53-9131-959998be4670}"
    testing.go:149: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.245Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: pipelining replication: peer="{Voter server-ea94504c-b9dd-9d53-9131-959998be4670 ea94504c-b9dd-9d53-9131-959998be4670}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:06.351Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:06.24541993 +0000 UTC m=+12.292308973 (1 nodes), 57.713897ms from start of poll, 58.136438ms from cluster start. Timeout at 2024-01-08 00:31:06.351375156 +0000 UTC m=+12.398264218, 105.955245ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:06.351Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:06.24541993 +0000 UTC m=+12.292308973 (1 nodes), 57.713897ms from start of poll, 58.136438ms from cluster start. Timeout at 2024-01-08 00:31:06.351375156 +0000 UTC m=+12.398264218, 105.955245ms after stability
testLoggerAdapter verbose: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.351Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: starting snapshot up to: index=12
    testing.go:149: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.351Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: starting snapshot up to: index=12
    testing.go:153: 2024-01-08T00:31:06.352Z [INFO]  creating new snapshot: path=/tmp/raft415338014/snapshots/2-12-1704673866352.tmp
testLoggerAdapter verbose: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.359Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: snapshot complete up to: index=12
    testing.go:149: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.359Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: snapshot complete up to: index=12
    testing.go:153: 2024-01-08T00:31:06.360Z [INFO]  creating new snapshot: path=/tmp/raft415338014/snapshots/2-10013-1704673866360.tmp
testLoggerAdapter verbose: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.368Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: copied to local snapshot: bytes=71
    testing.go:149: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.368Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: copied to local snapshot: bytes=71
testLoggerAdapter verbose: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.368Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: restored user snapshot: index=1
    testing.go:149: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.368Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: restored user snapshot: index=1
testLoggerAdapter verbose: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.369Z [ERROR] server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: failed to get log: index=23 error="log not found"
    testing.go:149: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.369Z [ERROR] server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: failed to get log: index=23 error="log not found"
testLoggerAdapter verbose: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.369Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: aborting pipeline replication: peer="{Voter server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0 de0fcf05-17c2-a10a-c09c-a660e2b9f5f0}"
    testing.go:149: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.369Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: aborting pipeline replication: peer="{Voter server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0 de0fcf05-17c2-a10a-c09c-a660e2b9f5f0}"
testLoggerAdapter verbose: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.369Z [ERROR] server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: failed to get log: index=23 error="log not found"
    testing.go:149: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.369Z [ERROR] server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: failed to get log: index=23 error="log not found"
testLoggerAdapter verbose: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.369Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: aborting pipeline replication: peer="{Voter server-ea94504c-b9dd-9d53-9131-959998be4670 ea94504c-b9dd-9d53-9131-959998be4670}"
    testing.go:149: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.369Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: aborting pipeline replication: peer="{Voter server-ea94504c-b9dd-9d53-9131-959998be4670 ea94504c-b9dd-9d53-9131-959998be4670}"
testLoggerAdapter verbose: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.376Z [ERROR] server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: failed to get log: index=23 error="log not found"
    testing.go:149: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.376Z [ERROR] server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: failed to get log: index=23 error="log not found"
testLoggerAdapter verbose: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.376Z [ERROR] server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: failed to get log: index=23 error="log not found"
    testing.go:149: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.376Z [ERROR] server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: failed to get log: index=23 error="log not found"
    testing.go:153: 2024-01-08T00:31:06.376Z [INFO]  creating new snapshot: path=/tmp/raft2915594357/snapshots/2-10013-1704673866376.tmp
    testing.go:153: 2024-01-08T00:31:06.377Z [INFO]  creating new snapshot: path=/tmp/raft1635674279/snapshots/2-10013-1704673866377.tmp
testLoggerAdapter verbose: server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0: 2024-01-08T00:31:06.388Z [INFO]  server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0: copied to local snapshot: bytes=71
    testing.go:149: server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0: 2024-01-08T00:31:06.388Z [INFO]  server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0: copied to local snapshot: bytes=71
testLoggerAdapter verbose: server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0: 2024-01-08T00:31:06.388Z [INFO]  server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0: Installed remote snapshot
    testing.go:149: server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0: 2024-01-08T00:31:06.388Z [INFO]  server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0: Installed remote snapshot
testLoggerAdapter verbose: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.388Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: pipelining replication: peer="{Voter server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0 de0fcf05-17c2-a10a-c09c-a660e2b9f5f0}"
    testing.go:149: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.388Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: pipelining replication: peer="{Voter server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0 de0fcf05-17c2-a10a-c09c-a660e2b9f5f0}"
testLoggerAdapter verbose: server-ea94504c-b9dd-9d53-9131-959998be4670: 2024-01-08T00:31:06.388Z [INFO]  server-ea94504c-b9dd-9d53-9131-959998be4670: copied to local snapshot: bytes=71
    testing.go:149: server-ea94504c-b9dd-9d53-9131-959998be4670: 2024-01-08T00:31:06.388Z [INFO]  server-ea94504c-b9dd-9d53-9131-959998be4670: copied to local snapshot: bytes=71
testLoggerAdapter verbose: server-ea94504c-b9dd-9d53-9131-959998be4670: 2024-01-08T00:31:06.388Z [INFO]  server-ea94504c-b9dd-9d53-9131-959998be4670: Installed remote snapshot
    testing.go:149: server-ea94504c-b9dd-9d53-9131-959998be4670: 2024-01-08T00:31:06.388Z [INFO]  server-ea94504c-b9dd-9d53-9131-959998be4670: Installed remote snapshot
testLoggerAdapter verbose: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.389Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: pipelining replication: peer="{Voter server-ea94504c-b9dd-9d53-9131-959998be4670 ea94504c-b9dd-9d53-9131-959998be4670}"
    testing.go:149: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.389Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: pipelining replication: peer="{Voter server-ea94504c-b9dd-9d53-9131-959998be4670 ea94504c-b9dd-9d53-9131-959998be4670}"
testLoggerAdapter verbose: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.405Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: aborting pipeline replication: peer="{Voter server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0 de0fcf05-17c2-a10a-c09c-a660e2b9f5f0}"
    testing.go:149: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.405Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: aborting pipeline replication: peer="{Voter server-de0fcf05-17c2-a10a-c09c-a660e2b9f5f0 de0fcf05-17c2-a10a-c09c-a660e2b9f5f0}"
testLoggerAdapter verbose: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.405Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: aborting pipeline replication: peer="{Voter server-ea94504c-b9dd-9d53-9131-959998be4670 ea94504c-b9dd-9d53-9131-959998be4670}"
    testing.go:149: server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: 2024-01-08T00:31:06.405Z [INFO]  server-7a391fc5-a4ec-4b75-abc3-0871ff3f0ec0: aborting pipeline replication: peer="{Voter server-ea94504c-b9dd-9d53-9131-959998be4670 ea94504c-b9dd-9d53-9131-959998be4670}"
--- PASS: TestRaft_UserRestore (1.39s)
=== RUN   TestRaft_SendSnapshotFollower
testLoggerAdapter verbose: server-e85a8395-e223-4c71-1a24-3b8df5b08a46: 2024-01-08T00:31:06.407Z [INFO]  server-e85a8395-e223-4c71-1a24-3b8df5b08a46: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e85a8395-e223-4c71-1a24-3b8df5b08a46 Address:e85a8395-e223-4c71-1a24-3b8df5b08a46} {Suffrage:Voter ID:server-cd07f062-9c4f-8158-b15c-e5b208152459 Address:cd07f062-9c4f-8158-b15c-e5b208152459} {Suffrage:Voter ID:server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89 Address:1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89}]"
    testing.go:149: server-e85a8395-e223-4c71-1a24-3b8df5b08a46: 2024-01-08T00:31:06.407Z [INFO]  server-e85a8395-e223-4c71-1a24-3b8df5b08a46: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e85a8395-e223-4c71-1a24-3b8df5b08a46 Address:e85a8395-e223-4c71-1a24-3b8df5b08a46} {Suffrage:Voter ID:server-cd07f062-9c4f-8158-b15c-e5b208152459 Address:cd07f062-9c4f-8158-b15c-e5b208152459} {Suffrage:Voter ID:server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89 Address:1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89}]"
testLoggerAdapter verbose: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.407Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e85a8395-e223-4c71-1a24-3b8df5b08a46 Address:e85a8395-e223-4c71-1a24-3b8df5b08a46} {Suffrage:Voter ID:server-cd07f062-9c4f-8158-b15c-e5b208152459 Address:cd07f062-9c4f-8158-b15c-e5b208152459} {Suffrage:Voter ID:server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89 Address:1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89}]"
    testing.go:149: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.407Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e85a8395-e223-4c71-1a24-3b8df5b08a46 Address:e85a8395-e223-4c71-1a24-3b8df5b08a46} {Suffrage:Voter ID:server-cd07f062-9c4f-8158-b15c-e5b208152459 Address:cd07f062-9c4f-8158-b15c-e5b208152459} {Suffrage:Voter ID:server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89 Address:1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89}]"
testLoggerAdapter verbose: server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89: 2024-01-08T00:31:06.407Z [INFO]  server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e85a8395-e223-4c71-1a24-3b8df5b08a46 Address:e85a8395-e223-4c71-1a24-3b8df5b08a46} {Suffrage:Voter ID:server-cd07f062-9c4f-8158-b15c-e5b208152459 Address:cd07f062-9c4f-8158-b15c-e5b208152459} {Suffrage:Voter ID:server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89 Address:1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89}]"
    testing.go:149: server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89: 2024-01-08T00:31:06.407Z [INFO]  server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e85a8395-e223-4c71-1a24-3b8df5b08a46 Address:e85a8395-e223-4c71-1a24-3b8df5b08a46} {Suffrage:Voter ID:server-cd07f062-9c4f-8158-b15c-e5b208152459 Address:cd07f062-9c4f-8158-b15c-e5b208152459} {Suffrage:Voter ID:server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89 Address:1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:06.407Z [INFO]  starting stability test: raft-state=Follower
    testing.go:149: cluster: 2024-01-08T00:31:06.407Z [INFO]  starting stability test: raft-state=Follower
testLoggerAdapter verbose: server-e85a8395-e223-4c71-1a24-3b8df5b08a46: 2024-01-08T00:31:06.407Z [INFO]  server-e85a8395-e223-4c71-1a24-3b8df5b08a46: entering follower state: follower="Node at e85a8395-e223-4c71-1a24-3b8df5b08a46 [Follower]" leader=
    testing.go:149: server-e85a8395-e223-4c71-1a24-3b8df5b08a46: 2024-01-08T00:31:06.407Z [INFO]  server-e85a8395-e223-4c71-1a24-3b8df5b08a46: entering follower state: follower="Node at e85a8395-e223-4c71-1a24-3b8df5b08a46 [Follower]" leader=
testLoggerAdapter verbose: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.407Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: entering follower state: follower="Node at cd07f062-9c4f-8158-b15c-e5b208152459 [Follower]" leader=
    testing.go:149: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.407Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: entering follower state: follower="Node at cd07f062-9c4f-8158-b15c-e5b208152459 [Follower]" leader=
testLoggerAdapter verbose: server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89: 2024-01-08T00:31:06.407Z [INFO]  server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89: entering follower state: follower="Node at 1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89 [Follower]" leader=
    testing.go:149: server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89: 2024-01-08T00:31:06.407Z [INFO]  server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89: entering follower state: follower="Node at 1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89 [Follower]" leader=
testLoggerAdapter verbose: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.470Z [WARN]  server-cd07f062-9c4f-8158-b15c-e5b208152459: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.470Z [WARN]  server-cd07f062-9c4f-8158-b15c-e5b208152459: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.470Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: entering candidate state: node="Node at cd07f062-9c4f-8158-b15c-e5b208152459 [Candidate]" term=2
    testing.go:149: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.470Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: entering candidate state: node="Node at cd07f062-9c4f-8158-b15c-e5b208152459 [Candidate]" term=2
testLoggerAdapter verbose: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.470Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: election won: tally=2
    testing.go:149: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.470Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: election won: tally=2
testLoggerAdapter verbose: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.470Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: entering leader state: leader="Node at cd07f062-9c4f-8158-b15c-e5b208152459 [Leader]"
    testing.go:149: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.470Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: entering leader state: leader="Node at cd07f062-9c4f-8158-b15c-e5b208152459 [Leader]"
testLoggerAdapter verbose: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.470Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: added peer, starting replication: peer=server-e85a8395-e223-4c71-1a24-3b8df5b08a46
    testing.go:149: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.470Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: added peer, starting replication: peer=server-e85a8395-e223-4c71-1a24-3b8df5b08a46
testLoggerAdapter verbose: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.470Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: added peer, starting replication: peer=server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89
    testing.go:149: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.470Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: added peer, starting replication: peer=server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89
testLoggerAdapter verbose: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.470Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: pipelining replication: peer="{Voter server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89 1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89}"
    testing.go:149: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.470Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: pipelining replication: peer="{Voter server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89 1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89}"
testLoggerAdapter verbose: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.471Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: pipelining replication: peer="{Voter server-e85a8395-e223-4c71-1a24-3b8df5b08a46 e85a8395-e223-4c71-1a24-3b8df5b08a46}"
    testing.go:149: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.471Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: pipelining replication: peer="{Voter server-e85a8395-e223-4c71-1a24-3b8df5b08a46 e85a8395-e223-4c71-1a24-3b8df5b08a46}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:06.576Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:06.471107604 +0000 UTC m=+12.517996669 (2 nodes), 63.365948ms from start of poll, 63.80465ms from cluster start. Timeout at 2024-01-08 00:31:06.576409529 +0000 UTC m=+12.623298585, 105.301916ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:06.576Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:06.471107604 +0000 UTC m=+12.517996669 (2 nodes), 63.365948ms from start of poll, 63.80465ms from cluster start. Timeout at 2024-01-08 00:31:06.576409529 +0000 UTC m=+12.623298585, 105.301916ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:06.576Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:06.576Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:06.681Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:06.576607318 +0000 UTC m=+12.623496368 (1 nodes), 2.053µs from start of poll, 169.304349ms from cluster start. Timeout at 2024-01-08 00:31:06.681932607 +0000 UTC m=+12.728821674, 105.325306ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:06.681Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:06.576607318 +0000 UTC m=+12.623496368 (1 nodes), 2.053µs from start of poll, 169.304349ms from cluster start. Timeout at 2024-01-08 00:31:06.681932607 +0000 UTC m=+12.728821674, 105.325306ms after stability
testLoggerAdapter verbose: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.682Z [ERROR] server-cd07f062-9c4f-8158-b15c-e5b208152459: failed to pipeline appendEntries: peer="{Voter server-e85a8395-e223-4c71-1a24-3b8df5b08a46 e85a8395-e223-4c71-1a24-3b8df5b08a46}" error="append pipeline closed"
    testing.go:149: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.682Z [ERROR] server-cd07f062-9c4f-8158-b15c-e5b208152459: failed to pipeline appendEntries: peer="{Voter server-e85a8395-e223-4c71-1a24-3b8df5b08a46 e85a8395-e223-4c71-1a24-3b8df5b08a46}" error="append pipeline closed"
testLoggerAdapter verbose: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.682Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: aborting pipeline replication: peer="{Voter server-e85a8395-e223-4c71-1a24-3b8df5b08a46 e85a8395-e223-4c71-1a24-3b8df5b08a46}"
    testing.go:149: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.682Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: aborting pipeline replication: peer="{Voter server-e85a8395-e223-4c71-1a24-3b8df5b08a46 e85a8395-e223-4c71-1a24-3b8df5b08a46}"
testLoggerAdapter verbose: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.682Z [ERROR] server-cd07f062-9c4f-8158-b15c-e5b208152459: failed to appendEntries to: peer="{Voter server-e85a8395-e223-4c71-1a24-3b8df5b08a46 e85a8395-e223-4c71-1a24-3b8df5b08a46}" error="failed to connect to peer: e85a8395-e223-4c71-1a24-3b8df5b08a46"
    testing.go:149: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.682Z [ERROR] server-cd07f062-9c4f-8158-b15c-e5b208152459: failed to appendEntries to: peer="{Voter server-e85a8395-e223-4c71-1a24-3b8df5b08a46 e85a8395-e223-4c71-1a24-3b8df5b08a46}" error="failed to connect to peer: e85a8395-e223-4c71-1a24-3b8df5b08a46"
    raft_test.go:1320: [INFO] Finished apply without behind follower
testLoggerAdapter verbose: server-e85a8395-e223-4c71-1a24-3b8df5b08a46: 2024-01-08T00:31:06.684Z [ERROR] server-e85a8395-e223-4c71-1a24-3b8df5b08a46: failed to take snapshot: error="nothing new to snapshot"
    testing.go:149: server-e85a8395-e223-4c71-1a24-3b8df5b08a46: 2024-01-08T00:31:06.684Z [ERROR] server-e85a8395-e223-4c71-1a24-3b8df5b08a46: failed to take snapshot: error="nothing new to snapshot"
testLoggerAdapter verbose: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.684Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: starting snapshot up to: index=102
    testing.go:149: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.684Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: starting snapshot up to: index=102
    testing.go:153: 2024-01-08T00:31:06.684Z [INFO]  creating new snapshot: path=/tmp/raft2173245344/snapshots/2-102-1704673866684.tmp
testLoggerAdapter verbose: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.691Z [ERROR] server-cd07f062-9c4f-8158-b15c-e5b208152459: failed to heartbeat to: peer=e85a8395-e223-4c71-1a24-3b8df5b08a46 error="failed to connect to peer: e85a8395-e223-4c71-1a24-3b8df5b08a46"
    testing.go:149: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.691Z [ERROR] server-cd07f062-9c4f-8158-b15c-e5b208152459: failed to heartbeat to: peer=e85a8395-e223-4c71-1a24-3b8df5b08a46 error="failed to connect to peer: e85a8395-e223-4c71-1a24-3b8df5b08a46"
testLoggerAdapter verbose: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.692Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: compacting logs: from=1 to=92
    testing.go:149: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.692Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: compacting logs: from=1 to=92
testLoggerAdapter verbose: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.692Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: snapshot complete up to: index=102
    testing.go:149: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.692Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: snapshot complete up to: index=102
testLoggerAdapter verbose: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.692Z [ERROR] server-cd07f062-9c4f-8158-b15c-e5b208152459: failed to get log: index=2 error="log not found"
    testing.go:149: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.692Z [ERROR] server-cd07f062-9c4f-8158-b15c-e5b208152459: failed to get log: index=2 error="log not found"
testLoggerAdapter verbose: server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89: 2024-01-08T00:31:06.692Z [INFO]  server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89: starting snapshot up to: index=97
    testing.go:149: server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89: 2024-01-08T00:31:06.692Z [INFO]  server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89: starting snapshot up to: index=97
    testing.go:153: 2024-01-08T00:31:06.692Z [INFO]  creating new snapshot: path=/tmp/raft4234533615/snapshots/2-97-1704673866692.tmp
testLoggerAdapter verbose: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.693Z [ERROR] server-cd07f062-9c4f-8158-b15c-e5b208152459: failed to install snapshot: id=2-102-1704673866684 error="failed to connect to peer: e85a8395-e223-4c71-1a24-3b8df5b08a46"
    testing.go:149: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.693Z [ERROR] server-cd07f062-9c4f-8158-b15c-e5b208152459: failed to install snapshot: id=2-102-1704673866684 error="failed to connect to peer: e85a8395-e223-4c71-1a24-3b8df5b08a46"
testLoggerAdapter verbose: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.693Z [ERROR] server-cd07f062-9c4f-8158-b15c-e5b208152459: failed to send snapshot to: peer="{Voter server-e85a8395-e223-4c71-1a24-3b8df5b08a46 e85a8395-e223-4c71-1a24-3b8df5b08a46}" error="failed to connect to peer: e85a8395-e223-4c71-1a24-3b8df5b08a46"
    testing.go:149: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.693Z [ERROR] server-cd07f062-9c4f-8158-b15c-e5b208152459: failed to send snapshot to: peer="{Voter server-e85a8395-e223-4c71-1a24-3b8df5b08a46 e85a8395-e223-4c71-1a24-3b8df5b08a46}" error="failed to connect to peer: e85a8395-e223-4c71-1a24-3b8df5b08a46"
testLoggerAdapter verbose: server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89: 2024-01-08T00:31:06.701Z [INFO]  server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89: compacting logs: from=1 to=92
    testing.go:149: server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89: 2024-01-08T00:31:06.701Z [INFO]  server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89: compacting logs: from=1 to=92
testLoggerAdapter verbose: server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89: 2024-01-08T00:31:06.701Z [INFO]  server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89: snapshot complete up to: index=97
    testing.go:149: server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89: 2024-01-08T00:31:06.701Z [INFO]  server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89: snapshot complete up to: index=97
testLoggerAdapter verbose: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.703Z [ERROR] server-cd07f062-9c4f-8158-b15c-e5b208152459: failed to get log: index=2 error="log not found"
    testing.go:149: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.703Z [ERROR] server-cd07f062-9c4f-8158-b15c-e5b208152459: failed to get log: index=2 error="log not found"
    testing.go:153: 2024-01-08T00:31:06.703Z [INFO]  creating new snapshot: path=/tmp/raft986211076/snapshots/2-102-1704673866703.tmp
testLoggerAdapter verbose: server-e85a8395-e223-4c71-1a24-3b8df5b08a46: 2024-01-08T00:31:06.710Z [INFO]  server-e85a8395-e223-4c71-1a24-3b8df5b08a46: copied to local snapshot: bytes=693
    testing.go:149: server-e85a8395-e223-4c71-1a24-3b8df5b08a46: 2024-01-08T00:31:06.710Z [INFO]  server-e85a8395-e223-4c71-1a24-3b8df5b08a46: copied to local snapshot: bytes=693
testLoggerAdapter verbose: server-e85a8395-e223-4c71-1a24-3b8df5b08a46: 2024-01-08T00:31:06.711Z [INFO]  server-e85a8395-e223-4c71-1a24-3b8df5b08a46: Installed remote snapshot
    testing.go:149: server-e85a8395-e223-4c71-1a24-3b8df5b08a46: 2024-01-08T00:31:06.711Z [INFO]  server-e85a8395-e223-4c71-1a24-3b8df5b08a46: Installed remote snapshot
testLoggerAdapter verbose: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.712Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: aborting pipeline replication: peer="{Voter server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89 1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89}"
    testing.go:149: server-cd07f062-9c4f-8158-b15c-e5b208152459: 2024-01-08T00:31:06.712Z [INFO]  server-cd07f062-9c4f-8158-b15c-e5b208152459: aborting pipeline replication: peer="{Voter server-1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89 1ebeebc9-fef4-e5bb-6ef5-64c33fe46a89}"
--- PASS: TestRaft_SendSnapshotFollower (0.31s)
=== RUN   TestRaft_SendSnapshotAndLogsFollower
testLoggerAdapter verbose: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:06.713Z [INFO]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e Address:d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e} {Suffrage:Voter ID:server-bdbd9fe5-b7e0-3828-169f-9446390443e2 Address:bdbd9fe5-b7e0-3828-169f-9446390443e2} {Suffrage:Voter ID:server-2b405c0f-7a8f-8c3d-909a-721aefefba5f Address:2b405c0f-7a8f-8c3d-909a-721aefefba5f}]"
    testing.go:149: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:06.713Z [INFO]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e Address:d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e} {Suffrage:Voter ID:server-bdbd9fe5-b7e0-3828-169f-9446390443e2 Address:bdbd9fe5-b7e0-3828-169f-9446390443e2} {Suffrage:Voter ID:server-2b405c0f-7a8f-8c3d-909a-721aefefba5f Address:2b405c0f-7a8f-8c3d-909a-721aefefba5f}]"
testLoggerAdapter verbose: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:06.713Z [INFO]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: entering follower state: follower="Node at d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e [Follower]" leader=
    testing.go:149: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:06.713Z [INFO]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: entering follower state: follower="Node at d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e [Follower]" leader=
testLoggerAdapter verbose: server-bdbd9fe5-b7e0-3828-169f-9446390443e2: 2024-01-08T00:31:06.713Z [INFO]  server-bdbd9fe5-b7e0-3828-169f-9446390443e2: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e Address:d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e} {Suffrage:Voter ID:server-bdbd9fe5-b7e0-3828-169f-9446390443e2 Address:bdbd9fe5-b7e0-3828-169f-9446390443e2} {Suffrage:Voter ID:server-2b405c0f-7a8f-8c3d-909a-721aefefba5f Address:2b405c0f-7a8f-8c3d-909a-721aefefba5f}]"
    testing.go:149: server-bdbd9fe5-b7e0-3828-169f-9446390443e2: 2024-01-08T00:31:06.713Z [INFO]  server-bdbd9fe5-b7e0-3828-169f-9446390443e2: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e Address:d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e} {Suffrage:Voter ID:server-bdbd9fe5-b7e0-3828-169f-9446390443e2 Address:bdbd9fe5-b7e0-3828-169f-9446390443e2} {Suffrage:Voter ID:server-2b405c0f-7a8f-8c3d-909a-721aefefba5f Address:2b405c0f-7a8f-8c3d-909a-721aefefba5f}]"
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.714Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e Address:d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e} {Suffrage:Voter ID:server-bdbd9fe5-b7e0-3828-169f-9446390443e2 Address:bdbd9fe5-b7e0-3828-169f-9446390443e2} {Suffrage:Voter ID:server-2b405c0f-7a8f-8c3d-909a-721aefefba5f Address:2b405c0f-7a8f-8c3d-909a-721aefefba5f}]"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.714Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e Address:d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e} {Suffrage:Voter ID:server-bdbd9fe5-b7e0-3828-169f-9446390443e2 Address:bdbd9fe5-b7e0-3828-169f-9446390443e2} {Suffrage:Voter ID:server-2b405c0f-7a8f-8c3d-909a-721aefefba5f Address:2b405c0f-7a8f-8c3d-909a-721aefefba5f}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:06.714Z [INFO]  starting stability test: raft-state=Follower
    testing.go:149: cluster: 2024-01-08T00:31:06.714Z [INFO]  starting stability test: raft-state=Follower
testLoggerAdapter verbose: server-bdbd9fe5-b7e0-3828-169f-9446390443e2: 2024-01-08T00:31:06.715Z [INFO]  server-bdbd9fe5-b7e0-3828-169f-9446390443e2: entering follower state: follower="Node at bdbd9fe5-b7e0-3828-169f-9446390443e2 [Follower]" leader=
    testing.go:149: server-bdbd9fe5-b7e0-3828-169f-9446390443e2: 2024-01-08T00:31:06.715Z [INFO]  server-bdbd9fe5-b7e0-3828-169f-9446390443e2: entering follower state: follower="Node at bdbd9fe5-b7e0-3828-169f-9446390443e2 [Follower]" leader=
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.715Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: entering follower state: follower="Node at 2b405c0f-7a8f-8c3d-909a-721aefefba5f [Follower]" leader=
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.715Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: entering follower state: follower="Node at 2b405c0f-7a8f-8c3d-909a-721aefefba5f [Follower]" leader=
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.777Z [WARN]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.777Z [WARN]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.777Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: entering candidate state: node="Node at 2b405c0f-7a8f-8c3d-909a-721aefefba5f [Candidate]" term=2
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.777Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: entering candidate state: node="Node at 2b405c0f-7a8f-8c3d-909a-721aefefba5f [Candidate]" term=2
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.777Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: election won: tally=2
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.777Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: election won: tally=2
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.778Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: entering leader state: leader="Node at 2b405c0f-7a8f-8c3d-909a-721aefefba5f [Leader]"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.778Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: entering leader state: leader="Node at 2b405c0f-7a8f-8c3d-909a-721aefefba5f [Leader]"
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.778Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: added peer, starting replication: peer=server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.778Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: added peer, starting replication: peer=server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.778Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: added peer, starting replication: peer=server-bdbd9fe5-b7e0-3828-169f-9446390443e2
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.778Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: added peer, starting replication: peer=server-bdbd9fe5-b7e0-3828-169f-9446390443e2
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.778Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: pipelining replication: peer="{Voter server-bdbd9fe5-b7e0-3828-169f-9446390443e2 bdbd9fe5-b7e0-3828-169f-9446390443e2}"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.778Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: pipelining replication: peer="{Voter server-bdbd9fe5-b7e0-3828-169f-9446390443e2 bdbd9fe5-b7e0-3828-169f-9446390443e2}"
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.778Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: pipelining replication: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.778Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: pipelining replication: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:06.883Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:06.778412978 +0000 UTC m=+12.825302022 (2 nodes), 63.911984ms from start of poll, 64.785513ms from cluster start. Timeout at 2024-01-08 00:31:06.883471893 +0000 UTC m=+12.930360922, 105.0589ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:06.883Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:06.778412978 +0000 UTC m=+12.825302022 (2 nodes), 63.911984ms from start of poll, 64.785513ms from cluster start. Timeout at 2024-01-08 00:31:06.883471893 +0000 UTC m=+12.930360922, 105.0589ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:06.883Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:06.883Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:06.989Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:06.883609032 +0000 UTC m=+12.930498064 (1 nodes), 1.99µs from start of poll, 169.981555ms from cluster start. Timeout at 2024-01-08 00:31:06.98933743 +0000 UTC m=+13.036226498, 105.728434ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:06.989Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:06.883609032 +0000 UTC m=+12.930498064 (1 nodes), 1.99µs from start of poll, 169.981555ms from cluster start. Timeout at 2024-01-08 00:31:06.98933743 +0000 UTC m=+13.036226498, 105.728434ms after stability
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.989Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to pipeline appendEntries: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}" error="append pipeline closed"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.989Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to pipeline appendEntries: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}" error="append pipeline closed"
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.989Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: aborting pipeline replication: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.989Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: aborting pipeline replication: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}"
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.989Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to appendEntries to: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}" error="failed to connect to peer: d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.989Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to appendEntries to: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}" error="failed to connect to peer: d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e"
    raft_test.go:1362: [INFO] Finished apply without behind follower
testLoggerAdapter verbose: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:06.991Z [ERROR] server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: failed to take snapshot: error="nothing new to snapshot"
    testing.go:149: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:06.991Z [ERROR] server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: failed to take snapshot: error="nothing new to snapshot"
testLoggerAdapter verbose: server-bdbd9fe5-b7e0-3828-169f-9446390443e2: 2024-01-08T00:31:06.992Z [INFO]  server-bdbd9fe5-b7e0-3828-169f-9446390443e2: starting snapshot up to: index=90
    testing.go:149: server-bdbd9fe5-b7e0-3828-169f-9446390443e2: 2024-01-08T00:31:06.992Z [INFO]  server-bdbd9fe5-b7e0-3828-169f-9446390443e2: starting snapshot up to: index=90
    testing.go:153: 2024-01-08T00:31:06.992Z [INFO]  creating new snapshot: path=/tmp/raft1382983678/snapshots/2-90-1704673866992.tmp
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.994Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to heartbeat to: peer=d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e error="failed to connect to peer: d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:06.994Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to heartbeat to: peer=d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e error="failed to connect to peer: d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e"
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.000Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to appendEntries to: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}" error="failed to connect to peer: d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.000Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to appendEntries to: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}" error="failed to connect to peer: d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e"
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.011Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to appendEntries to: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}" error="failed to connect to peer: d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.011Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to appendEntries to: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}" error="failed to connect to peer: d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e"
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.014Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to heartbeat to: peer=d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e error="failed to connect to peer: d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.014Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to heartbeat to: peer=d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e error="failed to connect to peer: d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e"
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.033Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to heartbeat to: peer=d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e error="failed to connect to peer: d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.033Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to heartbeat to: peer=d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e error="failed to connect to peer: d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e"
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.037Z [WARN]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to contact: server-id=server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e time=50.034278ms
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.037Z [WARN]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to contact: server-id=server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e time=50.034278ms
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.038Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to appendEntries to: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}" error="failed to connect to peer: d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.038Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to appendEntries to: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}" error="failed to connect to peer: d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e"
testLoggerAdapter verbose: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.039Z [WARN]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: heartbeat timeout reached, starting election: last-leader=2b405c0f-7a8f-8c3d-909a-721aefefba5f
    testing.go:149: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.039Z [WARN]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: heartbeat timeout reached, starting election: last-leader=2b405c0f-7a8f-8c3d-909a-721aefefba5f
testLoggerAdapter verbose: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.039Z [INFO]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: entering candidate state: node="Node at d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e [Candidate]" term=3
    testing.go:149: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.039Z [INFO]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: entering candidate state: node="Node at d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e [Candidate]" term=3
testLoggerAdapter verbose: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.039Z [ERROR] server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: failed to make requestVote RPC: target="{Voter server-2b405c0f-7a8f-8c3d-909a-721aefefba5f 2b405c0f-7a8f-8c3d-909a-721aefefba5f}" error="failed to connect to peer: 2b405c0f-7a8f-8c3d-909a-721aefefba5f"
    testing.go:149: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.039Z [ERROR] server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: failed to make requestVote RPC: target="{Voter server-2b405c0f-7a8f-8c3d-909a-721aefefba5f 2b405c0f-7a8f-8c3d-909a-721aefefba5f}" error="failed to connect to peer: 2b405c0f-7a8f-8c3d-909a-721aefefba5f"
testLoggerAdapter verbose: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.039Z [ERROR] server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: failed to make requestVote RPC: target="{Voter server-bdbd9fe5-b7e0-3828-169f-9446390443e2 bdbd9fe5-b7e0-3828-169f-9446390443e2}" error="failed to connect to peer: bdbd9fe5-b7e0-3828-169f-9446390443e2"
    testing.go:149: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.039Z [ERROR] server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: failed to make requestVote RPC: target="{Voter server-bdbd9fe5-b7e0-3828-169f-9446390443e2 bdbd9fe5-b7e0-3828-169f-9446390443e2}" error="failed to connect to peer: bdbd9fe5-b7e0-3828-169f-9446390443e2"
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.065Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to heartbeat to: peer=d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e error="failed to connect to peer: d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.065Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to heartbeat to: peer=d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e error="failed to connect to peer: d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e"
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.083Z [WARN]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to contact: server-id=server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e time=96.863125ms
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.083Z [WARN]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to contact: server-id=server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e time=96.863125ms
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.085Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to appendEntries to: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}" error="failed to connect to peer: d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.085Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to appendEntries to: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}" error="failed to connect to peer: d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e"
testLoggerAdapter verbose: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.103Z [WARN]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: Election timeout reached, restarting election
    testing.go:149: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.103Z [WARN]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: Election timeout reached, restarting election
testLoggerAdapter verbose: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.103Z [INFO]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: entering candidate state: node="Node at d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e [Candidate]" term=4
    testing.go:149: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.103Z [INFO]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: entering candidate state: node="Node at d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e [Candidate]" term=4
testLoggerAdapter verbose: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.104Z [ERROR] server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: failed to make requestVote RPC: target="{Voter server-2b405c0f-7a8f-8c3d-909a-721aefefba5f 2b405c0f-7a8f-8c3d-909a-721aefefba5f}" error="failed to connect to peer: 2b405c0f-7a8f-8c3d-909a-721aefefba5f"
    testing.go:149: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.104Z [ERROR] server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: failed to make requestVote RPC: target="{Voter server-2b405c0f-7a8f-8c3d-909a-721aefefba5f 2b405c0f-7a8f-8c3d-909a-721aefefba5f}" error="failed to connect to peer: 2b405c0f-7a8f-8c3d-909a-721aefefba5f"
testLoggerAdapter verbose: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.104Z [ERROR] server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: failed to make requestVote RPC: target="{Voter server-bdbd9fe5-b7e0-3828-169f-9446390443e2 bdbd9fe5-b7e0-3828-169f-9446390443e2}" error="failed to connect to peer: bdbd9fe5-b7e0-3828-169f-9446390443e2"
    testing.go:149: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.104Z [ERROR] server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: failed to make requestVote RPC: target="{Voter server-bdbd9fe5-b7e0-3828-169f-9446390443e2 bdbd9fe5-b7e0-3828-169f-9446390443e2}" error="failed to connect to peer: bdbd9fe5-b7e0-3828-169f-9446390443e2"
testLoggerAdapter verbose: server-bdbd9fe5-b7e0-3828-169f-9446390443e2: 2024-01-08T00:31:07.104Z [INFO]  server-bdbd9fe5-b7e0-3828-169f-9446390443e2: compacting logs: from=1 to=90
    testing.go:149: server-bdbd9fe5-b7e0-3828-169f-9446390443e2: 2024-01-08T00:31:07.104Z [INFO]  server-bdbd9fe5-b7e0-3828-169f-9446390443e2: compacting logs: from=1 to=90
testLoggerAdapter verbose: server-bdbd9fe5-b7e0-3828-169f-9446390443e2: 2024-01-08T00:31:07.104Z [INFO]  server-bdbd9fe5-b7e0-3828-169f-9446390443e2: snapshot complete up to: index=90
    testing.go:149: server-bdbd9fe5-b7e0-3828-169f-9446390443e2: 2024-01-08T00:31:07.104Z [INFO]  server-bdbd9fe5-b7e0-3828-169f-9446390443e2: snapshot complete up to: index=90
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.104Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: starting snapshot up to: index=102
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.104Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: starting snapshot up to: index=102
    testing.go:153: 2024-01-08T00:31:07.104Z [INFO]  creating new snapshot: path=/tmp/raft4239871505/snapshots/2-102-1704673867104.tmp
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.113Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: compacting logs: from=1 to=92
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.113Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: compacting logs: from=1 to=92
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.113Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: snapshot complete up to: index=102
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.113Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: snapshot complete up to: index=102
    raft_test.go:1383: [INFO] Finished apply without behind follower
testLoggerAdapter verbose: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.169Z [WARN]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: Election timeout reached, restarting election
    testing.go:149: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.169Z [WARN]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: Election timeout reached, restarting election
testLoggerAdapter verbose: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.169Z [INFO]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: entering candidate state: node="Node at d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e [Candidate]" term=5
    testing.go:149: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.169Z [INFO]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: entering candidate state: node="Node at d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e [Candidate]" term=5
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.169Z [WARN]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: rejecting vote request since we have a leader: from=d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e leader=2b405c0f-7a8f-8c3d-909a-721aefefba5f
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.169Z [WARN]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: rejecting vote request since we have a leader: from=d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e leader=2b405c0f-7a8f-8c3d-909a-721aefefba5f
testLoggerAdapter verbose: server-bdbd9fe5-b7e0-3828-169f-9446390443e2: 2024-01-08T00:31:07.169Z [WARN]  server-bdbd9fe5-b7e0-3828-169f-9446390443e2: rejecting vote request since we have a leader: from=d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e leader=2b405c0f-7a8f-8c3d-909a-721aefefba5f
    testing.go:149: server-bdbd9fe5-b7e0-3828-169f-9446390443e2: 2024-01-08T00:31:07.169Z [WARN]  server-bdbd9fe5-b7e0-3828-169f-9446390443e2: rejecting vote request since we have a leader: from=d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e leader=2b405c0f-7a8f-8c3d-909a-721aefefba5f
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.170Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to get log: index=2 error="log not found"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.170Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to get log: index=2 error="log not found"
testLoggerAdapter verbose: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.171Z [INFO]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: ignoring installSnapshot request with older term than current term: request-term=2 current-term=5
    testing.go:149: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.171Z [INFO]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: ignoring installSnapshot request with older term than current term: request-term=2 current-term=5
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.171Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: peer has newer term, stopping replication: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.171Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: peer has newer term, stopping replication: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}"
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.171Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: entering follower state: follower="Node at 2b405c0f-7a8f-8c3d-909a-721aefefba5f [Follower]" leader=
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.171Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: entering follower state: follower="Node at 2b405c0f-7a8f-8c3d-909a-721aefefba5f [Follower]" leader=
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.171Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: aborting pipeline replication: peer="{Voter server-bdbd9fe5-b7e0-3828-169f-9446390443e2 bdbd9fe5-b7e0-3828-169f-9446390443e2}"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.171Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: aborting pipeline replication: peer="{Voter server-bdbd9fe5-b7e0-3828-169f-9446390443e2 bdbd9fe5-b7e0-3828-169f-9446390443e2}"
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.234Z [WARN]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.234Z [WARN]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.234Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: entering candidate state: node="Node at 2b405c0f-7a8f-8c3d-909a-721aefefba5f [Candidate]" term=3
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.234Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: entering candidate state: node="Node at 2b405c0f-7a8f-8c3d-909a-721aefefba5f [Candidate]" term=3
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.234Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: election won: tally=2
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.234Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: election won: tally=2
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.234Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: entering leader state: leader="Node at 2b405c0f-7a8f-8c3d-909a-721aefefba5f [Leader]"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.234Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: entering leader state: leader="Node at 2b405c0f-7a8f-8c3d-909a-721aefefba5f [Leader]"
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.234Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: added peer, starting replication: peer=server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.234Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: added peer, starting replication: peer=server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.235Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: added peer, starting replication: peer=server-bdbd9fe5-b7e0-3828-169f-9446390443e2
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.235Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: added peer, starting replication: peer=server-bdbd9fe5-b7e0-3828-169f-9446390443e2
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.235Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: pipelining replication: peer="{Voter server-bdbd9fe5-b7e0-3828-169f-9446390443e2 bdbd9fe5-b7e0-3828-169f-9446390443e2}"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.235Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: pipelining replication: peer="{Voter server-bdbd9fe5-b7e0-3828-169f-9446390443e2 bdbd9fe5-b7e0-3828-169f-9446390443e2}"
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.235Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: peer has newer term, stopping replication: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.235Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: peer has newer term, stopping replication: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}"
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.235Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: entering follower state: follower="Node at 2b405c0f-7a8f-8c3d-909a-721aefefba5f [Follower]" leader=
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.235Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: entering follower state: follower="Node at 2b405c0f-7a8f-8c3d-909a-721aefefba5f [Follower]" leader=
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.235Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: aborting pipeline replication: peer="{Voter server-bdbd9fe5-b7e0-3828-169f-9446390443e2 bdbd9fe5-b7e0-3828-169f-9446390443e2}"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.235Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: aborting pipeline replication: peer="{Voter server-bdbd9fe5-b7e0-3828-169f-9446390443e2 bdbd9fe5-b7e0-3828-169f-9446390443e2}"
testLoggerAdapter verbose: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.242Z [WARN]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: Election timeout reached, restarting election
    testing.go:149: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.242Z [WARN]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: Election timeout reached, restarting election
testLoggerAdapter verbose: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.242Z [INFO]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: entering candidate state: node="Node at d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e [Candidate]" term=6
    testing.go:149: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.242Z [INFO]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: entering candidate state: node="Node at d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e [Candidate]" term=6
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.242Z [WARN]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: rejecting vote request since our last term is greater: candidate=d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e last-term=3 last-candidate-term=2
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.242Z [WARN]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: rejecting vote request since our last term is greater: candidate=d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e last-term=3 last-candidate-term=2
testLoggerAdapter verbose: server-bdbd9fe5-b7e0-3828-169f-9446390443e2: 2024-01-08T00:31:07.242Z [WARN]  server-bdbd9fe5-b7e0-3828-169f-9446390443e2: rejecting vote request since we have a leader: from=d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e leader=2b405c0f-7a8f-8c3d-909a-721aefefba5f
    testing.go:149: server-bdbd9fe5-b7e0-3828-169f-9446390443e2: 2024-01-08T00:31:07.242Z [WARN]  server-bdbd9fe5-b7e0-3828-169f-9446390443e2: rejecting vote request since we have a leader: from=d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e leader=2b405c0f-7a8f-8c3d-909a-721aefefba5f
testLoggerAdapter verbose: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.316Z [WARN]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: Election timeout reached, restarting election
    testing.go:149: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.316Z [WARN]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: Election timeout reached, restarting election
testLoggerAdapter verbose: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.316Z [INFO]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: entering candidate state: node="Node at d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e [Candidate]" term=7
    testing.go:149: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.316Z [INFO]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: entering candidate state: node="Node at d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e [Candidate]" term=7
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.316Z [WARN]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: rejecting vote request since our last term is greater: candidate=d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e last-term=3 last-candidate-term=2
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.316Z [WARN]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: rejecting vote request since our last term is greater: candidate=d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e last-term=3 last-candidate-term=2
testLoggerAdapter verbose: server-bdbd9fe5-b7e0-3828-169f-9446390443e2: 2024-01-08T00:31:07.316Z [WARN]  server-bdbd9fe5-b7e0-3828-169f-9446390443e2: rejecting vote request since we have a leader: from=d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e leader=2b405c0f-7a8f-8c3d-909a-721aefefba5f
    testing.go:149: server-bdbd9fe5-b7e0-3828-169f-9446390443e2: 2024-01-08T00:31:07.316Z [WARN]  server-bdbd9fe5-b7e0-3828-169f-9446390443e2: rejecting vote request since we have a leader: from=d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e leader=2b405c0f-7a8f-8c3d-909a-721aefefba5f
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.331Z [WARN]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.331Z [WARN]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.331Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: entering candidate state: node="Node at 2b405c0f-7a8f-8c3d-909a-721aefefba5f [Candidate]" term=8
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.331Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: entering candidate state: node="Node at 2b405c0f-7a8f-8c3d-909a-721aefefba5f [Candidate]" term=8
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.331Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: election won: tally=2
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.331Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: election won: tally=2
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.331Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: entering leader state: leader="Node at 2b405c0f-7a8f-8c3d-909a-721aefefba5f [Leader]"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.331Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: entering leader state: leader="Node at 2b405c0f-7a8f-8c3d-909a-721aefefba5f [Leader]"
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.331Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: added peer, starting replication: peer=server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.331Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: added peer, starting replication: peer=server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.331Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: added peer, starting replication: peer=server-bdbd9fe5-b7e0-3828-169f-9446390443e2
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.331Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: added peer, starting replication: peer=server-bdbd9fe5-b7e0-3828-169f-9446390443e2
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.331Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: pipelining replication: peer="{Voter server-bdbd9fe5-b7e0-3828-169f-9446390443e2 bdbd9fe5-b7e0-3828-169f-9446390443e2}"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.331Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: pipelining replication: peer="{Voter server-bdbd9fe5-b7e0-3828-169f-9446390443e2 bdbd9fe5-b7e0-3828-169f-9446390443e2}"
testLoggerAdapter verbose: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.331Z [INFO]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: entering follower state: follower="Node at d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e [Follower]" leader=
    testing.go:149: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.331Z [INFO]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: entering follower state: follower="Node at d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e [Follower]" leader=
testLoggerAdapter verbose: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.331Z [WARN]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: failed to get previous log: previous-index=203 last-index=2 error="log not found"
    testing.go:149: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.331Z [WARN]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: failed to get previous log: previous-index=203 last-index=2 error="log not found"
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.331Z [WARN]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: appendEntries rejected, sending older logs: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}" next=3
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.331Z [WARN]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: appendEntries rejected, sending older logs: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}" next=3
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.331Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to get log: index=2 error="log not found"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.331Z [ERROR] server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: failed to get log: index=2 error="log not found"
    testing.go:153: 2024-01-08T00:31:07.332Z [INFO]  creating new snapshot: path=/tmp/raft3284140110/snapshots/2-102-1704673867332.tmp
testLoggerAdapter verbose: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.341Z [INFO]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: copied to local snapshot: bytes=693
    testing.go:149: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.341Z [INFO]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: copied to local snapshot: bytes=693
testLoggerAdapter verbose: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.342Z [INFO]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: Installed remote snapshot
    testing.go:149: server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: 2024-01-08T00:31:07.342Z [INFO]  server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e: Installed remote snapshot
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.342Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: pipelining replication: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.342Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: pipelining replication: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}"
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.343Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: aborting pipeline replication: peer="{Voter server-bdbd9fe5-b7e0-3828-169f-9446390443e2 bdbd9fe5-b7e0-3828-169f-9446390443e2}"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.343Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: aborting pipeline replication: peer="{Voter server-bdbd9fe5-b7e0-3828-169f-9446390443e2 bdbd9fe5-b7e0-3828-169f-9446390443e2}"
testLoggerAdapter verbose: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.343Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: aborting pipeline replication: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}"
    testing.go:149: server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: 2024-01-08T00:31:07.343Z [INFO]  server-2b405c0f-7a8f-8c3d-909a-721aefefba5f: aborting pipeline replication: peer="{Voter server-d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e d65b2d96-cf22-f00b-f1c2-dd7f8e5daf2e}"
--- PASS: TestRaft_SendSnapshotAndLogsFollower (0.63s)
=== RUN   TestRaft_ReJoinFollower
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.345Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3f73f033-b0e0-fea2-6f6d-489334b4a58b Address:3f73f033-b0e0-fea2-6f6d-489334b4a58b} {Suffrage:Voter ID:server-18fcd28f-913b-5bc0-190a-6785fa6fd2db Address:18fcd28f-913b-5bc0-190a-6785fa6fd2db} {Suffrage:Voter ID:server-84780128-5416-124f-ec25-338c09410456 Address:84780128-5416-124f-ec25-338c09410456}]"
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.345Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3f73f033-b0e0-fea2-6f6d-489334b4a58b Address:3f73f033-b0e0-fea2-6f6d-489334b4a58b} {Suffrage:Voter ID:server-18fcd28f-913b-5bc0-190a-6785fa6fd2db Address:18fcd28f-913b-5bc0-190a-6785fa6fd2db} {Suffrage:Voter ID:server-84780128-5416-124f-ec25-338c09410456 Address:84780128-5416-124f-ec25-338c09410456}]"
testLoggerAdapter verbose: server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: 2024-01-08T00:31:07.345Z [INFO]  server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3f73f033-b0e0-fea2-6f6d-489334b4a58b Address:3f73f033-b0e0-fea2-6f6d-489334b4a58b} {Suffrage:Voter ID:server-18fcd28f-913b-5bc0-190a-6785fa6fd2db Address:18fcd28f-913b-5bc0-190a-6785fa6fd2db} {Suffrage:Voter ID:server-84780128-5416-124f-ec25-338c09410456 Address:84780128-5416-124f-ec25-338c09410456}]"
    testing.go:149: server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: 2024-01-08T00:31:07.345Z [INFO]  server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3f73f033-b0e0-fea2-6f6d-489334b4a58b Address:3f73f033-b0e0-fea2-6f6d-489334b4a58b} {Suffrage:Voter ID:server-18fcd28f-913b-5bc0-190a-6785fa6fd2db Address:18fcd28f-913b-5bc0-190a-6785fa6fd2db} {Suffrage:Voter ID:server-84780128-5416-124f-ec25-338c09410456 Address:84780128-5416-124f-ec25-338c09410456}]"
testLoggerAdapter verbose: server-84780128-5416-124f-ec25-338c09410456: 2024-01-08T00:31:07.345Z [INFO]  server-84780128-5416-124f-ec25-338c09410456: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3f73f033-b0e0-fea2-6f6d-489334b4a58b Address:3f73f033-b0e0-fea2-6f6d-489334b4a58b} {Suffrage:Voter ID:server-18fcd28f-913b-5bc0-190a-6785fa6fd2db Address:18fcd28f-913b-5bc0-190a-6785fa6fd2db} {Suffrage:Voter ID:server-84780128-5416-124f-ec25-338c09410456 Address:84780128-5416-124f-ec25-338c09410456}]"
    testing.go:149: server-84780128-5416-124f-ec25-338c09410456: 2024-01-08T00:31:07.345Z [INFO]  server-84780128-5416-124f-ec25-338c09410456: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3f73f033-b0e0-fea2-6f6d-489334b4a58b Address:3f73f033-b0e0-fea2-6f6d-489334b4a58b} {Suffrage:Voter ID:server-18fcd28f-913b-5bc0-190a-6785fa6fd2db Address:18fcd28f-913b-5bc0-190a-6785fa6fd2db} {Suffrage:Voter ID:server-84780128-5416-124f-ec25-338c09410456 Address:84780128-5416-124f-ec25-338c09410456}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:07.345Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:07.345Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.345Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: entering follower state: follower="Node at 3f73f033-b0e0-fea2-6f6d-489334b4a58b [Follower]" leader=
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.345Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: entering follower state: follower="Node at 3f73f033-b0e0-fea2-6f6d-489334b4a58b [Follower]" leader=
testLoggerAdapter verbose: server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: 2024-01-08T00:31:07.345Z [INFO]  server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: entering follower state: follower="Node at 18fcd28f-913b-5bc0-190a-6785fa6fd2db [Follower]" leader=
    testing.go:149: server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: 2024-01-08T00:31:07.345Z [INFO]  server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: entering follower state: follower="Node at 18fcd28f-913b-5bc0-190a-6785fa6fd2db [Follower]" leader=
testLoggerAdapter verbose: server-84780128-5416-124f-ec25-338c09410456: 2024-01-08T00:31:07.345Z [INFO]  server-84780128-5416-124f-ec25-338c09410456: entering follower state: follower="Node at 84780128-5416-124f-ec25-338c09410456 [Follower]" leader=
    testing.go:149: server-84780128-5416-124f-ec25-338c09410456: 2024-01-08T00:31:07.345Z [INFO]  server-84780128-5416-124f-ec25-338c09410456: entering follower state: follower="Node at 84780128-5416-124f-ec25-338c09410456 [Follower]" leader=
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.410Z [WARN]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.410Z [WARN]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.410Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: entering candidate state: node="Node at 3f73f033-b0e0-fea2-6f6d-489334b4a58b [Candidate]" term=2
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.410Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: entering candidate state: node="Node at 3f73f033-b0e0-fea2-6f6d-489334b4a58b [Candidate]" term=2
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.410Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: election won: tally=2
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.410Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: election won: tally=2
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.410Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: entering leader state: leader="Node at 3f73f033-b0e0-fea2-6f6d-489334b4a58b [Leader]"
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.410Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: entering leader state: leader="Node at 3f73f033-b0e0-fea2-6f6d-489334b4a58b [Leader]"
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.410Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: added peer, starting replication: peer=server-18fcd28f-913b-5bc0-190a-6785fa6fd2db
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.410Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: added peer, starting replication: peer=server-18fcd28f-913b-5bc0-190a-6785fa6fd2db
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.411Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: added peer, starting replication: peer=server-84780128-5416-124f-ec25-338c09410456
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.411Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: added peer, starting replication: peer=server-84780128-5416-124f-ec25-338c09410456
testLoggerAdapter verbose: server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: 2024-01-08T00:31:07.411Z [WARN]  server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: 2024-01-08T00:31:07.411Z [WARN]  server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: 2024-01-08T00:31:07.411Z [INFO]  server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: entering candidate state: node="Node at 18fcd28f-913b-5bc0-190a-6785fa6fd2db [Candidate]" term=2
    testing.go:149: server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: 2024-01-08T00:31:07.411Z [INFO]  server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: entering candidate state: node="Node at 18fcd28f-913b-5bc0-190a-6785fa6fd2db [Candidate]" term=2
testLoggerAdapter verbose: server-84780128-5416-124f-ec25-338c09410456: 2024-01-08T00:31:07.411Z [INFO]  server-84780128-5416-124f-ec25-338c09410456: duplicate requestVote for same term: term=2
    testing.go:149: server-84780128-5416-124f-ec25-338c09410456: 2024-01-08T00:31:07.411Z [INFO]  server-84780128-5416-124f-ec25-338c09410456: duplicate requestVote for same term: term=2
testLoggerAdapter verbose: server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: 2024-01-08T00:31:07.411Z [INFO]  server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: duplicate requestVote for same term: term=2
    testing.go:149: server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: 2024-01-08T00:31:07.411Z [INFO]  server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: duplicate requestVote for same term: term=2
testLoggerAdapter verbose: server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: 2024-01-08T00:31:07.411Z [INFO]  server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: entering follower state: follower="Node at 18fcd28f-913b-5bc0-190a-6785fa6fd2db [Follower]" leader=3f73f033-b0e0-fea2-6f6d-489334b4a58b
    testing.go:149: server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: 2024-01-08T00:31:07.411Z [INFO]  server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: entering follower state: follower="Node at 18fcd28f-913b-5bc0-190a-6785fa6fd2db [Follower]" leader=3f73f033-b0e0-fea2-6f6d-489334b4a58b
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.411Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: pipelining replication: peer="{Voter server-18fcd28f-913b-5bc0-190a-6785fa6fd2db 18fcd28f-913b-5bc0-190a-6785fa6fd2db}"
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.411Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: pipelining replication: peer="{Voter server-18fcd28f-913b-5bc0-190a-6785fa6fd2db 18fcd28f-913b-5bc0-190a-6785fa6fd2db}"
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.411Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: pipelining replication: peer="{Voter server-84780128-5416-124f-ec25-338c09410456 84780128-5416-124f-ec25-338c09410456}"
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.411Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: pipelining replication: peer="{Voter server-84780128-5416-124f-ec25-338c09410456 84780128-5416-124f-ec25-338c09410456}"
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.411Z [WARN]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: rejecting vote request since we have a leader: from=18fcd28f-913b-5bc0-190a-6785fa6fd2db leader=3f73f033-b0e0-fea2-6f6d-489334b4a58b
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.411Z [WARN]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: rejecting vote request since we have a leader: from=18fcd28f-913b-5bc0-190a-6785fa6fd2db leader=3f73f033-b0e0-fea2-6f6d-489334b4a58b
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:07.517Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:07.411660657 +0000 UTC m=+13.458549716 (1 nodes), 66.231366ms from start of poll, 66.65976ms from cluster start. Timeout at 2024-01-08 00:31:07.517311534 +0000 UTC m=+13.564200610, 105.650894ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:07.517Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:07.411660657 +0000 UTC m=+13.458549716 (1 nodes), 66.231366ms from start of poll, 66.65976ms from cluster start. Timeout at 2024-01-08 00:31:07.517311534 +0000 UTC m=+13.564200610, 105.650894ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:07.523Z [INFO]  starting stability test: raft-state=Follower
    testing.go:149: cluster: 2024-01-08T00:31:07.523Z [INFO]  starting stability test: raft-state=Follower
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:07.629Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:07.52317304 +0000 UTC m=+13.570062089 (2 nodes), 2.159µs from start of poll, 178.172133ms from cluster start. Timeout at 2024-01-08 00:31:07.629069058 +0000 UTC m=+13.675958141, 105.896052ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:07.629Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:07.52317304 +0000 UTC m=+13.570062089 (2 nodes), 2.159µs from start of poll, 178.172133ms from cluster start. Timeout at 2024-01-08 00:31:07.629069058 +0000 UTC m=+13.675958141, 105.896052ms after stability
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.629Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: updating configuration: command=RemoveServer server-id=server-18fcd28f-913b-5bc0-190a-6785fa6fd2db server-addr= servers="[{Suffrage:Voter ID:server-3f73f033-b0e0-fea2-6f6d-489334b4a58b Address:3f73f033-b0e0-fea2-6f6d-489334b4a58b} {Suffrage:Voter ID:server-84780128-5416-124f-ec25-338c09410456 Address:84780128-5416-124f-ec25-338c09410456}]"
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.629Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: updating configuration: command=RemoveServer server-id=server-18fcd28f-913b-5bc0-190a-6785fa6fd2db server-addr= servers="[{Suffrage:Voter ID:server-3f73f033-b0e0-fea2-6f6d-489334b4a58b Address:3f73f033-b0e0-fea2-6f6d-489334b4a58b} {Suffrage:Voter ID:server-84780128-5416-124f-ec25-338c09410456 Address:84780128-5416-124f-ec25-338c09410456}]"
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.629Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: removed peer, stopping replication: peer=server-18fcd28f-913b-5bc0-190a-6785fa6fd2db last-index=3
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.629Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: removed peer, stopping replication: peer=server-18fcd28f-913b-5bc0-190a-6785fa6fd2db last-index=3
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.629Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: aborting pipeline replication: peer="{Voter server-18fcd28f-913b-5bc0-190a-6785fa6fd2db 18fcd28f-913b-5bc0-190a-6785fa6fd2db}"
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.629Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: aborting pipeline replication: peer="{Voter server-18fcd28f-913b-5bc0-190a-6785fa6fd2db 18fcd28f-913b-5bc0-190a-6785fa6fd2db}"
testLoggerAdapter verbose: server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: 2024-01-08T00:31:07.725Z [WARN]  server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: heartbeat timeout reached, starting election: last-leader=3f73f033-b0e0-fea2-6f6d-489334b4a58b
    testing.go:149: server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: 2024-01-08T00:31:07.725Z [WARN]  server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: heartbeat timeout reached, starting election: last-leader=3f73f033-b0e0-fea2-6f6d-489334b4a58b
testLoggerAdapter verbose: server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: 2024-01-08T00:31:07.725Z [INFO]  server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: entering candidate state: node="Node at 18fcd28f-913b-5bc0-190a-6785fa6fd2db [Candidate]" term=3
    testing.go:149: server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: 2024-01-08T00:31:07.725Z [INFO]  server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: entering candidate state: node="Node at 18fcd28f-913b-5bc0-190a-6785fa6fd2db [Candidate]" term=3
testLoggerAdapter verbose: server-84780128-5416-124f-ec25-338c09410456: 2024-01-08T00:31:07.725Z [WARN]  server-84780128-5416-124f-ec25-338c09410456: rejecting vote request since we have a leader: from=18fcd28f-913b-5bc0-190a-6785fa6fd2db leader=3f73f033-b0e0-fea2-6f6d-489334b4a58b
    testing.go:149: server-84780128-5416-124f-ec25-338c09410456: 2024-01-08T00:31:07.725Z [WARN]  server-84780128-5416-124f-ec25-338c09410456: rejecting vote request since we have a leader: from=18fcd28f-913b-5bc0-190a-6785fa6fd2db leader=3f73f033-b0e0-fea2-6f6d-489334b4a58b
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.725Z [WARN]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: rejecting vote request since we have a leader: from=18fcd28f-913b-5bc0-190a-6785fa6fd2db leader=3f73f033-b0e0-fea2-6f6d-489334b4a58b
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.725Z [WARN]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: rejecting vote request since we have a leader: from=18fcd28f-913b-5bc0-190a-6785fa6fd2db leader=3f73f033-b0e0-fea2-6f6d-489334b4a58b
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.734Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: updating configuration: command=AddStaging server-id=server-18fcd28f-913b-5bc0-190a-6785fa6fd2db server-addr=18fcd28f-913b-5bc0-190a-6785fa6fd2db servers="[{Suffrage:Voter ID:server-3f73f033-b0e0-fea2-6f6d-489334b4a58b Address:3f73f033-b0e0-fea2-6f6d-489334b4a58b} {Suffrage:Voter ID:server-84780128-5416-124f-ec25-338c09410456 Address:84780128-5416-124f-ec25-338c09410456} {Suffrage:Voter ID:server-18fcd28f-913b-5bc0-190a-6785fa6fd2db Address:18fcd28f-913b-5bc0-190a-6785fa6fd2db}]"
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.734Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: updating configuration: command=AddStaging server-id=server-18fcd28f-913b-5bc0-190a-6785fa6fd2db server-addr=18fcd28f-913b-5bc0-190a-6785fa6fd2db servers="[{Suffrage:Voter ID:server-3f73f033-b0e0-fea2-6f6d-489334b4a58b Address:3f73f033-b0e0-fea2-6f6d-489334b4a58b} {Suffrage:Voter ID:server-84780128-5416-124f-ec25-338c09410456 Address:84780128-5416-124f-ec25-338c09410456} {Suffrage:Voter ID:server-18fcd28f-913b-5bc0-190a-6785fa6fd2db Address:18fcd28f-913b-5bc0-190a-6785fa6fd2db}]"
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.734Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: added peer, starting replication: peer=server-18fcd28f-913b-5bc0-190a-6785fa6fd2db
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.734Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: added peer, starting replication: peer=server-18fcd28f-913b-5bc0-190a-6785fa6fd2db
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.734Z [ERROR] server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: peer has newer term, stopping replication: peer="{Voter server-18fcd28f-913b-5bc0-190a-6785fa6fd2db 18fcd28f-913b-5bc0-190a-6785fa6fd2db}"
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.734Z [ERROR] server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: peer has newer term, stopping replication: peer="{Voter server-18fcd28f-913b-5bc0-190a-6785fa6fd2db 18fcd28f-913b-5bc0-190a-6785fa6fd2db}"
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.734Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: entering follower state: follower="Node at 3f73f033-b0e0-fea2-6f6d-489334b4a58b [Follower]" leader=
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.734Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: entering follower state: follower="Node at 3f73f033-b0e0-fea2-6f6d-489334b4a58b [Follower]" leader=
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:07.734Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:07.734Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.735Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: aborting pipeline replication: peer="{Voter server-84780128-5416-124f-ec25-338c09410456 84780128-5416-124f-ec25-338c09410456}"
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.735Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: aborting pipeline replication: peer="{Voter server-84780128-5416-124f-ec25-338c09410456 84780128-5416-124f-ec25-338c09410456}"
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.792Z [WARN]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.792Z [WARN]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.792Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: entering candidate state: node="Node at 3f73f033-b0e0-fea2-6f6d-489334b4a58b [Candidate]" term=3
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.792Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: entering candidate state: node="Node at 3f73f033-b0e0-fea2-6f6d-489334b4a58b [Candidate]" term=3
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.792Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: election won: tally=2
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.792Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: election won: tally=2
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.792Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: entering leader state: leader="Node at 3f73f033-b0e0-fea2-6f6d-489334b4a58b [Leader]"
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.792Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: entering leader state: leader="Node at 3f73f033-b0e0-fea2-6f6d-489334b4a58b [Leader]"
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.792Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: added peer, starting replication: peer=server-84780128-5416-124f-ec25-338c09410456
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.792Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: added peer, starting replication: peer=server-84780128-5416-124f-ec25-338c09410456
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.792Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: added peer, starting replication: peer=server-18fcd28f-913b-5bc0-190a-6785fa6fd2db
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.792Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: added peer, starting replication: peer=server-18fcd28f-913b-5bc0-190a-6785fa6fd2db
testLoggerAdapter verbose: server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: 2024-01-08T00:31:07.792Z [WARN]  server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: failed to get previous log: previous-index=4 last-index=3 error="log not found"
    testing.go:149: server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: 2024-01-08T00:31:07.792Z [WARN]  server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: failed to get previous log: previous-index=4 last-index=3 error="log not found"
testLoggerAdapter verbose: server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: 2024-01-08T00:31:07.792Z [INFO]  server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: entering follower state: follower="Node at 18fcd28f-913b-5bc0-190a-6785fa6fd2db [Follower]" leader=3f73f033-b0e0-fea2-6f6d-489334b4a58b
    testing.go:149: server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: 2024-01-08T00:31:07.792Z [INFO]  server-18fcd28f-913b-5bc0-190a-6785fa6fd2db: entering follower state: follower="Node at 18fcd28f-913b-5bc0-190a-6785fa6fd2db [Follower]" leader=3f73f033-b0e0-fea2-6f6d-489334b4a58b
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.792Z [WARN]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: appendEntries rejected, sending older logs: peer="{Voter server-18fcd28f-913b-5bc0-190a-6785fa6fd2db 18fcd28f-913b-5bc0-190a-6785fa6fd2db}" next=4
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.792Z [WARN]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: appendEntries rejected, sending older logs: peer="{Voter server-18fcd28f-913b-5bc0-190a-6785fa6fd2db 18fcd28f-913b-5bc0-190a-6785fa6fd2db}" next=4
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.792Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: pipelining replication: peer="{Voter server-18fcd28f-913b-5bc0-190a-6785fa6fd2db 18fcd28f-913b-5bc0-190a-6785fa6fd2db}"
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.792Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: pipelining replication: peer="{Voter server-18fcd28f-913b-5bc0-190a-6785fa6fd2db 18fcd28f-913b-5bc0-190a-6785fa6fd2db}"
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.793Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: pipelining replication: peer="{Voter server-84780128-5416-124f-ec25-338c09410456 84780128-5416-124f-ec25-338c09410456}"
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.793Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: pipelining replication: peer="{Voter server-84780128-5416-124f-ec25-338c09410456 84780128-5416-124f-ec25-338c09410456}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:07.898Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:07.793079797 +0000 UTC m=+13.839968838 (1 nodes), 58.109013ms from start of poll, 448.078882ms from cluster start. Timeout at 2024-01-08 00:31:07.898427875 +0000 UTC m=+13.945317109, 105.348271ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:07.898Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:07.793079797 +0000 UTC m=+13.839968838 (1 nodes), 58.109013ms from start of poll, 448.078882ms from cluster start. Timeout at 2024-01-08 00:31:07.898427875 +0000 UTC m=+13.945317109, 105.348271ms after stability
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.898Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: aborting pipeline replication: peer="{Voter server-18fcd28f-913b-5bc0-190a-6785fa6fd2db 18fcd28f-913b-5bc0-190a-6785fa6fd2db}"
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.898Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: aborting pipeline replication: peer="{Voter server-18fcd28f-913b-5bc0-190a-6785fa6fd2db 18fcd28f-913b-5bc0-190a-6785fa6fd2db}"
testLoggerAdapter verbose: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.898Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: aborting pipeline replication: peer="{Voter server-84780128-5416-124f-ec25-338c09410456 84780128-5416-124f-ec25-338c09410456}"
    testing.go:149: server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: 2024-01-08T00:31:07.898Z [INFO]  server-3f73f033-b0e0-fea2-6f6d-489334b4a58b: aborting pipeline replication: peer="{Voter server-84780128-5416-124f-ec25-338c09410456 84780128-5416-124f-ec25-338c09410456}"
--- PASS: TestRaft_ReJoinFollower (0.55s)
=== RUN   TestRaft_LeaderLeaseExpire
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:07.899Z [INFO]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9 Address:b0b383b6-dd43-0559-bb4f-8d2b4981f9f9} {Suffrage:Voter ID:server-1ab6f754-920c-98ef-c632-9c21a16b617a Address:1ab6f754-920c-98ef-c632-9c21a16b617a}]"
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:07.899Z [INFO]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9 Address:b0b383b6-dd43-0559-bb4f-8d2b4981f9f9} {Suffrage:Voter ID:server-1ab6f754-920c-98ef-c632-9c21a16b617a Address:1ab6f754-920c-98ef-c632-9c21a16b617a}]"
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:07.899Z [INFO]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: entering follower state: follower="Node at b0b383b6-dd43-0559-bb4f-8d2b4981f9f9 [Follower]" leader=
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:07.899Z [INFO]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: entering follower state: follower="Node at b0b383b6-dd43-0559-bb4f-8d2b4981f9f9 [Follower]" leader=
testLoggerAdapter verbose: server-1ab6f754-920c-98ef-c632-9c21a16b617a: 2024-01-08T00:31:07.899Z [INFO]  server-1ab6f754-920c-98ef-c632-9c21a16b617a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9 Address:b0b383b6-dd43-0559-bb4f-8d2b4981f9f9} {Suffrage:Voter ID:server-1ab6f754-920c-98ef-c632-9c21a16b617a Address:1ab6f754-920c-98ef-c632-9c21a16b617a}]"
    testing.go:149: server-1ab6f754-920c-98ef-c632-9c21a16b617a: 2024-01-08T00:31:07.899Z [INFO]  server-1ab6f754-920c-98ef-c632-9c21a16b617a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9 Address:b0b383b6-dd43-0559-bb4f-8d2b4981f9f9} {Suffrage:Voter ID:server-1ab6f754-920c-98ef-c632-9c21a16b617a Address:1ab6f754-920c-98ef-c632-9c21a16b617a}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:07.900Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:07.900Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-1ab6f754-920c-98ef-c632-9c21a16b617a: 2024-01-08T00:31:07.900Z [INFO]  server-1ab6f754-920c-98ef-c632-9c21a16b617a: entering follower state: follower="Node at 1ab6f754-920c-98ef-c632-9c21a16b617a [Follower]" leader=
    testing.go:149: server-1ab6f754-920c-98ef-c632-9c21a16b617a: 2024-01-08T00:31:07.900Z [INFO]  server-1ab6f754-920c-98ef-c632-9c21a16b617a: entering follower state: follower="Node at 1ab6f754-920c-98ef-c632-9c21a16b617a [Follower]" leader=
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:07.953Z [WARN]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:07.953Z [WARN]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:07.953Z [INFO]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: entering candidate state: node="Node at b0b383b6-dd43-0559-bb4f-8d2b4981f9f9 [Candidate]" term=2
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:07.953Z [INFO]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: entering candidate state: node="Node at b0b383b6-dd43-0559-bb4f-8d2b4981f9f9 [Candidate]" term=2
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:07.953Z [INFO]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: election won: tally=2
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:07.953Z [INFO]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: election won: tally=2
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:07.953Z [INFO]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: entering leader state: leader="Node at b0b383b6-dd43-0559-bb4f-8d2b4981f9f9 [Leader]"
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:07.953Z [INFO]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: entering leader state: leader="Node at b0b383b6-dd43-0559-bb4f-8d2b4981f9f9 [Leader]"
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:07.954Z [INFO]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: added peer, starting replication: peer=server-1ab6f754-920c-98ef-c632-9c21a16b617a
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:07.954Z [INFO]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: added peer, starting replication: peer=server-1ab6f754-920c-98ef-c632-9c21a16b617a
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:07.954Z [INFO]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: pipelining replication: peer="{Voter server-1ab6f754-920c-98ef-c632-9c21a16b617a 1ab6f754-920c-98ef-c632-9c21a16b617a}"
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:07.954Z [INFO]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: pipelining replication: peer="{Voter server-1ab6f754-920c-98ef-c632-9c21a16b617a 1ab6f754-920c-98ef-c632-9c21a16b617a}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:08.060Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:07.953985543 +0000 UTC m=+14.000874588 (1 nodes), 53.897798ms from start of poll, 54.300972ms from cluster start. Timeout at 2024-01-08 00:31:08.060117103 +0000 UTC m=+14.107006146, 106.131558ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:08.060Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:07.953985543 +0000 UTC m=+14.000874588 (1 nodes), 53.897798ms from start of poll, 54.300972ms from cluster start. Timeout at 2024-01-08 00:31:08.060117103 +0000 UTC m=+14.107006146, 106.131558ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:08.065Z [INFO]  starting stability test: raft-state=Follower
    testing.go:149: cluster: 2024-01-08T00:31:08.065Z [INFO]  starting stability test: raft-state=Follower
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:08.171Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:08.065754205 +0000 UTC m=+14.112643256 (1 nodes), 1.74µs from start of poll, 166.06964ms from cluster start. Timeout at 2024-01-08 00:31:08.171824342 +0000 UTC m=+14.218713419, 106.070163ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:08.171Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:08.065754205 +0000 UTC m=+14.112643256 (1 nodes), 1.74µs from start of poll, 166.06964ms from cluster start. Timeout at 2024-01-08 00:31:08.171824342 +0000 UTC m=+14.218713419, 106.070163ms after stability
    raft_test.go:1492: [INFO] Disconnecting Node at 1ab6f754-920c-98ef-c632-9c21a16b617a [Follower]
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.177Z [ERROR] server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: failed to heartbeat to: peer=1ab6f754-920c-98ef-c632-9c21a16b617a error="failed to connect to peer: 1ab6f754-920c-98ef-c632-9c21a16b617a"
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.177Z [ERROR] server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: failed to heartbeat to: peer=1ab6f754-920c-98ef-c632-9c21a16b617a error="failed to connect to peer: 1ab6f754-920c-98ef-c632-9c21a16b617a"
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.180Z [ERROR] server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: failed to pipeline appendEntries: peer="{Voter server-1ab6f754-920c-98ef-c632-9c21a16b617a 1ab6f754-920c-98ef-c632-9c21a16b617a}" error="append pipeline closed"
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.180Z [ERROR] server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: failed to pipeline appendEntries: peer="{Voter server-1ab6f754-920c-98ef-c632-9c21a16b617a 1ab6f754-920c-98ef-c632-9c21a16b617a}" error="append pipeline closed"
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.180Z [INFO]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: aborting pipeline replication: peer="{Voter server-1ab6f754-920c-98ef-c632-9c21a16b617a 1ab6f754-920c-98ef-c632-9c21a16b617a}"
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.180Z [INFO]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: aborting pipeline replication: peer="{Voter server-1ab6f754-920c-98ef-c632-9c21a16b617a 1ab6f754-920c-98ef-c632-9c21a16b617a}"
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.189Z [ERROR] server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: failed to appendEntries to: peer="{Voter server-1ab6f754-920c-98ef-c632-9c21a16b617a 1ab6f754-920c-98ef-c632-9c21a16b617a}" error="failed to connect to peer: 1ab6f754-920c-98ef-c632-9c21a16b617a"
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.189Z [ERROR] server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: failed to appendEntries to: peer="{Voter server-1ab6f754-920c-98ef-c632-9c21a16b617a 1ab6f754-920c-98ef-c632-9c21a16b617a}" error="failed to connect to peer: 1ab6f754-920c-98ef-c632-9c21a16b617a"
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.196Z [ERROR] server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: failed to heartbeat to: peer=1ab6f754-920c-98ef-c632-9c21a16b617a error="failed to connect to peer: 1ab6f754-920c-98ef-c632-9c21a16b617a"
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.196Z [ERROR] server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: failed to heartbeat to: peer=1ab6f754-920c-98ef-c632-9c21a16b617a error="failed to connect to peer: 1ab6f754-920c-98ef-c632-9c21a16b617a"
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.208Z [ERROR] server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: failed to appendEntries to: peer="{Voter server-1ab6f754-920c-98ef-c632-9c21a16b617a 1ab6f754-920c-98ef-c632-9c21a16b617a}" error="failed to connect to peer: 1ab6f754-920c-98ef-c632-9c21a16b617a"
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.208Z [ERROR] server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: failed to appendEntries to: peer="{Voter server-1ab6f754-920c-98ef-c632-9c21a16b617a 1ab6f754-920c-98ef-c632-9c21a16b617a}" error="failed to connect to peer: 1ab6f754-920c-98ef-c632-9c21a16b617a"
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.215Z [ERROR] server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: failed to heartbeat to: peer=1ab6f754-920c-98ef-c632-9c21a16b617a error="failed to connect to peer: 1ab6f754-920c-98ef-c632-9c21a16b617a"
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.215Z [ERROR] server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: failed to heartbeat to: peer=1ab6f754-920c-98ef-c632-9c21a16b617a error="failed to connect to peer: 1ab6f754-920c-98ef-c632-9c21a16b617a"
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.221Z [WARN]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: failed to contact: server-id=server-1ab6f754-920c-98ef-c632-9c21a16b617a time=51.037614ms
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.221Z [WARN]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: failed to contact: server-id=server-1ab6f754-920c-98ef-c632-9c21a16b617a time=51.037614ms
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.221Z [WARN]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: failed to contact quorum of nodes, stepping down
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.221Z [WARN]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: failed to contact quorum of nodes, stepping down
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.221Z [INFO]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: entering follower state: follower="Node at b0b383b6-dd43-0559-bb4f-8d2b4981f9f9 [Follower]" leader=
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.221Z [INFO]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: entering follower state: follower="Node at b0b383b6-dd43-0559-bb4f-8d2b4981f9f9 [Follower]" leader=
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:08.221Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:08.221Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.227Z [ERROR] server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: failed to appendEntries to: peer="{Voter server-1ab6f754-920c-98ef-c632-9c21a16b617a 1ab6f754-920c-98ef-c632-9c21a16b617a}" error="failed to connect to peer: 1ab6f754-920c-98ef-c632-9c21a16b617a"
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.227Z [ERROR] server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: failed to appendEntries to: peer="{Voter server-1ab6f754-920c-98ef-c632-9c21a16b617a 1ab6f754-920c-98ef-c632-9c21a16b617a}" error="failed to connect to peer: 1ab6f754-920c-98ef-c632-9c21a16b617a"
testLoggerAdapter verbose: server-1ab6f754-920c-98ef-c632-9c21a16b617a: 2024-01-08T00:31:08.252Z [WARN]  server-1ab6f754-920c-98ef-c632-9c21a16b617a: heartbeat timeout reached, starting election: last-leader=b0b383b6-dd43-0559-bb4f-8d2b4981f9f9
    testing.go:149: server-1ab6f754-920c-98ef-c632-9c21a16b617a: 2024-01-08T00:31:08.252Z [WARN]  server-1ab6f754-920c-98ef-c632-9c21a16b617a: heartbeat timeout reached, starting election: last-leader=b0b383b6-dd43-0559-bb4f-8d2b4981f9f9
testLoggerAdapter verbose: server-1ab6f754-920c-98ef-c632-9c21a16b617a: 2024-01-08T00:31:08.252Z [INFO]  server-1ab6f754-920c-98ef-c632-9c21a16b617a: entering candidate state: node="Node at 1ab6f754-920c-98ef-c632-9c21a16b617a [Candidate]" term=3
    testing.go:149: server-1ab6f754-920c-98ef-c632-9c21a16b617a: 2024-01-08T00:31:08.252Z [INFO]  server-1ab6f754-920c-98ef-c632-9c21a16b617a: entering candidate state: node="Node at 1ab6f754-920c-98ef-c632-9c21a16b617a [Candidate]" term=3
testLoggerAdapter verbose: server-1ab6f754-920c-98ef-c632-9c21a16b617a: 2024-01-08T00:31:08.252Z [ERROR] server-1ab6f754-920c-98ef-c632-9c21a16b617a: failed to make requestVote RPC: target="{Voter server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9 b0b383b6-dd43-0559-bb4f-8d2b4981f9f9}" error="failed to connect to peer: b0b383b6-dd43-0559-bb4f-8d2b4981f9f9"
    testing.go:149: server-1ab6f754-920c-98ef-c632-9c21a16b617a: 2024-01-08T00:31:08.252Z [ERROR] server-1ab6f754-920c-98ef-c632-9c21a16b617a: failed to make requestVote RPC: target="{Voter server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9 b0b383b6-dd43-0559-bb4f-8d2b4981f9f9}" error="failed to connect to peer: b0b383b6-dd43-0559-bb4f-8d2b4981f9f9"
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.317Z [WARN]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.317Z [WARN]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.317Z [INFO]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: entering candidate state: node="Node at b0b383b6-dd43-0559-bb4f-8d2b4981f9f9 [Candidate]" term=3
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.317Z [INFO]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: entering candidate state: node="Node at b0b383b6-dd43-0559-bb4f-8d2b4981f9f9 [Candidate]" term=3
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.317Z [ERROR] server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: failed to make requestVote RPC: target="{Voter server-1ab6f754-920c-98ef-c632-9c21a16b617a 1ab6f754-920c-98ef-c632-9c21a16b617a}" error="failed to connect to peer: 1ab6f754-920c-98ef-c632-9c21a16b617a"
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.317Z [ERROR] server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: failed to make requestVote RPC: target="{Voter server-1ab6f754-920c-98ef-c632-9c21a16b617a 1ab6f754-920c-98ef-c632-9c21a16b617a}" error="failed to connect to peer: 1ab6f754-920c-98ef-c632-9c21a16b617a"
testLoggerAdapter verbose: server-1ab6f754-920c-98ef-c632-9c21a16b617a: 2024-01-08T00:31:08.332Z [WARN]  server-1ab6f754-920c-98ef-c632-9c21a16b617a: Election timeout reached, restarting election
    testing.go:149: server-1ab6f754-920c-98ef-c632-9c21a16b617a: 2024-01-08T00:31:08.332Z [WARN]  server-1ab6f754-920c-98ef-c632-9c21a16b617a: Election timeout reached, restarting election
testLoggerAdapter verbose: server-1ab6f754-920c-98ef-c632-9c21a16b617a: 2024-01-08T00:31:08.332Z [INFO]  server-1ab6f754-920c-98ef-c632-9c21a16b617a: entering candidate state: node="Node at 1ab6f754-920c-98ef-c632-9c21a16b617a [Candidate]" term=4
    testing.go:149: server-1ab6f754-920c-98ef-c632-9c21a16b617a: 2024-01-08T00:31:08.332Z [INFO]  server-1ab6f754-920c-98ef-c632-9c21a16b617a: entering candidate state: node="Node at 1ab6f754-920c-98ef-c632-9c21a16b617a [Candidate]" term=4
testLoggerAdapter verbose: server-1ab6f754-920c-98ef-c632-9c21a16b617a: 2024-01-08T00:31:08.332Z [ERROR] server-1ab6f754-920c-98ef-c632-9c21a16b617a: failed to make requestVote RPC: target="{Voter server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9 b0b383b6-dd43-0559-bb4f-8d2b4981f9f9}" error="failed to connect to peer: b0b383b6-dd43-0559-bb4f-8d2b4981f9f9"
    testing.go:149: server-1ab6f754-920c-98ef-c632-9c21a16b617a: 2024-01-08T00:31:08.332Z [ERROR] server-1ab6f754-920c-98ef-c632-9c21a16b617a: failed to make requestVote RPC: target="{Voter server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9 b0b383b6-dd43-0559-bb4f-8d2b4981f9f9}" error="failed to connect to peer: b0b383b6-dd43-0559-bb4f-8d2b4981f9f9"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:08.358Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:08.25224629 +0000 UTC m=+14.299135403 (0 nodes), 30.291665ms from start of poll, 352.561787ms from cluster start. Timeout at 2024-01-08 00:31:08.357952798 +0000 UTC m=+14.404841896, 105.706493ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:08.358Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:08.25224629 +0000 UTC m=+14.299135403 (0 nodes), 30.291665ms from start of poll, 352.561787ms from cluster start. Timeout at 2024-01-08 00:31:08.357952798 +0000 UTC m=+14.404841896, 105.706493ms after stability
testLoggerAdapter verbose: server-1ab6f754-920c-98ef-c632-9c21a16b617a: 2024-01-08T00:31:08.397Z [WARN]  server-1ab6f754-920c-98ef-c632-9c21a16b617a: Election timeout reached, restarting election
    testing.go:149: server-1ab6f754-920c-98ef-c632-9c21a16b617a: 2024-01-08T00:31:08.397Z [WARN]  server-1ab6f754-920c-98ef-c632-9c21a16b617a: Election timeout reached, restarting election
testLoggerAdapter verbose: server-1ab6f754-920c-98ef-c632-9c21a16b617a: 2024-01-08T00:31:08.397Z [INFO]  server-1ab6f754-920c-98ef-c632-9c21a16b617a: entering candidate state: node="Node at 1ab6f754-920c-98ef-c632-9c21a16b617a [Candidate]" term=5
    testing.go:149: server-1ab6f754-920c-98ef-c632-9c21a16b617a: 2024-01-08T00:31:08.397Z [INFO]  server-1ab6f754-920c-98ef-c632-9c21a16b617a: entering candidate state: node="Node at 1ab6f754-920c-98ef-c632-9c21a16b617a [Candidate]" term=5
testLoggerAdapter verbose: server-1ab6f754-920c-98ef-c632-9c21a16b617a: 2024-01-08T00:31:08.397Z [ERROR] server-1ab6f754-920c-98ef-c632-9c21a16b617a: failed to make requestVote RPC: target="{Voter server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9 b0b383b6-dd43-0559-bb4f-8d2b4981f9f9}" error="failed to connect to peer: b0b383b6-dd43-0559-bb4f-8d2b4981f9f9"
    testing.go:149: server-1ab6f754-920c-98ef-c632-9c21a16b617a: 2024-01-08T00:31:08.397Z [ERROR] server-1ab6f754-920c-98ef-c632-9c21a16b617a: failed to make requestVote RPC: target="{Voter server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9 b0b383b6-dd43-0559-bb4f-8d2b4981f9f9}" error="failed to connect to peer: b0b383b6-dd43-0559-bb4f-8d2b4981f9f9"
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.399Z [WARN]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: Election timeout reached, restarting election
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.399Z [WARN]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: Election timeout reached, restarting election
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.399Z [INFO]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: entering candidate state: node="Node at b0b383b6-dd43-0559-bb4f-8d2b4981f9f9 [Candidate]" term=4
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.399Z [INFO]  server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: entering candidate state: node="Node at b0b383b6-dd43-0559-bb4f-8d2b4981f9f9 [Candidate]" term=4
testLoggerAdapter verbose: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.399Z [ERROR] server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: failed to make requestVote RPC: target="{Voter server-1ab6f754-920c-98ef-c632-9c21a16b617a 1ab6f754-920c-98ef-c632-9c21a16b617a}" error="failed to connect to peer: 1ab6f754-920c-98ef-c632-9c21a16b617a"
    testing.go:149: server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: 2024-01-08T00:31:08.399Z [ERROR] server-b0b383b6-dd43-0559-bb4f-8d2b4981f9f9: failed to make requestVote RPC: target="{Voter server-1ab6f754-920c-98ef-c632-9c21a16b617a 1ab6f754-920c-98ef-c632-9c21a16b617a}" error="failed to connect to peer: 1ab6f754-920c-98ef-c632-9c21a16b617a"
--- PASS: TestRaft_LeaderLeaseExpire (0.56s)
=== RUN   TestRaft_Barrier
testLoggerAdapter verbose: server-7822456f-2ca2-2584-f424-e83c56de5b2e: 2024-01-08T00:31:08.464Z [INFO]  server-7822456f-2ca2-2584-f424-e83c56de5b2e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7822456f-2ca2-2584-f424-e83c56de5b2e Address:7822456f-2ca2-2584-f424-e83c56de5b2e} {Suffrage:Voter ID:server-b877f982-a861-9ef1-b421-e46f5abdff30 Address:b877f982-a861-9ef1-b421-e46f5abdff30} {Suffrage:Voter ID:server-52e9f313-d0a2-1b7c-0541-bb792953c801 Address:52e9f313-d0a2-1b7c-0541-bb792953c801}]"
    testing.go:149: server-7822456f-2ca2-2584-f424-e83c56de5b2e: 2024-01-08T00:31:08.464Z [INFO]  server-7822456f-2ca2-2584-f424-e83c56de5b2e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7822456f-2ca2-2584-f424-e83c56de5b2e Address:7822456f-2ca2-2584-f424-e83c56de5b2e} {Suffrage:Voter ID:server-b877f982-a861-9ef1-b421-e46f5abdff30 Address:b877f982-a861-9ef1-b421-e46f5abdff30} {Suffrage:Voter ID:server-52e9f313-d0a2-1b7c-0541-bb792953c801 Address:52e9f313-d0a2-1b7c-0541-bb792953c801}]"
testLoggerAdapter verbose: server-b877f982-a861-9ef1-b421-e46f5abdff30: 2024-01-08T00:31:08.465Z [INFO]  server-b877f982-a861-9ef1-b421-e46f5abdff30: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7822456f-2ca2-2584-f424-e83c56de5b2e Address:7822456f-2ca2-2584-f424-e83c56de5b2e} {Suffrage:Voter ID:server-b877f982-a861-9ef1-b421-e46f5abdff30 Address:b877f982-a861-9ef1-b421-e46f5abdff30} {Suffrage:Voter ID:server-52e9f313-d0a2-1b7c-0541-bb792953c801 Address:52e9f313-d0a2-1b7c-0541-bb792953c801}]"
    testing.go:149: server-b877f982-a861-9ef1-b421-e46f5abdff30: 2024-01-08T00:31:08.465Z [INFO]  server-b877f982-a861-9ef1-b421-e46f5abdff30: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7822456f-2ca2-2584-f424-e83c56de5b2e Address:7822456f-2ca2-2584-f424-e83c56de5b2e} {Suffrage:Voter ID:server-b877f982-a861-9ef1-b421-e46f5abdff30 Address:b877f982-a861-9ef1-b421-e46f5abdff30} {Suffrage:Voter ID:server-52e9f313-d0a2-1b7c-0541-bb792953c801 Address:52e9f313-d0a2-1b7c-0541-bb792953c801}]"
testLoggerAdapter verbose: server-7822456f-2ca2-2584-f424-e83c56de5b2e: 2024-01-08T00:31:08.464Z [INFO]  server-7822456f-2ca2-2584-f424-e83c56de5b2e: entering follower state: follower="Node at 7822456f-2ca2-2584-f424-e83c56de5b2e [Follower]" leader=
    testing.go:149: server-7822456f-2ca2-2584-f424-e83c56de5b2e: 2024-01-08T00:31:08.464Z [INFO]  server-7822456f-2ca2-2584-f424-e83c56de5b2e: entering follower state: follower="Node at 7822456f-2ca2-2584-f424-e83c56de5b2e [Follower]" leader=
testLoggerAdapter verbose: server-b877f982-a861-9ef1-b421-e46f5abdff30: 2024-01-08T00:31:08.465Z [INFO]  server-b877f982-a861-9ef1-b421-e46f5abdff30: entering follower state: follower="Node at b877f982-a861-9ef1-b421-e46f5abdff30 [Follower]" leader=
testLoggerAdapter verbose: server-52e9f313-d0a2-1b7c-0541-bb792953c801: 2024-01-08T00:31:08.465Z [INFO]  server-52e9f313-d0a2-1b7c-0541-bb792953c801: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7822456f-2ca2-2584-f424-e83c56de5b2e Address:7822456f-2ca2-2584-f424-e83c56de5b2e} {Suffrage:Voter ID:server-b877f982-a861-9ef1-b421-e46f5abdff30 Address:b877f982-a861-9ef1-b421-e46f5abdff30} {Suffrage:Voter ID:server-52e9f313-d0a2-1b7c-0541-bb792953c801 Address:52e9f313-d0a2-1b7c-0541-bb792953c801}]"
    testing.go:149: server-b877f982-a861-9ef1-b421-e46f5abdff30: 2024-01-08T00:31:08.465Z [INFO]  server-b877f982-a861-9ef1-b421-e46f5abdff30: entering follower state: follower="Node at b877f982-a861-9ef1-b421-e46f5abdff30 [Follower]" leader=
    testing.go:149: server-52e9f313-d0a2-1b7c-0541-bb792953c801: 2024-01-08T00:31:08.465Z [INFO]  server-52e9f313-d0a2-1b7c-0541-bb792953c801: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-7822456f-2ca2-2584-f424-e83c56de5b2e Address:7822456f-2ca2-2584-f424-e83c56de5b2e} {Suffrage:Voter ID:server-b877f982-a861-9ef1-b421-e46f5abdff30 Address:b877f982-a861-9ef1-b421-e46f5abdff30} {Suffrage:Voter ID:server-52e9f313-d0a2-1b7c-0541-bb792953c801 Address:52e9f313-d0a2-1b7c-0541-bb792953c801}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:08.465Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:08.465Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-52e9f313-d0a2-1b7c-0541-bb792953c801: 2024-01-08T00:31:08.465Z [INFO]  server-52e9f313-d0a2-1b7c-0541-bb792953c801: entering follower state: follower="Node at 52e9f313-d0a2-1b7c-0541-bb792953c801 [Follower]" leader=
    testing.go:149: server-52e9f313-d0a2-1b7c-0541-bb792953c801: 2024-01-08T00:31:08.465Z [INFO]  server-52e9f313-d0a2-1b7c-0541-bb792953c801: entering follower state: follower="Node at 52e9f313-d0a2-1b7c-0541-bb792953c801 [Follower]" leader=
testLoggerAdapter verbose: server-52e9f313-d0a2-1b7c-0541-bb792953c801: 2024-01-08T00:31:08.554Z [WARN]  server-52e9f313-d0a2-1b7c-0541-bb792953c801: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-52e9f313-d0a2-1b7c-0541-bb792953c801: 2024-01-08T00:31:08.554Z [WARN]  server-52e9f313-d0a2-1b7c-0541-bb792953c801: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-52e9f313-d0a2-1b7c-0541-bb792953c801: 2024-01-08T00:31:08.554Z [INFO]  server-52e9f313-d0a2-1b7c-0541-bb792953c801: entering candidate state: node="Node at 52e9f313-d0a2-1b7c-0541-bb792953c801 [Candidate]" term=2
    testing.go:149: server-52e9f313-d0a2-1b7c-0541-bb792953c801: 2024-01-08T00:31:08.554Z [INFO]  server-52e9f313-d0a2-1b7c-0541-bb792953c801: entering candidate state: node="Node at 52e9f313-d0a2-1b7c-0541-bb792953c801 [Candidate]" term=2
testLoggerAdapter verbose: server-52e9f313-d0a2-1b7c-0541-bb792953c801: 2024-01-08T00:31:08.554Z [INFO]  server-52e9f313-d0a2-1b7c-0541-bb792953c801: election won: tally=2
    testing.go:149: server-52e9f313-d0a2-1b7c-0541-bb792953c801: 2024-01-08T00:31:08.554Z [INFO]  server-52e9f313-d0a2-1b7c-0541-bb792953c801: election won: tally=2
testLoggerAdapter verbose: server-52e9f313-d0a2-1b7c-0541-bb792953c801: 2024-01-08T00:31:08.554Z [INFO]  server-52e9f313-d0a2-1b7c-0541-bb792953c801: entering leader state: leader="Node at 52e9f313-d0a2-1b7c-0541-bb792953c801 [Leader]"
    testing.go:149: server-52e9f313-d0a2-1b7c-0541-bb792953c801: 2024-01-08T00:31:08.554Z [INFO]  server-52e9f313-d0a2-1b7c-0541-bb792953c801: entering leader state: leader="Node at 52e9f313-d0a2-1b7c-0541-bb792953c801 [Leader]"
testLoggerAdapter verbose: server-52e9f313-d0a2-1b7c-0541-bb792953c801: 2024-01-08T00:31:08.554Z [INFO]  server-52e9f313-d0a2-1b7c-0541-bb792953c801: added peer, starting replication: peer=server-7822456f-2ca2-2584-f424-e83c56de5b2e
    testing.go:149: server-52e9f313-d0a2-1b7c-0541-bb792953c801: 2024-01-08T00:31:08.554Z [INFO]  server-52e9f313-d0a2-1b7c-0541-bb792953c801: added peer, starting replication: peer=server-7822456f-2ca2-2584-f424-e83c56de5b2e
testLoggerAdapter verbose: server-52e9f313-d0a2-1b7c-0541-bb792953c801: 2024-01-08T00:31:08.554Z [INFO]  server-52e9f313-d0a2-1b7c-0541-bb792953c801: added peer, starting replication: peer=server-b877f982-a861-9ef1-b421-e46f5abdff30
    testing.go:149: server-52e9f313-d0a2-1b7c-0541-bb792953c801: 2024-01-08T00:31:08.554Z [INFO]  server-52e9f313-d0a2-1b7c-0541-bb792953c801: added peer, starting replication: peer=server-b877f982-a861-9ef1-b421-e46f5abdff30
testLoggerAdapter verbose: server-52e9f313-d0a2-1b7c-0541-bb792953c801: 2024-01-08T00:31:08.554Z [INFO]  server-52e9f313-d0a2-1b7c-0541-bb792953c801: pipelining replication: peer="{Voter server-7822456f-2ca2-2584-f424-e83c56de5b2e 7822456f-2ca2-2584-f424-e83c56de5b2e}"
    testing.go:149: server-52e9f313-d0a2-1b7c-0541-bb792953c801: 2024-01-08T00:31:08.554Z [INFO]  server-52e9f313-d0a2-1b7c-0541-bb792953c801: pipelining replication: peer="{Voter server-7822456f-2ca2-2584-f424-e83c56de5b2e 7822456f-2ca2-2584-f424-e83c56de5b2e}"
testLoggerAdapter verbose: server-52e9f313-d0a2-1b7c-0541-bb792953c801: 2024-01-08T00:31:08.554Z [INFO]  server-52e9f313-d0a2-1b7c-0541-bb792953c801: pipelining replication: peer="{Voter server-b877f982-a861-9ef1-b421-e46f5abdff30 b877f982-a861-9ef1-b421-e46f5abdff30}"
    testing.go:149: server-52e9f313-d0a2-1b7c-0541-bb792953c801: 2024-01-08T00:31:08.554Z [INFO]  server-52e9f313-d0a2-1b7c-0541-bb792953c801: pipelining replication: peer="{Voter server-b877f982-a861-9ef1-b421-e46f5abdff30 b877f982-a861-9ef1-b421-e46f5abdff30}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:08.659Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:08.554337352 +0000 UTC m=+14.601226401 (1 nodes), 89.041993ms from start of poll, 89.61124ms from cluster start. Timeout at 2024-01-08 00:31:08.659486325 +0000 UTC m=+14.706375357, 105.148956ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:08.659Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:08.554337352 +0000 UTC m=+14.601226401 (1 nodes), 89.041993ms from start of poll, 89.61124ms from cluster start. Timeout at 2024-01-08 00:31:08.659486325 +0000 UTC m=+14.706375357, 105.148956ms after stability
testLoggerAdapter verbose: server-52e9f313-d0a2-1b7c-0541-bb792953c801: 2024-01-08T00:31:08.674Z [INFO]  server-52e9f313-d0a2-1b7c-0541-bb792953c801: aborting pipeline replication: peer="{Voter server-b877f982-a861-9ef1-b421-e46f5abdff30 b877f982-a861-9ef1-b421-e46f5abdff30}"
    testing.go:149: server-52e9f313-d0a2-1b7c-0541-bb792953c801: 2024-01-08T00:31:08.674Z [INFO]  server-52e9f313-d0a2-1b7c-0541-bb792953c801: aborting pipeline replication: peer="{Voter server-b877f982-a861-9ef1-b421-e46f5abdff30 b877f982-a861-9ef1-b421-e46f5abdff30}"
testLoggerAdapter verbose: server-52e9f313-d0a2-1b7c-0541-bb792953c801: 2024-01-08T00:31:08.674Z [INFO]  server-52e9f313-d0a2-1b7c-0541-bb792953c801: aborting pipeline replication: peer="{Voter server-7822456f-2ca2-2584-f424-e83c56de5b2e 7822456f-2ca2-2584-f424-e83c56de5b2e}"
    testing.go:149: server-52e9f313-d0a2-1b7c-0541-bb792953c801: 2024-01-08T00:31:08.674Z [INFO]  server-52e9f313-d0a2-1b7c-0541-bb792953c801: aborting pipeline replication: peer="{Voter server-7822456f-2ca2-2584-f424-e83c56de5b2e 7822456f-2ca2-2584-f424-e83c56de5b2e}"
--- PASS: TestRaft_Barrier (0.21s)
=== RUN   TestRaft_VerifyLeader
testLoggerAdapter verbose: server-188a475d-6dd8-002c-4d94-2cbadbb1b86c: 2024-01-08T00:31:08.675Z [INFO]  server-188a475d-6dd8-002c-4d94-2cbadbb1b86c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-188a475d-6dd8-002c-4d94-2cbadbb1b86c Address:188a475d-6dd8-002c-4d94-2cbadbb1b86c} {Suffrage:Voter ID:server-f3e81a4e-2bee-4fd5-059a-5d2946712dc9 Address:f3e81a4e-2bee-4fd5-059a-5d2946712dc9} {Suffrage:Voter ID:server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff Address:f4e4e770-9a99-7a80-38b6-ff9dbe8637ff}]"
    testing.go:149: server-188a475d-6dd8-002c-4d94-2cbadbb1b86c: 2024-01-08T00:31:08.675Z [INFO]  server-188a475d-6dd8-002c-4d94-2cbadbb1b86c: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-188a475d-6dd8-002c-4d94-2cbadbb1b86c Address:188a475d-6dd8-002c-4d94-2cbadbb1b86c} {Suffrage:Voter ID:server-f3e81a4e-2bee-4fd5-059a-5d2946712dc9 Address:f3e81a4e-2bee-4fd5-059a-5d2946712dc9} {Suffrage:Voter ID:server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff Address:f4e4e770-9a99-7a80-38b6-ff9dbe8637ff}]"
testLoggerAdapter verbose: server-f3e81a4e-2bee-4fd5-059a-5d2946712dc9: 2024-01-08T00:31:08.675Z [INFO]  server-f3e81a4e-2bee-4fd5-059a-5d2946712dc9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-188a475d-6dd8-002c-4d94-2cbadbb1b86c Address:188a475d-6dd8-002c-4d94-2cbadbb1b86c} {Suffrage:Voter ID:server-f3e81a4e-2bee-4fd5-059a-5d2946712dc9 Address:f3e81a4e-2bee-4fd5-059a-5d2946712dc9} {Suffrage:Voter ID:server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff Address:f4e4e770-9a99-7a80-38b6-ff9dbe8637ff}]"
    testing.go:149: server-f3e81a4e-2bee-4fd5-059a-5d2946712dc9: 2024-01-08T00:31:08.675Z [INFO]  server-f3e81a4e-2bee-4fd5-059a-5d2946712dc9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-188a475d-6dd8-002c-4d94-2cbadbb1b86c Address:188a475d-6dd8-002c-4d94-2cbadbb1b86c} {Suffrage:Voter ID:server-f3e81a4e-2bee-4fd5-059a-5d2946712dc9 Address:f3e81a4e-2bee-4fd5-059a-5d2946712dc9} {Suffrage:Voter ID:server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff Address:f4e4e770-9a99-7a80-38b6-ff9dbe8637ff}]"
testLoggerAdapter verbose: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.675Z [INFO]  server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-188a475d-6dd8-002c-4d94-2cbadbb1b86c Address:188a475d-6dd8-002c-4d94-2cbadbb1b86c} {Suffrage:Voter ID:server-f3e81a4e-2bee-4fd5-059a-5d2946712dc9 Address:f3e81a4e-2bee-4fd5-059a-5d2946712dc9} {Suffrage:Voter ID:server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff Address:f4e4e770-9a99-7a80-38b6-ff9dbe8637ff}]"
    testing.go:149: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.675Z [INFO]  server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-188a475d-6dd8-002c-4d94-2cbadbb1b86c Address:188a475d-6dd8-002c-4d94-2cbadbb1b86c} {Suffrage:Voter ID:server-f3e81a4e-2bee-4fd5-059a-5d2946712dc9 Address:f3e81a4e-2bee-4fd5-059a-5d2946712dc9} {Suffrage:Voter ID:server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff Address:f4e4e770-9a99-7a80-38b6-ff9dbe8637ff}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:08.675Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:08.675Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-188a475d-6dd8-002c-4d94-2cbadbb1b86c: 2024-01-08T00:31:08.675Z [INFO]  server-188a475d-6dd8-002c-4d94-2cbadbb1b86c: entering follower state: follower="Node at 188a475d-6dd8-002c-4d94-2cbadbb1b86c [Follower]" leader=
    testing.go:149: server-188a475d-6dd8-002c-4d94-2cbadbb1b86c: 2024-01-08T00:31:08.675Z [INFO]  server-188a475d-6dd8-002c-4d94-2cbadbb1b86c: entering follower state: follower="Node at 188a475d-6dd8-002c-4d94-2cbadbb1b86c [Follower]" leader=
testLoggerAdapter verbose: server-f3e81a4e-2bee-4fd5-059a-5d2946712dc9: 2024-01-08T00:31:08.675Z [INFO]  server-f3e81a4e-2bee-4fd5-059a-5d2946712dc9: entering follower state: follower="Node at f3e81a4e-2bee-4fd5-059a-5d2946712dc9 [Follower]" leader=
    testing.go:149: server-f3e81a4e-2bee-4fd5-059a-5d2946712dc9: 2024-01-08T00:31:08.675Z [INFO]  server-f3e81a4e-2bee-4fd5-059a-5d2946712dc9: entering follower state: follower="Node at f3e81a4e-2bee-4fd5-059a-5d2946712dc9 [Follower]" leader=
testLoggerAdapter verbose: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.675Z [INFO]  server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: entering follower state: follower="Node at f4e4e770-9a99-7a80-38b6-ff9dbe8637ff [Follower]" leader=
    testing.go:149: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.675Z [INFO]  server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: entering follower state: follower="Node at f4e4e770-9a99-7a80-38b6-ff9dbe8637ff [Follower]" leader=
testLoggerAdapter verbose: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.735Z [WARN]  server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.735Z [WARN]  server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.735Z [INFO]  server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: entering candidate state: node="Node at f4e4e770-9a99-7a80-38b6-ff9dbe8637ff [Candidate]" term=2
    testing.go:149: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.735Z [INFO]  server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: entering candidate state: node="Node at f4e4e770-9a99-7a80-38b6-ff9dbe8637ff [Candidate]" term=2
testLoggerAdapter verbose: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.735Z [INFO]  server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: election won: tally=2
    testing.go:149: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.735Z [INFO]  server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: election won: tally=2
testLoggerAdapter verbose: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.735Z [INFO]  server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: entering leader state: leader="Node at f4e4e770-9a99-7a80-38b6-ff9dbe8637ff [Leader]"
    testing.go:149: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.735Z [INFO]  server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: entering leader state: leader="Node at f4e4e770-9a99-7a80-38b6-ff9dbe8637ff [Leader]"
testLoggerAdapter verbose: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.735Z [INFO]  server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: added peer, starting replication: peer=server-188a475d-6dd8-002c-4d94-2cbadbb1b86c
    testing.go:149: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.735Z [INFO]  server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: added peer, starting replication: peer=server-188a475d-6dd8-002c-4d94-2cbadbb1b86c
testLoggerAdapter verbose: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.736Z [INFO]  server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: added peer, starting replication: peer=server-f3e81a4e-2bee-4fd5-059a-5d2946712dc9
    testing.go:149: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.736Z [INFO]  server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: added peer, starting replication: peer=server-f3e81a4e-2bee-4fd5-059a-5d2946712dc9
testLoggerAdapter verbose: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.736Z [INFO]  server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: pipelining replication: peer="{Voter server-f3e81a4e-2bee-4fd5-059a-5d2946712dc9 f3e81a4e-2bee-4fd5-059a-5d2946712dc9}"
    testing.go:149: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.736Z [INFO]  server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: pipelining replication: peer="{Voter server-f3e81a4e-2bee-4fd5-059a-5d2946712dc9 f3e81a4e-2bee-4fd5-059a-5d2946712dc9}"
testLoggerAdapter verbose: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.736Z [INFO]  server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: pipelining replication: peer="{Voter server-188a475d-6dd8-002c-4d94-2cbadbb1b86c 188a475d-6dd8-002c-4d94-2cbadbb1b86c}"
    testing.go:149: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.736Z [INFO]  server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: pipelining replication: peer="{Voter server-188a475d-6dd8-002c-4d94-2cbadbb1b86c 188a475d-6dd8-002c-4d94-2cbadbb1b86c}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:08.841Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:08.736276817 +0000 UTC m=+14.783165863 (1 nodes), 60.651818ms from start of poll, 61.035272ms from cluster start. Timeout at 2024-01-08 00:31:08.841319059 +0000 UTC m=+14.888208159, 105.042296ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:08.841Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:08.736276817 +0000 UTC m=+14.783165863 (1 nodes), 60.651818ms from start of poll, 61.035272ms from cluster start. Timeout at 2024-01-08 00:31:08.841319059 +0000 UTC m=+14.888208159, 105.042296ms after stability
testLoggerAdapter verbose: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.841Z [INFO]  server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: aborting pipeline replication: peer="{Voter server-188a475d-6dd8-002c-4d94-2cbadbb1b86c 188a475d-6dd8-002c-4d94-2cbadbb1b86c}"
    testing.go:149: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.841Z [INFO]  server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: aborting pipeline replication: peer="{Voter server-188a475d-6dd8-002c-4d94-2cbadbb1b86c 188a475d-6dd8-002c-4d94-2cbadbb1b86c}"
testLoggerAdapter verbose: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.841Z [INFO]  server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: aborting pipeline replication: peer="{Voter server-f3e81a4e-2bee-4fd5-059a-5d2946712dc9 f3e81a4e-2bee-4fd5-059a-5d2946712dc9}"
    testing.go:149: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.841Z [INFO]  server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: aborting pipeline replication: peer="{Voter server-f3e81a4e-2bee-4fd5-059a-5d2946712dc9 f3e81a4e-2bee-4fd5-059a-5d2946712dc9}"
testLoggerAdapter verbose: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.892Z [ERROR] server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: failed to heartbeat to: peer=188a475d-6dd8-002c-4d94-2cbadbb1b86c error="command timed out"
    testing.go:149: server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: 2024-01-08T00:31:08.892Z [ERROR] server-f4e4e770-9a99-7a80-38b6-ff9dbe8637ff: failed to heartbeat to: peer=188a475d-6dd8-002c-4d94-2cbadbb1b86c error="command timed out"
--- PASS: TestRaft_VerifyLeader (0.22s)
=== RUN   TestRaft_VerifyLeader_Single
testLoggerAdapter verbose: server-6cca274f-8eff-4373-bae8-4a685a54ea06: 2024-01-08T00:31:08.893Z [INFO]  server-6cca274f-8eff-4373-bae8-4a685a54ea06: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6cca274f-8eff-4373-bae8-4a685a54ea06 Address:6cca274f-8eff-4373-bae8-4a685a54ea06}]"
    testing.go:149: server-6cca274f-8eff-4373-bae8-4a685a54ea06: 2024-01-08T00:31:08.893Z [INFO]  server-6cca274f-8eff-4373-bae8-4a685a54ea06: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6cca274f-8eff-4373-bae8-4a685a54ea06 Address:6cca274f-8eff-4373-bae8-4a685a54ea06}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:08.893Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:08.893Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-6cca274f-8eff-4373-bae8-4a685a54ea06: 2024-01-08T00:31:08.893Z [INFO]  server-6cca274f-8eff-4373-bae8-4a685a54ea06: entering follower state: follower="Node at 6cca274f-8eff-4373-bae8-4a685a54ea06 [Follower]" leader=
    testing.go:149: server-6cca274f-8eff-4373-bae8-4a685a54ea06: 2024-01-08T00:31:08.893Z [INFO]  server-6cca274f-8eff-4373-bae8-4a685a54ea06: entering follower state: follower="Node at 6cca274f-8eff-4373-bae8-4a685a54ea06 [Follower]" leader=
testLoggerAdapter verbose: server-6cca274f-8eff-4373-bae8-4a685a54ea06: 2024-01-08T00:31:08.990Z [WARN]  server-6cca274f-8eff-4373-bae8-4a685a54ea06: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-6cca274f-8eff-4373-bae8-4a685a54ea06: 2024-01-08T00:31:08.990Z [WARN]  server-6cca274f-8eff-4373-bae8-4a685a54ea06: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-6cca274f-8eff-4373-bae8-4a685a54ea06: 2024-01-08T00:31:08.990Z [INFO]  server-6cca274f-8eff-4373-bae8-4a685a54ea06: entering candidate state: node="Node at 6cca274f-8eff-4373-bae8-4a685a54ea06 [Candidate]" term=2
    testing.go:149: server-6cca274f-8eff-4373-bae8-4a685a54ea06: 2024-01-08T00:31:08.990Z [INFO]  server-6cca274f-8eff-4373-bae8-4a685a54ea06: entering candidate state: node="Node at 6cca274f-8eff-4373-bae8-4a685a54ea06 [Candidate]" term=2
testLoggerAdapter verbose: server-6cca274f-8eff-4373-bae8-4a685a54ea06: 2024-01-08T00:31:08.990Z [INFO]  server-6cca274f-8eff-4373-bae8-4a685a54ea06: election won: tally=1
    testing.go:149: server-6cca274f-8eff-4373-bae8-4a685a54ea06: 2024-01-08T00:31:08.990Z [INFO]  server-6cca274f-8eff-4373-bae8-4a685a54ea06: election won: tally=1
testLoggerAdapter verbose: server-6cca274f-8eff-4373-bae8-4a685a54ea06: 2024-01-08T00:31:08.990Z [INFO]  server-6cca274f-8eff-4373-bae8-4a685a54ea06: entering leader state: leader="Node at 6cca274f-8eff-4373-bae8-4a685a54ea06 [Leader]"
    testing.go:149: server-6cca274f-8eff-4373-bae8-4a685a54ea06: 2024-01-08T00:31:08.990Z [INFO]  server-6cca274f-8eff-4373-bae8-4a685a54ea06: entering leader state: leader="Node at 6cca274f-8eff-4373-bae8-4a685a54ea06 [Leader]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:09.097Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:08.990954156 +0000 UTC m=+15.037843199 (1 nodes), 97.734594ms from start of poll, 97.943889ms from cluster start. Timeout at 2024-01-08 00:31:09.096958909 +0000 UTC m=+15.143848004, 106.004805ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:09.097Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:08.990954156 +0000 UTC m=+15.037843199 (1 nodes), 97.734594ms from start of poll, 97.943889ms from cluster start. Timeout at 2024-01-08 00:31:09.096958909 +0000 UTC m=+15.143848004, 106.004805ms after stability
--- PASS: TestRaft_VerifyLeader_Single (0.20s)
=== RUN   TestRaft_VerifyLeader_Fail
testLoggerAdapter verbose: server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: 2024-01-08T00:31:09.097Z [INFO]  server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1 Address:03f1d90c-c4d6-545b-5b55-dee0fcac31b1} {Suffrage:Voter ID:server-06abc411-bf4b-1321-e69e-5851116249c5 Address:06abc411-bf4b-1321-e69e-5851116249c5}]"
    testing.go:149: server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: 2024-01-08T00:31:09.097Z [INFO]  server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1 Address:03f1d90c-c4d6-545b-5b55-dee0fcac31b1} {Suffrage:Voter ID:server-06abc411-bf4b-1321-e69e-5851116249c5 Address:06abc411-bf4b-1321-e69e-5851116249c5}]"
testLoggerAdapter verbose: server-06abc411-bf4b-1321-e69e-5851116249c5: 2024-01-08T00:31:09.097Z [INFO]  server-06abc411-bf4b-1321-e69e-5851116249c5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1 Address:03f1d90c-c4d6-545b-5b55-dee0fcac31b1} {Suffrage:Voter ID:server-06abc411-bf4b-1321-e69e-5851116249c5 Address:06abc411-bf4b-1321-e69e-5851116249c5}]"
    testing.go:149: server-06abc411-bf4b-1321-e69e-5851116249c5: 2024-01-08T00:31:09.097Z [INFO]  server-06abc411-bf4b-1321-e69e-5851116249c5: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1 Address:03f1d90c-c4d6-545b-5b55-dee0fcac31b1} {Suffrage:Voter ID:server-06abc411-bf4b-1321-e69e-5851116249c5 Address:06abc411-bf4b-1321-e69e-5851116249c5}]"
testLoggerAdapter verbose: server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: 2024-01-08T00:31:09.097Z [INFO]  server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: entering follower state: follower="Node at 03f1d90c-c4d6-545b-5b55-dee0fcac31b1 [Follower]" leader=
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:09.098Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: 2024-01-08T00:31:09.097Z [INFO]  server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: entering follower state: follower="Node at 03f1d90c-c4d6-545b-5b55-dee0fcac31b1 [Follower]" leader=
testLoggerAdapter verbose: server-06abc411-bf4b-1321-e69e-5851116249c5: 2024-01-08T00:31:09.098Z [INFO]  server-06abc411-bf4b-1321-e69e-5851116249c5: entering follower state: follower="Node at 06abc411-bf4b-1321-e69e-5851116249c5 [Follower]" leader=
    testing.go:149: server-06abc411-bf4b-1321-e69e-5851116249c5: 2024-01-08T00:31:09.098Z [INFO]  server-06abc411-bf4b-1321-e69e-5851116249c5: entering follower state: follower="Node at 06abc411-bf4b-1321-e69e-5851116249c5 [Follower]" leader=
    testing.go:149: cluster: 2024-01-08T00:31:09.098Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: 2024-01-08T00:31:09.153Z [WARN]  server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: 2024-01-08T00:31:09.153Z [WARN]  server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: 2024-01-08T00:31:09.153Z [INFO]  server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: entering candidate state: node="Node at 03f1d90c-c4d6-545b-5b55-dee0fcac31b1 [Candidate]" term=2
    testing.go:149: server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: 2024-01-08T00:31:09.153Z [INFO]  server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: entering candidate state: node="Node at 03f1d90c-c4d6-545b-5b55-dee0fcac31b1 [Candidate]" term=2
testLoggerAdapter verbose: server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: 2024-01-08T00:31:09.154Z [INFO]  server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: election won: tally=2
    testing.go:149: server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: 2024-01-08T00:31:09.154Z [INFO]  server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: election won: tally=2
testLoggerAdapter verbose: server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: 2024-01-08T00:31:09.154Z [INFO]  server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: entering leader state: leader="Node at 03f1d90c-c4d6-545b-5b55-dee0fcac31b1 [Leader]"
    testing.go:149: server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: 2024-01-08T00:31:09.154Z [INFO]  server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: entering leader state: leader="Node at 03f1d90c-c4d6-545b-5b55-dee0fcac31b1 [Leader]"
testLoggerAdapter verbose: server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: 2024-01-08T00:31:09.154Z [INFO]  server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: added peer, starting replication: peer=server-06abc411-bf4b-1321-e69e-5851116249c5
    testing.go:149: server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: 2024-01-08T00:31:09.154Z [INFO]  server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: added peer, starting replication: peer=server-06abc411-bf4b-1321-e69e-5851116249c5
testLoggerAdapter verbose: server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: 2024-01-08T00:31:09.154Z [INFO]  server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: pipelining replication: peer="{Voter server-06abc411-bf4b-1321-e69e-5851116249c5 06abc411-bf4b-1321-e69e-5851116249c5}"
    testing.go:149: server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: 2024-01-08T00:31:09.154Z [INFO]  server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: pipelining replication: peer="{Voter server-06abc411-bf4b-1321-e69e-5851116249c5 06abc411-bf4b-1321-e69e-5851116249c5}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:09.259Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:09.154274902 +0000 UTC m=+15.201163949 (1 nodes), 56.176291ms from start of poll, 56.550823ms from cluster start. Timeout at 2024-01-08 00:31:09.259765366 +0000 UTC m=+15.306654414, 105.490465ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:09.259Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:09.154274902 +0000 UTC m=+15.201163949 (1 nodes), 56.176291ms from start of poll, 56.550823ms from cluster start. Timeout at 2024-01-08 00:31:09.259765366 +0000 UTC m=+15.306654414, 105.490465ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:09.259Z [INFO]  starting stability test: raft-state=Follower
    testing.go:149: cluster: 2024-01-08T00:31:09.259Z [INFO]  starting stability test: raft-state=Follower
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:09.366Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:09.259905116 +0000 UTC m=+15.306794159 (1 nodes), 1.84µs from start of poll, 162.181033ms from cluster start. Timeout at 2024-01-08 00:31:09.365980386 +0000 UTC m=+15.412869751, 106.075592ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:09.366Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:09.259905116 +0000 UTC m=+15.306794159 (1 nodes), 1.84µs from start of poll, 162.181033ms from cluster start. Timeout at 2024-01-08 00:31:09.365980386 +0000 UTC m=+15.412869751, 106.075592ms after stability
testLoggerAdapter verbose: server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: 2024-01-08T00:31:09.366Z [WARN]  server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: new leader elected, stepping down
    testing.go:149: server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: 2024-01-08T00:31:09.366Z [WARN]  server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: new leader elected, stepping down
testLoggerAdapter verbose: server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: 2024-01-08T00:31:09.366Z [INFO]  server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: entering follower state: follower="Node at 03f1d90c-c4d6-545b-5b55-dee0fcac31b1 [Follower]" leader=
    testing.go:149: server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: 2024-01-08T00:31:09.366Z [INFO]  server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: entering follower state: follower="Node at 03f1d90c-c4d6-545b-5b55-dee0fcac31b1 [Follower]" leader=
testLoggerAdapter verbose: server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: 2024-01-08T00:31:09.366Z [INFO]  server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: aborting pipeline replication: peer="{Voter server-06abc411-bf4b-1321-e69e-5851116249c5 06abc411-bf4b-1321-e69e-5851116249c5}"
    testing.go:149: server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: 2024-01-08T00:31:09.366Z [INFO]  server-03f1d90c-c4d6-545b-5b55-dee0fcac31b1: aborting pipeline replication: peer="{Voter server-06abc411-bf4b-1321-e69e-5851116249c5 06abc411-bf4b-1321-e69e-5851116249c5}"
--- PASS: TestRaft_VerifyLeader_Fail (0.27s)
=== RUN   TestRaft_VerifyLeader_PartialConnect
testLoggerAdapter verbose: server-6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a: 2024-01-08T00:31:09.367Z [INFO]  server-6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a Address:6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a} {Suffrage:Voter ID:server-ffd8c2cb-99b3-4123-382d-5c3643aed726 Address:ffd8c2cb-99b3-4123-382d-5c3643aed726} {Suffrage:Voter ID:server-d15f7885-77e5-b765-8bfa-0d47db2d2caa Address:d15f7885-77e5-b765-8bfa-0d47db2d2caa}]"
    testing.go:149: server-6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a: 2024-01-08T00:31:09.367Z [INFO]  server-6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a Address:6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a} {Suffrage:Voter ID:server-ffd8c2cb-99b3-4123-382d-5c3643aed726 Address:ffd8c2cb-99b3-4123-382d-5c3643aed726} {Suffrage:Voter ID:server-d15f7885-77e5-b765-8bfa-0d47db2d2caa Address:d15f7885-77e5-b765-8bfa-0d47db2d2caa}]"
testLoggerAdapter verbose: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.367Z [INFO]  server-ffd8c2cb-99b3-4123-382d-5c3643aed726: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a Address:6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a} {Suffrage:Voter ID:server-ffd8c2cb-99b3-4123-382d-5c3643aed726 Address:ffd8c2cb-99b3-4123-382d-5c3643aed726} {Suffrage:Voter ID:server-d15f7885-77e5-b765-8bfa-0d47db2d2caa Address:d15f7885-77e5-b765-8bfa-0d47db2d2caa}]"
    testing.go:149: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.367Z [INFO]  server-ffd8c2cb-99b3-4123-382d-5c3643aed726: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a Address:6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a} {Suffrage:Voter ID:server-ffd8c2cb-99b3-4123-382d-5c3643aed726 Address:ffd8c2cb-99b3-4123-382d-5c3643aed726} {Suffrage:Voter ID:server-d15f7885-77e5-b765-8bfa-0d47db2d2caa Address:d15f7885-77e5-b765-8bfa-0d47db2d2caa}]"
testLoggerAdapter verbose: server-d15f7885-77e5-b765-8bfa-0d47db2d2caa: 2024-01-08T00:31:09.367Z [INFO]  server-d15f7885-77e5-b765-8bfa-0d47db2d2caa: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a Address:6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a} {Suffrage:Voter ID:server-ffd8c2cb-99b3-4123-382d-5c3643aed726 Address:ffd8c2cb-99b3-4123-382d-5c3643aed726} {Suffrage:Voter ID:server-d15f7885-77e5-b765-8bfa-0d47db2d2caa Address:d15f7885-77e5-b765-8bfa-0d47db2d2caa}]"
    testing.go:149: server-d15f7885-77e5-b765-8bfa-0d47db2d2caa: 2024-01-08T00:31:09.367Z [INFO]  server-d15f7885-77e5-b765-8bfa-0d47db2d2caa: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a Address:6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a} {Suffrage:Voter ID:server-ffd8c2cb-99b3-4123-382d-5c3643aed726 Address:ffd8c2cb-99b3-4123-382d-5c3643aed726} {Suffrage:Voter ID:server-d15f7885-77e5-b765-8bfa-0d47db2d2caa Address:d15f7885-77e5-b765-8bfa-0d47db2d2caa}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:09.367Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:09.367Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a: 2024-01-08T00:31:09.367Z [INFO]  server-6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a: entering follower state: follower="Node at 6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a [Follower]" leader=
    testing.go:149: server-6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a: 2024-01-08T00:31:09.367Z [INFO]  server-6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a: entering follower state: follower="Node at 6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a [Follower]" leader=
testLoggerAdapter verbose: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.367Z [INFO]  server-ffd8c2cb-99b3-4123-382d-5c3643aed726: entering follower state: follower="Node at ffd8c2cb-99b3-4123-382d-5c3643aed726 [Follower]" leader=
    testing.go:149: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.367Z [INFO]  server-ffd8c2cb-99b3-4123-382d-5c3643aed726: entering follower state: follower="Node at ffd8c2cb-99b3-4123-382d-5c3643aed726 [Follower]" leader=
testLoggerAdapter verbose: server-d15f7885-77e5-b765-8bfa-0d47db2d2caa: 2024-01-08T00:31:09.367Z [INFO]  server-d15f7885-77e5-b765-8bfa-0d47db2d2caa: entering follower state: follower="Node at d15f7885-77e5-b765-8bfa-0d47db2d2caa [Follower]" leader=
    testing.go:149: server-d15f7885-77e5-b765-8bfa-0d47db2d2caa: 2024-01-08T00:31:09.367Z [INFO]  server-d15f7885-77e5-b765-8bfa-0d47db2d2caa: entering follower state: follower="Node at d15f7885-77e5-b765-8bfa-0d47db2d2caa [Follower]" leader=
testLoggerAdapter verbose: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.420Z [WARN]  server-ffd8c2cb-99b3-4123-382d-5c3643aed726: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.420Z [WARN]  server-ffd8c2cb-99b3-4123-382d-5c3643aed726: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.420Z [INFO]  server-ffd8c2cb-99b3-4123-382d-5c3643aed726: entering candidate state: node="Node at ffd8c2cb-99b3-4123-382d-5c3643aed726 [Candidate]" term=2
    testing.go:149: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.420Z [INFO]  server-ffd8c2cb-99b3-4123-382d-5c3643aed726: entering candidate state: node="Node at ffd8c2cb-99b3-4123-382d-5c3643aed726 [Candidate]" term=2
testLoggerAdapter verbose: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.421Z [INFO]  server-ffd8c2cb-99b3-4123-382d-5c3643aed726: election won: tally=2
    testing.go:149: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.421Z [INFO]  server-ffd8c2cb-99b3-4123-382d-5c3643aed726: election won: tally=2
testLoggerAdapter verbose: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.421Z [INFO]  server-ffd8c2cb-99b3-4123-382d-5c3643aed726: entering leader state: leader="Node at ffd8c2cb-99b3-4123-382d-5c3643aed726 [Leader]"
    testing.go:149: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.421Z [INFO]  server-ffd8c2cb-99b3-4123-382d-5c3643aed726: entering leader state: leader="Node at ffd8c2cb-99b3-4123-382d-5c3643aed726 [Leader]"
testLoggerAdapter verbose: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.421Z [INFO]  server-ffd8c2cb-99b3-4123-382d-5c3643aed726: added peer, starting replication: peer=server-6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a
    testing.go:149: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.421Z [INFO]  server-ffd8c2cb-99b3-4123-382d-5c3643aed726: added peer, starting replication: peer=server-6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a
testLoggerAdapter verbose: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.421Z [INFO]  server-ffd8c2cb-99b3-4123-382d-5c3643aed726: added peer, starting replication: peer=server-d15f7885-77e5-b765-8bfa-0d47db2d2caa
    testing.go:149: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.421Z [INFO]  server-ffd8c2cb-99b3-4123-382d-5c3643aed726: added peer, starting replication: peer=server-d15f7885-77e5-b765-8bfa-0d47db2d2caa
testLoggerAdapter verbose: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.421Z [INFO]  server-ffd8c2cb-99b3-4123-382d-5c3643aed726: pipelining replication: peer="{Voter server-6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a 6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a}"
    testing.go:149: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.421Z [INFO]  server-ffd8c2cb-99b3-4123-382d-5c3643aed726: pipelining replication: peer="{Voter server-6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a 6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a}"
testLoggerAdapter verbose: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.421Z [INFO]  server-ffd8c2cb-99b3-4123-382d-5c3643aed726: pipelining replication: peer="{Voter server-d15f7885-77e5-b765-8bfa-0d47db2d2caa d15f7885-77e5-b765-8bfa-0d47db2d2caa}"
    testing.go:149: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.421Z [INFO]  server-ffd8c2cb-99b3-4123-382d-5c3643aed726: pipelining replication: peer="{Voter server-d15f7885-77e5-b765-8bfa-0d47db2d2caa d15f7885-77e5-b765-8bfa-0d47db2d2caa}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:09.527Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:09.421131217 +0000 UTC m=+15.468020365 (1 nodes), 53.248254ms from start of poll, 53.730583ms from cluster start. Timeout at 2024-01-08 00:31:09.527127422 +0000 UTC m=+15.574016577, 105.996212ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:09.527Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:09.421131217 +0000 UTC m=+15.468020365 (1 nodes), 53.248254ms from start of poll, 53.730583ms from cluster start. Timeout at 2024-01-08 00:31:09.527127422 +0000 UTC m=+15.574016577, 105.996212ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:09.532Z [INFO]  starting stability test: raft-state=Follower
    testing.go:149: cluster: 2024-01-08T00:31:09.532Z [INFO]  starting stability test: raft-state=Follower
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:09.638Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:09.532928484 +0000 UTC m=+15.579817630 (2 nodes), 2.52µs from start of poll, 165.527848ms from cluster start. Timeout at 2024-01-08 00:31:09.638814989 +0000 UTC m=+15.685704150, 105.88652ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:09.638Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:09.532928484 +0000 UTC m=+15.579817630 (2 nodes), 2.52µs from start of poll, 165.527848ms from cluster start. Timeout at 2024-01-08 00:31:09.638814989 +0000 UTC m=+15.685704150, 105.88652ms after stability
    raft_test.go:1647: [INFO] Disconnecting Node at 6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a [Follower]
testLoggerAdapter verbose: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.639Z [ERROR] server-ffd8c2cb-99b3-4123-382d-5c3643aed726: failed to heartbeat to: peer=6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a error="failed to connect to peer: 6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a"
    testing.go:149: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.639Z [ERROR] server-ffd8c2cb-99b3-4123-382d-5c3643aed726: failed to heartbeat to: peer=6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a error="failed to connect to peer: 6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a"
testLoggerAdapter verbose: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.639Z [INFO]  server-ffd8c2cb-99b3-4123-382d-5c3643aed726: aborting pipeline replication: peer="{Voter server-d15f7885-77e5-b765-8bfa-0d47db2d2caa d15f7885-77e5-b765-8bfa-0d47db2d2caa}"
    testing.go:149: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.639Z [INFO]  server-ffd8c2cb-99b3-4123-382d-5c3643aed726: aborting pipeline replication: peer="{Voter server-d15f7885-77e5-b765-8bfa-0d47db2d2caa d15f7885-77e5-b765-8bfa-0d47db2d2caa}"
testLoggerAdapter verbose: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.639Z [INFO]  server-ffd8c2cb-99b3-4123-382d-5c3643aed726: aborting pipeline replication: peer="{Voter server-6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a 6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a}"
    testing.go:149: server-ffd8c2cb-99b3-4123-382d-5c3643aed726: 2024-01-08T00:31:09.639Z [INFO]  server-ffd8c2cb-99b3-4123-382d-5c3643aed726: aborting pipeline replication: peer="{Voter server-6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a 6f4e87c3-88bf-b8c0-c46b-bd2b0b39475a}"
--- PASS: TestRaft_VerifyLeader_PartialConnect (0.27s)
=== RUN   TestRaft_StartAsLeader
testLoggerAdapter verbose: server-efe4450a-1b1c-9bc6-d9d8-4215e6db3736: 2024-01-08T00:31:09.640Z [INFO]  server-efe4450a-1b1c-9bc6-d9d8-4215e6db3736: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-efe4450a-1b1c-9bc6-d9d8-4215e6db3736 Address:efe4450a-1b1c-9bc6-d9d8-4215e6db3736}]"
    testing.go:149: server-efe4450a-1b1c-9bc6-d9d8-4215e6db3736: 2024-01-08T00:31:09.640Z [INFO]  server-efe4450a-1b1c-9bc6-d9d8-4215e6db3736: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-efe4450a-1b1c-9bc6-d9d8-4215e6db3736 Address:efe4450a-1b1c-9bc6-d9d8-4215e6db3736}]"
testLoggerAdapter verbose: server-efe4450a-1b1c-9bc6-d9d8-4215e6db3736: 2024-01-08T00:31:09.640Z [INFO]  server-efe4450a-1b1c-9bc6-d9d8-4215e6db3736: entering leader state: leader="Node at efe4450a-1b1c-9bc6-d9d8-4215e6db3736 [Leader]"
    testing.go:149: server-efe4450a-1b1c-9bc6-d9d8-4215e6db3736: 2024-01-08T00:31:09.640Z [INFO]  server-efe4450a-1b1c-9bc6-d9d8-4215e6db3736: entering leader state: leader="Node at efe4450a-1b1c-9bc6-d9d8-4215e6db3736 [Leader]"
--- PASS: TestRaft_StartAsLeader (0.00s)
=== RUN   TestRaft_NotifyCh
testLoggerAdapter verbose: server-34e697e9-b19b-e0f5-b353-004d99039ae9: 2024-01-08T00:31:09.640Z [INFO]  server-34e697e9-b19b-e0f5-b353-004d99039ae9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-34e697e9-b19b-e0f5-b353-004d99039ae9 Address:34e697e9-b19b-e0f5-b353-004d99039ae9}]"
    testing.go:149: server-34e697e9-b19b-e0f5-b353-004d99039ae9: 2024-01-08T00:31:09.640Z [INFO]  server-34e697e9-b19b-e0f5-b353-004d99039ae9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-34e697e9-b19b-e0f5-b353-004d99039ae9 Address:34e697e9-b19b-e0f5-b353-004d99039ae9}]"
testLoggerAdapter verbose: server-34e697e9-b19b-e0f5-b353-004d99039ae9: 2024-01-08T00:31:09.640Z [INFO]  server-34e697e9-b19b-e0f5-b353-004d99039ae9: entering follower state: follower="Node at 34e697e9-b19b-e0f5-b353-004d99039ae9 [Follower]" leader=
    testing.go:149: server-34e697e9-b19b-e0f5-b353-004d99039ae9: 2024-01-08T00:31:09.640Z [INFO]  server-34e697e9-b19b-e0f5-b353-004d99039ae9: entering follower state: follower="Node at 34e697e9-b19b-e0f5-b353-004d99039ae9 [Follower]" leader=
testLoggerAdapter verbose: server-34e697e9-b19b-e0f5-b353-004d99039ae9: 2024-01-08T00:31:09.721Z [WARN]  server-34e697e9-b19b-e0f5-b353-004d99039ae9: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-34e697e9-b19b-e0f5-b353-004d99039ae9: 2024-01-08T00:31:09.721Z [WARN]  server-34e697e9-b19b-e0f5-b353-004d99039ae9: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-34e697e9-b19b-e0f5-b353-004d99039ae9: 2024-01-08T00:31:09.721Z [INFO]  server-34e697e9-b19b-e0f5-b353-004d99039ae9: entering candidate state: node="Node at 34e697e9-b19b-e0f5-b353-004d99039ae9 [Candidate]" term=2
    testing.go:149: server-34e697e9-b19b-e0f5-b353-004d99039ae9: 2024-01-08T00:31:09.721Z [INFO]  server-34e697e9-b19b-e0f5-b353-004d99039ae9: entering candidate state: node="Node at 34e697e9-b19b-e0f5-b353-004d99039ae9 [Candidate]" term=2
testLoggerAdapter verbose: server-34e697e9-b19b-e0f5-b353-004d99039ae9: 2024-01-08T00:31:09.721Z [INFO]  server-34e697e9-b19b-e0f5-b353-004d99039ae9: election won: tally=1
    testing.go:149: server-34e697e9-b19b-e0f5-b353-004d99039ae9: 2024-01-08T00:31:09.721Z [INFO]  server-34e697e9-b19b-e0f5-b353-004d99039ae9: election won: tally=1
testLoggerAdapter verbose: server-34e697e9-b19b-e0f5-b353-004d99039ae9: 2024-01-08T00:31:09.721Z [INFO]  server-34e697e9-b19b-e0f5-b353-004d99039ae9: entering leader state: leader="Node at 34e697e9-b19b-e0f5-b353-004d99039ae9 [Leader]"
    testing.go:149: server-34e697e9-b19b-e0f5-b353-004d99039ae9: 2024-01-08T00:31:09.721Z [INFO]  server-34e697e9-b19b-e0f5-b353-004d99039ae9: entering leader state: leader="Node at 34e697e9-b19b-e0f5-b353-004d99039ae9 [Leader]"
--- PASS: TestRaft_NotifyCh (0.08s)
=== RUN   TestRaft_Voting
testLoggerAdapter verbose: server-f1529f8c-c92a-34e8-eab1-fd91c6ec97b8: 2024-01-08T00:31:09.722Z [INFO]  server-f1529f8c-c92a-34e8-eab1-fd91c6ec97b8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f1529f8c-c92a-34e8-eab1-fd91c6ec97b8 Address:f1529f8c-c92a-34e8-eab1-fd91c6ec97b8} {Suffrage:Voter ID:server-c10dc359-caf1-e611-5c61-3ef4559792d0 Address:c10dc359-caf1-e611-5c61-3ef4559792d0} {Suffrage:Voter ID:server-329dec1e-6988-8a18-7460-fa0ada9e7bcd Address:329dec1e-6988-8a18-7460-fa0ada9e7bcd}]"
    testing.go:149: server-f1529f8c-c92a-34e8-eab1-fd91c6ec97b8: 2024-01-08T00:31:09.722Z [INFO]  server-f1529f8c-c92a-34e8-eab1-fd91c6ec97b8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f1529f8c-c92a-34e8-eab1-fd91c6ec97b8 Address:f1529f8c-c92a-34e8-eab1-fd91c6ec97b8} {Suffrage:Voter ID:server-c10dc359-caf1-e611-5c61-3ef4559792d0 Address:c10dc359-caf1-e611-5c61-3ef4559792d0} {Suffrage:Voter ID:server-329dec1e-6988-8a18-7460-fa0ada9e7bcd Address:329dec1e-6988-8a18-7460-fa0ada9e7bcd}]"
testLoggerAdapter verbose: server-c10dc359-caf1-e611-5c61-3ef4559792d0: 2024-01-08T00:31:09.722Z [INFO]  server-c10dc359-caf1-e611-5c61-3ef4559792d0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f1529f8c-c92a-34e8-eab1-fd91c6ec97b8 Address:f1529f8c-c92a-34e8-eab1-fd91c6ec97b8} {Suffrage:Voter ID:server-c10dc359-caf1-e611-5c61-3ef4559792d0 Address:c10dc359-caf1-e611-5c61-3ef4559792d0} {Suffrage:Voter ID:server-329dec1e-6988-8a18-7460-fa0ada9e7bcd Address:329dec1e-6988-8a18-7460-fa0ada9e7bcd}]"
    testing.go:149: server-c10dc359-caf1-e611-5c61-3ef4559792d0: 2024-01-08T00:31:09.722Z [INFO]  server-c10dc359-caf1-e611-5c61-3ef4559792d0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f1529f8c-c92a-34e8-eab1-fd91c6ec97b8 Address:f1529f8c-c92a-34e8-eab1-fd91c6ec97b8} {Suffrage:Voter ID:server-c10dc359-caf1-e611-5c61-3ef4559792d0 Address:c10dc359-caf1-e611-5c61-3ef4559792d0} {Suffrage:Voter ID:server-329dec1e-6988-8a18-7460-fa0ada9e7bcd Address:329dec1e-6988-8a18-7460-fa0ada9e7bcd}]"
testLoggerAdapter verbose: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:09.723Z [INFO]  server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f1529f8c-c92a-34e8-eab1-fd91c6ec97b8 Address:f1529f8c-c92a-34e8-eab1-fd91c6ec97b8} {Suffrage:Voter ID:server-c10dc359-caf1-e611-5c61-3ef4559792d0 Address:c10dc359-caf1-e611-5c61-3ef4559792d0} {Suffrage:Voter ID:server-329dec1e-6988-8a18-7460-fa0ada9e7bcd Address:329dec1e-6988-8a18-7460-fa0ada9e7bcd}]"
    testing.go:149: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:09.723Z [INFO]  server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f1529f8c-c92a-34e8-eab1-fd91c6ec97b8 Address:f1529f8c-c92a-34e8-eab1-fd91c6ec97b8} {Suffrage:Voter ID:server-c10dc359-caf1-e611-5c61-3ef4559792d0 Address:c10dc359-caf1-e611-5c61-3ef4559792d0} {Suffrage:Voter ID:server-329dec1e-6988-8a18-7460-fa0ada9e7bcd Address:329dec1e-6988-8a18-7460-fa0ada9e7bcd}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:09.723Z [INFO]  starting stability test: raft-state=Follower
    testing.go:149: cluster: 2024-01-08T00:31:09.723Z [INFO]  starting stability test: raft-state=Follower
testLoggerAdapter verbose: server-f1529f8c-c92a-34e8-eab1-fd91c6ec97b8: 2024-01-08T00:31:09.723Z [INFO]  server-f1529f8c-c92a-34e8-eab1-fd91c6ec97b8: entering follower state: follower="Node at f1529f8c-c92a-34e8-eab1-fd91c6ec97b8 [Follower]" leader=
    testing.go:149: server-f1529f8c-c92a-34e8-eab1-fd91c6ec97b8: 2024-01-08T00:31:09.723Z [INFO]  server-f1529f8c-c92a-34e8-eab1-fd91c6ec97b8: entering follower state: follower="Node at f1529f8c-c92a-34e8-eab1-fd91c6ec97b8 [Follower]" leader=
testLoggerAdapter verbose: server-c10dc359-caf1-e611-5c61-3ef4559792d0: 2024-01-08T00:31:09.723Z [INFO]  server-c10dc359-caf1-e611-5c61-3ef4559792d0: entering follower state: follower="Node at c10dc359-caf1-e611-5c61-3ef4559792d0 [Follower]" leader=
    testing.go:149: server-c10dc359-caf1-e611-5c61-3ef4559792d0: 2024-01-08T00:31:09.723Z [INFO]  server-c10dc359-caf1-e611-5c61-3ef4559792d0: entering follower state: follower="Node at c10dc359-caf1-e611-5c61-3ef4559792d0 [Follower]" leader=
testLoggerAdapter verbose: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:09.723Z [INFO]  server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: entering follower state: follower="Node at 329dec1e-6988-8a18-7460-fa0ada9e7bcd [Follower]" leader=
    testing.go:149: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:09.723Z [INFO]  server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: entering follower state: follower="Node at 329dec1e-6988-8a18-7460-fa0ada9e7bcd [Follower]" leader=
testLoggerAdapter verbose: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:09.802Z [WARN]  server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:09.802Z [WARN]  server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:09.802Z [INFO]  server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: entering candidate state: node="Node at 329dec1e-6988-8a18-7460-fa0ada9e7bcd [Candidate]" term=2
    testing.go:149: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:09.802Z [INFO]  server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: entering candidate state: node="Node at 329dec1e-6988-8a18-7460-fa0ada9e7bcd [Candidate]" term=2
testLoggerAdapter verbose: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:09.803Z [INFO]  server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: election won: tally=2
    testing.go:149: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:09.803Z [INFO]  server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: election won: tally=2
testLoggerAdapter verbose: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:09.803Z [INFO]  server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: entering leader state: leader="Node at 329dec1e-6988-8a18-7460-fa0ada9e7bcd [Leader]"
    testing.go:149: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:09.803Z [INFO]  server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: entering leader state: leader="Node at 329dec1e-6988-8a18-7460-fa0ada9e7bcd [Leader]"
testLoggerAdapter verbose: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:09.803Z [INFO]  server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: added peer, starting replication: peer=server-f1529f8c-c92a-34e8-eab1-fd91c6ec97b8
    testing.go:149: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:09.803Z [INFO]  server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: added peer, starting replication: peer=server-f1529f8c-c92a-34e8-eab1-fd91c6ec97b8
testLoggerAdapter verbose: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:09.803Z [INFO]  server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: added peer, starting replication: peer=server-c10dc359-caf1-e611-5c61-3ef4559792d0
    testing.go:149: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:09.803Z [INFO]  server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: added peer, starting replication: peer=server-c10dc359-caf1-e611-5c61-3ef4559792d0
testLoggerAdapter verbose: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:09.803Z [INFO]  server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: pipelining replication: peer="{Voter server-c10dc359-caf1-e611-5c61-3ef4559792d0 c10dc359-caf1-e611-5c61-3ef4559792d0}"
    testing.go:149: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:09.803Z [INFO]  server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: pipelining replication: peer="{Voter server-c10dc359-caf1-e611-5c61-3ef4559792d0 c10dc359-caf1-e611-5c61-3ef4559792d0}"
testLoggerAdapter verbose: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:09.803Z [INFO]  server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: pipelining replication: peer="{Voter server-f1529f8c-c92a-34e8-eab1-fd91c6ec97b8 f1529f8c-c92a-34e8-eab1-fd91c6ec97b8}"
    testing.go:149: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:09.803Z [INFO]  server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: pipelining replication: peer="{Voter server-f1529f8c-c92a-34e8-eab1-fd91c6ec97b8 f1529f8c-c92a-34e8-eab1-fd91c6ec97b8}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:09.909Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:09.803439447 +0000 UTC m=+15.850328499 (2 nodes), 80.283947ms from start of poll, 80.652183ms from cluster start. Timeout at 2024-01-08 00:31:09.909083411 +0000 UTC m=+15.955972557, 105.644058ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:09.909Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:09.803439447 +0000 UTC m=+15.850328499 (2 nodes), 80.283947ms from start of poll, 80.652183ms from cluster start. Timeout at 2024-01-08 00:31:09.909083411 +0000 UTC m=+15.955972557, 105.644058ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:09.909Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:09.909Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:10.014Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:09.909228198 +0000 UTC m=+15.956117339 (1 nodes), 2.273µs from start of poll, 186.441023ms from cluster start. Timeout at 2024-01-08 00:31:10.014647857 +0000 UTC m=+16.061536899, 105.41956ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:10.014Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:09.909228198 +0000 UTC m=+15.956117339 (1 nodes), 2.273µs from start of poll, 186.441023ms from cluster start. Timeout at 2024-01-08 00:31:10.014647857 +0000 UTC m=+16.061536899, 105.41956ms after stability
testLoggerAdapter verbose: server-c10dc359-caf1-e611-5c61-3ef4559792d0: 2024-01-08T00:31:10.014Z [WARN]  server-c10dc359-caf1-e611-5c61-3ef4559792d0: rejecting vote request since we have a leader: from=f1529f8c-c92a-34e8-eab1-fd91c6ec97b8 leader=329dec1e-6988-8a18-7460-fa0ada9e7bcd
    testing.go:149: server-c10dc359-caf1-e611-5c61-3ef4559792d0: 2024-01-08T00:31:10.014Z [WARN]  server-c10dc359-caf1-e611-5c61-3ef4559792d0: rejecting vote request since we have a leader: from=f1529f8c-c92a-34e8-eab1-fd91c6ec97b8 leader=329dec1e-6988-8a18-7460-fa0ada9e7bcd
testLoggerAdapter verbose: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:10.014Z [INFO]  server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: aborting pipeline replication: peer="{Voter server-c10dc359-caf1-e611-5c61-3ef4559792d0 c10dc359-caf1-e611-5c61-3ef4559792d0}"
    testing.go:149: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:10.014Z [INFO]  server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: aborting pipeline replication: peer="{Voter server-c10dc359-caf1-e611-5c61-3ef4559792d0 c10dc359-caf1-e611-5c61-3ef4559792d0}"
testLoggerAdapter verbose: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:10.014Z [INFO]  server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: aborting pipeline replication: peer="{Voter server-f1529f8c-c92a-34e8-eab1-fd91c6ec97b8 f1529f8c-c92a-34e8-eab1-fd91c6ec97b8}"
    testing.go:149: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:10.014Z [INFO]  server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: aborting pipeline replication: peer="{Voter server-f1529f8c-c92a-34e8-eab1-fd91c6ec97b8 f1529f8c-c92a-34e8-eab1-fd91c6ec97b8}"
testLoggerAdapter verbose: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:10.065Z [ERROR] server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: failed to heartbeat to: peer=c10dc359-caf1-e611-5c61-3ef4559792d0 error="command timed out"
    testing.go:149: server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: 2024-01-08T00:31:10.065Z [ERROR] server-329dec1e-6988-8a18-7460-fa0ada9e7bcd: failed to heartbeat to: peer=c10dc359-caf1-e611-5c61-3ef4559792d0 error="command timed out"
--- PASS: TestRaft_Voting (0.34s)
=== RUN   TestRaft_ProtocolVersion_RejectRPC
testLoggerAdapter verbose: server-8bc78e41-10f1-c37a-9251-fc6d2aeb45c0: 2024-01-08T00:31:10.066Z [INFO]  server-8bc78e41-10f1-c37a-9251-fc6d2aeb45c0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8bc78e41-10f1-c37a-9251-fc6d2aeb45c0 Address:8bc78e41-10f1-c37a-9251-fc6d2aeb45c0} {Suffrage:Voter ID:server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122 Address:50419f4d-0d2e-80ad-c6de-5f74e0f3b122} {Suffrage:Voter ID:server-8236ebae-d6f0-77f0-856c-ceaecf9995a9 Address:8236ebae-d6f0-77f0-856c-ceaecf9995a9}]"
    testing.go:149: server-8bc78e41-10f1-c37a-9251-fc6d2aeb45c0: 2024-01-08T00:31:10.066Z [INFO]  server-8bc78e41-10f1-c37a-9251-fc6d2aeb45c0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8bc78e41-10f1-c37a-9251-fc6d2aeb45c0 Address:8bc78e41-10f1-c37a-9251-fc6d2aeb45c0} {Suffrage:Voter ID:server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122 Address:50419f4d-0d2e-80ad-c6de-5f74e0f3b122} {Suffrage:Voter ID:server-8236ebae-d6f0-77f0-856c-ceaecf9995a9 Address:8236ebae-d6f0-77f0-856c-ceaecf9995a9}]"
testLoggerAdapter verbose: server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: 2024-01-08T00:31:10.066Z [INFO]  server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8bc78e41-10f1-c37a-9251-fc6d2aeb45c0 Address:8bc78e41-10f1-c37a-9251-fc6d2aeb45c0} {Suffrage:Voter ID:server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122 Address:50419f4d-0d2e-80ad-c6de-5f74e0f3b122} {Suffrage:Voter ID:server-8236ebae-d6f0-77f0-856c-ceaecf9995a9 Address:8236ebae-d6f0-77f0-856c-ceaecf9995a9}]"
    testing.go:149: server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: 2024-01-08T00:31:10.066Z [INFO]  server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8bc78e41-10f1-c37a-9251-fc6d2aeb45c0 Address:8bc78e41-10f1-c37a-9251-fc6d2aeb45c0} {Suffrage:Voter ID:server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122 Address:50419f4d-0d2e-80ad-c6de-5f74e0f3b122} {Suffrage:Voter ID:server-8236ebae-d6f0-77f0-856c-ceaecf9995a9 Address:8236ebae-d6f0-77f0-856c-ceaecf9995a9}]"
testLoggerAdapter verbose: server-8236ebae-d6f0-77f0-856c-ceaecf9995a9: 2024-01-08T00:31:10.066Z [INFO]  server-8236ebae-d6f0-77f0-856c-ceaecf9995a9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8bc78e41-10f1-c37a-9251-fc6d2aeb45c0 Address:8bc78e41-10f1-c37a-9251-fc6d2aeb45c0} {Suffrage:Voter ID:server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122 Address:50419f4d-0d2e-80ad-c6de-5f74e0f3b122} {Suffrage:Voter ID:server-8236ebae-d6f0-77f0-856c-ceaecf9995a9 Address:8236ebae-d6f0-77f0-856c-ceaecf9995a9}]"
    testing.go:149: server-8236ebae-d6f0-77f0-856c-ceaecf9995a9: 2024-01-08T00:31:10.066Z [INFO]  server-8236ebae-d6f0-77f0-856c-ceaecf9995a9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-8bc78e41-10f1-c37a-9251-fc6d2aeb45c0 Address:8bc78e41-10f1-c37a-9251-fc6d2aeb45c0} {Suffrage:Voter ID:server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122 Address:50419f4d-0d2e-80ad-c6de-5f74e0f3b122} {Suffrage:Voter ID:server-8236ebae-d6f0-77f0-856c-ceaecf9995a9 Address:8236ebae-d6f0-77f0-856c-ceaecf9995a9}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:10.066Z [INFO]  starting stability test: raft-state=Follower
    testing.go:149: cluster: 2024-01-08T00:31:10.066Z [INFO]  starting stability test: raft-state=Follower
testLoggerAdapter verbose: server-8bc78e41-10f1-c37a-9251-fc6d2aeb45c0: 2024-01-08T00:31:10.066Z [INFO]  server-8bc78e41-10f1-c37a-9251-fc6d2aeb45c0: entering follower state: follower="Node at 8bc78e41-10f1-c37a-9251-fc6d2aeb45c0 [Follower]" leader=
    testing.go:149: server-8bc78e41-10f1-c37a-9251-fc6d2aeb45c0: 2024-01-08T00:31:10.066Z [INFO]  server-8bc78e41-10f1-c37a-9251-fc6d2aeb45c0: entering follower state: follower="Node at 8bc78e41-10f1-c37a-9251-fc6d2aeb45c0 [Follower]" leader=
testLoggerAdapter verbose: server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: 2024-01-08T00:31:10.066Z [INFO]  server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: entering follower state: follower="Node at 50419f4d-0d2e-80ad-c6de-5f74e0f3b122 [Follower]" leader=
    testing.go:149: server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: 2024-01-08T00:31:10.066Z [INFO]  server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: entering follower state: follower="Node at 50419f4d-0d2e-80ad-c6de-5f74e0f3b122 [Follower]" leader=
testLoggerAdapter verbose: server-8236ebae-d6f0-77f0-856c-ceaecf9995a9: 2024-01-08T00:31:10.066Z [INFO]  server-8236ebae-d6f0-77f0-856c-ceaecf9995a9: entering follower state: follower="Node at 8236ebae-d6f0-77f0-856c-ceaecf9995a9 [Follower]" leader=
    testing.go:149: server-8236ebae-d6f0-77f0-856c-ceaecf9995a9: 2024-01-08T00:31:10.066Z [INFO]  server-8236ebae-d6f0-77f0-856c-ceaecf9995a9: entering follower state: follower="Node at 8236ebae-d6f0-77f0-856c-ceaecf9995a9 [Follower]" leader=
testLoggerAdapter verbose: server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: 2024-01-08T00:31:10.129Z [WARN]  server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: 2024-01-08T00:31:10.129Z [WARN]  server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: 2024-01-08T00:31:10.130Z [INFO]  server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: entering candidate state: node="Node at 50419f4d-0d2e-80ad-c6de-5f74e0f3b122 [Candidate]" term=2
    testing.go:149: server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: 2024-01-08T00:31:10.130Z [INFO]  server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: entering candidate state: node="Node at 50419f4d-0d2e-80ad-c6de-5f74e0f3b122 [Candidate]" term=2
testLoggerAdapter verbose: server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: 2024-01-08T00:31:10.130Z [INFO]  server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: election won: tally=2
    testing.go:149: server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: 2024-01-08T00:31:10.130Z [INFO]  server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: election won: tally=2
testLoggerAdapter verbose: server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: 2024-01-08T00:31:10.130Z [INFO]  server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: entering leader state: leader="Node at 50419f4d-0d2e-80ad-c6de-5f74e0f3b122 [Leader]"
    testing.go:149: server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: 2024-01-08T00:31:10.130Z [INFO]  server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: entering leader state: leader="Node at 50419f4d-0d2e-80ad-c6de-5f74e0f3b122 [Leader]"
testLoggerAdapter verbose: server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: 2024-01-08T00:31:10.130Z [INFO]  server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: added peer, starting replication: peer=server-8bc78e41-10f1-c37a-9251-fc6d2aeb45c0
    testing.go:149: server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: 2024-01-08T00:31:10.130Z [INFO]  server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: added peer, starting replication: peer=server-8bc78e41-10f1-c37a-9251-fc6d2aeb45c0
testLoggerAdapter verbose: server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: 2024-01-08T00:31:10.130Z [INFO]  server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: added peer, starting replication: peer=server-8236ebae-d6f0-77f0-856c-ceaecf9995a9
    testing.go:149: server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: 2024-01-08T00:31:10.130Z [INFO]  server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: added peer, starting replication: peer=server-8236ebae-d6f0-77f0-856c-ceaecf9995a9
testLoggerAdapter verbose: server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: 2024-01-08T00:31:10.130Z [INFO]  server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: pipelining replication: peer="{Voter server-8236ebae-d6f0-77f0-856c-ceaecf9995a9 8236ebae-d6f0-77f0-856c-ceaecf9995a9}"
    testing.go:149: server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: 2024-01-08T00:31:10.130Z [INFO]  server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: pipelining replication: peer="{Voter server-8236ebae-d6f0-77f0-856c-ceaecf9995a9 8236ebae-d6f0-77f0-856c-ceaecf9995a9}"
testLoggerAdapter verbose: server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: 2024-01-08T00:31:10.130Z [INFO]  server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: pipelining replication: peer="{Voter server-8bc78e41-10f1-c37a-9251-fc6d2aeb45c0 8bc78e41-10f1-c37a-9251-fc6d2aeb45c0}"
    testing.go:149: server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: 2024-01-08T00:31:10.130Z [INFO]  server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: pipelining replication: peer="{Voter server-8bc78e41-10f1-c37a-9251-fc6d2aeb45c0 8bc78e41-10f1-c37a-9251-fc6d2aeb45c0}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:10.235Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:10.130442448 +0000 UTC m=+16.177331502 (2 nodes), 63.598614ms from start of poll, 64.064199ms from cluster start. Timeout at 2024-01-08 00:31:10.235870757 +0000 UTC m=+16.282759847, 105.428345ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:10.235Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:10.130442448 +0000 UTC m=+16.177331502 (2 nodes), 63.598614ms from start of poll, 64.064199ms from cluster start. Timeout at 2024-01-08 00:31:10.235870757 +0000 UTC m=+16.282759847, 105.428345ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:10.235Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:10.235Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:10.341Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:10.236015264 +0000 UTC m=+16.282904320 (1 nodes), 1.97µs from start of poll, 169.637017ms from cluster start. Timeout at 2024-01-08 00:31:10.341323374 +0000 UTC m=+16.388212552, 105.308232ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:10.341Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:10.236015264 +0000 UTC m=+16.282904320 (1 nodes), 1.97µs from start of poll, 169.637017ms from cluster start. Timeout at 2024-01-08 00:31:10.341323374 +0000 UTC m=+16.388212552, 105.308232ms after stability
testLoggerAdapter verbose: server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: 2024-01-08T00:31:10.341Z [INFO]  server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: aborting pipeline replication: peer="{Voter server-8bc78e41-10f1-c37a-9251-fc6d2aeb45c0 8bc78e41-10f1-c37a-9251-fc6d2aeb45c0}"
    testing.go:149: server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: 2024-01-08T00:31:10.341Z [INFO]  server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: aborting pipeline replication: peer="{Voter server-8bc78e41-10f1-c37a-9251-fc6d2aeb45c0 8bc78e41-10f1-c37a-9251-fc6d2aeb45c0}"
testLoggerAdapter verbose: server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: 2024-01-08T00:31:10.341Z [INFO]  server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: aborting pipeline replication: peer="{Voter server-8236ebae-d6f0-77f0-856c-ceaecf9995a9 8236ebae-d6f0-77f0-856c-ceaecf9995a9}"
    testing.go:149: server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: 2024-01-08T00:31:10.341Z [INFO]  server-50419f4d-0d2e-80ad-c6de-5f74e0f3b122: aborting pipeline replication: peer="{Voter server-8236ebae-d6f0-77f0-856c-ceaecf9995a9 8236ebae-d6f0-77f0-856c-ceaecf9995a9}"
--- PASS: TestRaft_ProtocolVersion_RejectRPC (0.28s)
=== RUN   TestRaft_ProtocolVersion_Upgrade_1_2
testLoggerAdapter verbose: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.342Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: initial configuration: index=1 servers="[{Suffrage:Voter ID:b9457696-ac28-8234-e302-d4d80c0ecbab Address:b9457696-ac28-8234-e302-d4d80c0ecbab} {Suffrage:Voter ID:a103ce52-02c2-7fc3-414c-c6f567929843 Address:a103ce52-02c2-7fc3-414c-c6f567929843}]"
    testing.go:149: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.342Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: initial configuration: index=1 servers="[{Suffrage:Voter ID:b9457696-ac28-8234-e302-d4d80c0ecbab Address:b9457696-ac28-8234-e302-d4d80c0ecbab} {Suffrage:Voter ID:a103ce52-02c2-7fc3-414c-c6f567929843 Address:a103ce52-02c2-7fc3-414c-c6f567929843}]"
testLoggerAdapter verbose: a103ce52-02c2-7fc3-414c-c6f567929843: 2024-01-08T00:31:10.342Z [INFO]  a103ce52-02c2-7fc3-414c-c6f567929843: initial configuration: index=1 servers="[{Suffrage:Voter ID:b9457696-ac28-8234-e302-d4d80c0ecbab Address:b9457696-ac28-8234-e302-d4d80c0ecbab} {Suffrage:Voter ID:a103ce52-02c2-7fc3-414c-c6f567929843 Address:a103ce52-02c2-7fc3-414c-c6f567929843}]"
    testing.go:149: a103ce52-02c2-7fc3-414c-c6f567929843: 2024-01-08T00:31:10.342Z [INFO]  a103ce52-02c2-7fc3-414c-c6f567929843: initial configuration: index=1 servers="[{Suffrage:Voter ID:b9457696-ac28-8234-e302-d4d80c0ecbab Address:b9457696-ac28-8234-e302-d4d80c0ecbab} {Suffrage:Voter ID:a103ce52-02c2-7fc3-414c-c6f567929843 Address:a103ce52-02c2-7fc3-414c-c6f567929843}]"
testLoggerAdapter verbose: 5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5: 2024-01-08T00:31:10.342Z [INFO]  5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5: initial configuration: index=0 servers=[]
    testing.go:149: 5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5: 2024-01-08T00:31:10.342Z [INFO]  5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5: initial configuration: index=0 servers=[]
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:10.342Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:10.342Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.343Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: entering follower state: follower="Node at b9457696-ac28-8234-e302-d4d80c0ecbab [Follower]" leader=
    testing.go:149: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.343Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: entering follower state: follower="Node at b9457696-ac28-8234-e302-d4d80c0ecbab [Follower]" leader=
testLoggerAdapter verbose: a103ce52-02c2-7fc3-414c-c6f567929843: 2024-01-08T00:31:10.343Z [INFO]  a103ce52-02c2-7fc3-414c-c6f567929843: entering follower state: follower="Node at a103ce52-02c2-7fc3-414c-c6f567929843 [Follower]" leader=
    testing.go:149: a103ce52-02c2-7fc3-414c-c6f567929843: 2024-01-08T00:31:10.343Z [INFO]  a103ce52-02c2-7fc3-414c-c6f567929843: entering follower state: follower="Node at a103ce52-02c2-7fc3-414c-c6f567929843 [Follower]" leader=
testLoggerAdapter verbose: 5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5: 2024-01-08T00:31:10.343Z [INFO]  5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5: entering follower state: follower="Node at 5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5 [Follower]" leader=
    testing.go:149: 5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5: 2024-01-08T00:31:10.343Z [INFO]  5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5: entering follower state: follower="Node at 5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5 [Follower]" leader=
testLoggerAdapter verbose: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.398Z [WARN]  b9457696-ac28-8234-e302-d4d80c0ecbab: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.398Z [WARN]  b9457696-ac28-8234-e302-d4d80c0ecbab: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.398Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: entering candidate state: node="Node at b9457696-ac28-8234-e302-d4d80c0ecbab [Candidate]" term=2
    testing.go:149: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.398Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: entering candidate state: node="Node at b9457696-ac28-8234-e302-d4d80c0ecbab [Candidate]" term=2
testLoggerAdapter verbose: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.398Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: election won: tally=2
    testing.go:149: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.398Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: election won: tally=2
testLoggerAdapter verbose: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.399Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: entering leader state: leader="Node at b9457696-ac28-8234-e302-d4d80c0ecbab [Leader]"
    testing.go:149: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.399Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: entering leader state: leader="Node at b9457696-ac28-8234-e302-d4d80c0ecbab [Leader]"
testLoggerAdapter verbose: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.399Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: added peer, starting replication: peer=a103ce52-02c2-7fc3-414c-c6f567929843
    testing.go:149: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.399Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: added peer, starting replication: peer=a103ce52-02c2-7fc3-414c-c6f567929843
testLoggerAdapter verbose: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.399Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: pipelining replication: peer="{Voter a103ce52-02c2-7fc3-414c-c6f567929843 a103ce52-02c2-7fc3-414c-c6f567929843}"
    testing.go:149: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.399Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: pipelining replication: peer="{Voter a103ce52-02c2-7fc3-414c-c6f567929843 a103ce52-02c2-7fc3-414c-c6f567929843}"
testLoggerAdapter verbose: 5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5: 2024-01-08T00:31:10.434Z [WARN]  5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5: no known peers, aborting election
    testing.go:149: 5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5: 2024-01-08T00:31:10.434Z [WARN]  5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5: no known peers, aborting election
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:10.504Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:10.399205932 +0000 UTC m=+16.446095073 (1 nodes), 56.210767ms from start of poll, 56.664996ms from cluster start. Timeout at 2024-01-08 00:31:10.504279951 +0000 UTC m=+16.551169081, 105.074008ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:10.504Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:10.399205932 +0000 UTC m=+16.446095073 (1 nodes), 56.210767ms from start of poll, 56.664996ms from cluster start. Timeout at 2024-01-08 00:31:10.504279951 +0000 UTC m=+16.551169081, 105.074008ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:10.504Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:10.504Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:10.610Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:10.504469733 +0000 UTC m=+16.551358873 (1 nodes), 2.172µs from start of poll, 161.928796ms from cluster start. Timeout at 2024-01-08 00:31:10.609961397 +0000 UTC m=+16.656850443, 105.49157ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:10.610Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:10.504469733 +0000 UTC m=+16.551358873 (1 nodes), 2.172µs from start of poll, 161.928796ms from cluster start. Timeout at 2024-01-08 00:31:10.609961397 +0000 UTC m=+16.656850443, 105.49157ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:10.610Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:10.610Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:10.715Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:10.610107923 +0000 UTC m=+16.656996976 (1 nodes), 2.233µs from start of poll, 267.566899ms from cluster start. Timeout at 2024-01-08 00:31:10.715696827 +0000 UTC m=+16.762585944, 105.588968ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:10.715Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:10.610107923 +0000 UTC m=+16.656996976 (1 nodes), 2.233µs from start of poll, 267.566899ms from cluster start. Timeout at 2024-01-08 00:31:10.715696827 +0000 UTC m=+16.762585944, 105.588968ms after stability
testLoggerAdapter verbose: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.715Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: updating configuration: command=AddStaging server-id=5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5 server-addr=5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5 servers="[{Suffrage:Voter ID:b9457696-ac28-8234-e302-d4d80c0ecbab Address:b9457696-ac28-8234-e302-d4d80c0ecbab} {Suffrage:Voter ID:a103ce52-02c2-7fc3-414c-c6f567929843 Address:a103ce52-02c2-7fc3-414c-c6f567929843} {Suffrage:Voter ID:5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5 Address:5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5}]"
    testing.go:149: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.715Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: updating configuration: command=AddStaging server-id=5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5 server-addr=5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5 servers="[{Suffrage:Voter ID:b9457696-ac28-8234-e302-d4d80c0ecbab Address:b9457696-ac28-8234-e302-d4d80c0ecbab} {Suffrage:Voter ID:a103ce52-02c2-7fc3-414c-c6f567929843 Address:a103ce52-02c2-7fc3-414c-c6f567929843} {Suffrage:Voter ID:5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5 Address:5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5}]"
testLoggerAdapter verbose: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.715Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: added peer, starting replication: peer=5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5
    testing.go:149: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.715Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: added peer, starting replication: peer=5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5
testLoggerAdapter verbose: 5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5: 2024-01-08T00:31:10.716Z [WARN]  5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5: failed to get previous log: previous-index=3 last-index=0 error="log not found"
    testing.go:149: 5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5: 2024-01-08T00:31:10.716Z [WARN]  5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5: failed to get previous log: previous-index=3 last-index=0 error="log not found"
testLoggerAdapter verbose: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.716Z [WARN]  b9457696-ac28-8234-e302-d4d80c0ecbab: appendEntries rejected, sending older logs: peer="{Voter 5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5 5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5}" next=1
    testing.go:149: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.716Z [WARN]  b9457696-ac28-8234-e302-d4d80c0ecbab: appendEntries rejected, sending older logs: peer="{Voter 5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5 5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5}" next=1
testLoggerAdapter verbose: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.716Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: pipelining replication: peer="{Voter 5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5 5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5}"
    testing.go:149: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.716Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: pipelining replication: peer="{Voter 5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5 5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:10.716Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:10.716Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:10.821Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:10.716318794 +0000 UTC m=+16.763207912 (1 nodes), 2.071µs from start of poll, 373.777835ms from cluster start. Timeout at 2024-01-08 00:31:10.821598467 +0000 UTC m=+16.868487631, 105.279719ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:10.821Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:10.716318794 +0000 UTC m=+16.763207912 (1 nodes), 2.071µs from start of poll, 373.777835ms from cluster start. Timeout at 2024-01-08 00:31:10.821598467 +0000 UTC m=+16.868487631, 105.279719ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:10.821Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:10.821Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:10.927Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:10.821766659 +0000 UTC m=+16.868655796 (1 nodes), 2.168µs from start of poll, 479.225719ms from cluster start. Timeout at 2024-01-08 00:31:10.927784329 +0000 UTC m=+16.974673372, 106.017576ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:10.927Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:10.821766659 +0000 UTC m=+16.868655796 (1 nodes), 2.168µs from start of poll, 479.225719ms from cluster start. Timeout at 2024-01-08 00:31:10.927784329 +0000 UTC m=+16.974673372, 106.017576ms after stability
testLoggerAdapter verbose: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.927Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: updating configuration: command=RemoveServer server-id=5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5 server-addr= servers="[{Suffrage:Voter ID:b9457696-ac28-8234-e302-d4d80c0ecbab Address:b9457696-ac28-8234-e302-d4d80c0ecbab} {Suffrage:Voter ID:a103ce52-02c2-7fc3-414c-c6f567929843 Address:a103ce52-02c2-7fc3-414c-c6f567929843}]"
    testing.go:149: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.927Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: updating configuration: command=RemoveServer server-id=5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5 server-addr= servers="[{Suffrage:Voter ID:b9457696-ac28-8234-e302-d4d80c0ecbab Address:b9457696-ac28-8234-e302-d4d80c0ecbab} {Suffrage:Voter ID:a103ce52-02c2-7fc3-414c-c6f567929843 Address:a103ce52-02c2-7fc3-414c-c6f567929843}]"
testLoggerAdapter verbose: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.928Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: removed peer, stopping replication: peer=5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5 last-index=4
    testing.go:149: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.928Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: removed peer, stopping replication: peer=5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5 last-index=4
testLoggerAdapter verbose: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.928Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: aborting pipeline replication: peer="{Voter 5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5 5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5}"
    testing.go:149: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.928Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: aborting pipeline replication: peer="{Voter 5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5 5e7010f2-f7be-ebb0-77fc-bfc3f13e34f5}"
testLoggerAdapter verbose: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.928Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: aborting pipeline replication: peer="{Voter a103ce52-02c2-7fc3-414c-c6f567929843 a103ce52-02c2-7fc3-414c-c6f567929843}"
    testing.go:149: b9457696-ac28-8234-e302-d4d80c0ecbab: 2024-01-08T00:31:10.928Z [INFO]  b9457696-ac28-8234-e302-d4d80c0ecbab: aborting pipeline replication: peer="{Voter a103ce52-02c2-7fc3-414c-c6f567929843 a103ce52-02c2-7fc3-414c-c6f567929843}"
--- PASS: TestRaft_ProtocolVersion_Upgrade_1_2 (0.59s)
=== RUN   TestRaft_ProtocolVersion_Upgrade_2_3
testLoggerAdapter verbose: 783def06-61e2-6284-e103-d42777c4e599: 2024-01-08T00:31:10.929Z [INFO]  783def06-61e2-6284-e103-d42777c4e599: initial configuration: index=1 servers="[{Suffrage:Voter ID:783def06-61e2-6284-e103-d42777c4e599 Address:783def06-61e2-6284-e103-d42777c4e599} {Suffrage:Voter ID:3a19eeea-b34d-9c5b-75ab-9592d167d11d Address:3a19eeea-b34d-9c5b-75ab-9592d167d11d}]"
    testing.go:149: 783def06-61e2-6284-e103-d42777c4e599: 2024-01-08T00:31:10.929Z [INFO]  783def06-61e2-6284-e103-d42777c4e599: initial configuration: index=1 servers="[{Suffrage:Voter ID:783def06-61e2-6284-e103-d42777c4e599 Address:783def06-61e2-6284-e103-d42777c4e599} {Suffrage:Voter ID:3a19eeea-b34d-9c5b-75ab-9592d167d11d Address:3a19eeea-b34d-9c5b-75ab-9592d167d11d}]"
testLoggerAdapter verbose: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:10.929Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: initial configuration: index=1 servers="[{Suffrage:Voter ID:783def06-61e2-6284-e103-d42777c4e599 Address:783def06-61e2-6284-e103-d42777c4e599} {Suffrage:Voter ID:3a19eeea-b34d-9c5b-75ab-9592d167d11d Address:3a19eeea-b34d-9c5b-75ab-9592d167d11d}]"
    testing.go:149: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:10.929Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: initial configuration: index=1 servers="[{Suffrage:Voter ID:783def06-61e2-6284-e103-d42777c4e599 Address:783def06-61e2-6284-e103-d42777c4e599} {Suffrage:Voter ID:3a19eeea-b34d-9c5b-75ab-9592d167d11d Address:3a19eeea-b34d-9c5b-75ab-9592d167d11d}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:10.929Z [INFO]  starting stability test: raft-state=Follower
    testing.go:149: cluster: 2024-01-08T00:31:10.929Z [INFO]  starting stability test: raft-state=Follower
testLoggerAdapter verbose: 783def06-61e2-6284-e103-d42777c4e599: 2024-01-08T00:31:10.929Z [INFO]  783def06-61e2-6284-e103-d42777c4e599: entering follower state: follower="Node at 783def06-61e2-6284-e103-d42777c4e599 [Follower]" leader=
    testing.go:149: 783def06-61e2-6284-e103-d42777c4e599: 2024-01-08T00:31:10.929Z [INFO]  783def06-61e2-6284-e103-d42777c4e599: entering follower state: follower="Node at 783def06-61e2-6284-e103-d42777c4e599 [Follower]" leader=
testLoggerAdapter verbose: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:10.929Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: entering follower state: follower="Node at 3a19eeea-b34d-9c5b-75ab-9592d167d11d [Follower]" leader=
    testing.go:149: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:10.929Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: entering follower state: follower="Node at 3a19eeea-b34d-9c5b-75ab-9592d167d11d [Follower]" leader=
testLoggerAdapter verbose: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:10.991Z [WARN]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:10.991Z [WARN]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:10.991Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: entering candidate state: node="Node at 3a19eeea-b34d-9c5b-75ab-9592d167d11d [Candidate]" term=2
    testing.go:149: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:10.991Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: entering candidate state: node="Node at 3a19eeea-b34d-9c5b-75ab-9592d167d11d [Candidate]" term=2
testLoggerAdapter verbose: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:10.991Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: election won: tally=2
    testing.go:149: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:10.991Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: election won: tally=2
testLoggerAdapter verbose: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:10.991Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: entering leader state: leader="Node at 3a19eeea-b34d-9c5b-75ab-9592d167d11d [Leader]"
    testing.go:149: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:10.991Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: entering leader state: leader="Node at 3a19eeea-b34d-9c5b-75ab-9592d167d11d [Leader]"
testLoggerAdapter verbose: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:10.991Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: added peer, starting replication: peer=783def06-61e2-6284-e103-d42777c4e599
    testing.go:149: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:10.991Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: added peer, starting replication: peer=783def06-61e2-6284-e103-d42777c4e599
testLoggerAdapter verbose: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:10.991Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: pipelining replication: peer="{Voter 783def06-61e2-6284-e103-d42777c4e599 783def06-61e2-6284-e103-d42777c4e599}"
    testing.go:149: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:10.991Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: pipelining replication: peer="{Voter 783def06-61e2-6284-e103-d42777c4e599 783def06-61e2-6284-e103-d42777c4e599}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:11.097Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:10.991966166 +0000 UTC m=+17.038855307 (1 nodes), 62.563749ms from start of poll, 62.783608ms from cluster start. Timeout at 2024-01-08 00:31:11.097106562 +0000 UTC m=+17.143995710, 105.140403ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:11.097Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:10.991966166 +0000 UTC m=+17.038855307 (1 nodes), 62.563749ms from start of poll, 62.783608ms from cluster start. Timeout at 2024-01-08 00:31:11.097106562 +0000 UTC m=+17.143995710, 105.140403ms after stability
testLoggerAdapter verbose: server-36a8cd05-eb25-d102-ba2d-fb1cdd102525: 2024-01-08T00:31:11.097Z [INFO]  server-36a8cd05-eb25-d102-ba2d-fb1cdd102525: initial configuration: index=0 servers=[]
    testing.go:149: server-36a8cd05-eb25-d102-ba2d-fb1cdd102525: 2024-01-08T00:31:11.097Z [INFO]  server-36a8cd05-eb25-d102-ba2d-fb1cdd102525: initial configuration: index=0 servers=[]
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:11.097Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:11.097Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-36a8cd05-eb25-d102-ba2d-fb1cdd102525: 2024-01-08T00:31:11.097Z [INFO]  server-36a8cd05-eb25-d102-ba2d-fb1cdd102525: entering follower state: follower="Node at 36a8cd05-eb25-d102-ba2d-fb1cdd102525 [Follower]" leader=
    testing.go:149: server-36a8cd05-eb25-d102-ba2d-fb1cdd102525: 2024-01-08T00:31:11.097Z [INFO]  server-36a8cd05-eb25-d102-ba2d-fb1cdd102525: entering follower state: follower="Node at 36a8cd05-eb25-d102-ba2d-fb1cdd102525 [Follower]" leader=
testLoggerAdapter verbose: server-36a8cd05-eb25-d102-ba2d-fb1cdd102525: 2024-01-08T00:31:11.196Z [WARN]  server-36a8cd05-eb25-d102-ba2d-fb1cdd102525: no known peers, aborting election
    testing.go:149: server-36a8cd05-eb25-d102-ba2d-fb1cdd102525: 2024-01-08T00:31:11.196Z [WARN]  server-36a8cd05-eb25-d102-ba2d-fb1cdd102525: no known peers, aborting election
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:11.203Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:11.097730621 +0000 UTC m=+17.144619758 (1 nodes), 2.014µs from start of poll, 168.548059ms from cluster start. Timeout at 2024-01-08 00:31:11.203093211 +0000 UTC m=+17.249982423, 105.362665ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:11.203Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:11.097730621 +0000 UTC m=+17.144619758 (1 nodes), 2.014µs from start of poll, 168.548059ms from cluster start. Timeout at 2024-01-08 00:31:11.203093211 +0000 UTC m=+17.249982423, 105.362665ms after stability
testLoggerAdapter verbose: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:11.203Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: updating configuration: command=AddStaging server-id=server-36a8cd05-eb25-d102-ba2d-fb1cdd102525 server-addr=36a8cd05-eb25-d102-ba2d-fb1cdd102525 servers="[{Suffrage:Voter ID:783def06-61e2-6284-e103-d42777c4e599 Address:783def06-61e2-6284-e103-d42777c4e599} {Suffrage:Voter ID:3a19eeea-b34d-9c5b-75ab-9592d167d11d Address:3a19eeea-b34d-9c5b-75ab-9592d167d11d} {Suffrage:Voter ID:server-36a8cd05-eb25-d102-ba2d-fb1cdd102525 Address:36a8cd05-eb25-d102-ba2d-fb1cdd102525}]"
    testing.go:149: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:11.203Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: updating configuration: command=AddStaging server-id=server-36a8cd05-eb25-d102-ba2d-fb1cdd102525 server-addr=36a8cd05-eb25-d102-ba2d-fb1cdd102525 servers="[{Suffrage:Voter ID:783def06-61e2-6284-e103-d42777c4e599 Address:783def06-61e2-6284-e103-d42777c4e599} {Suffrage:Voter ID:3a19eeea-b34d-9c5b-75ab-9592d167d11d Address:3a19eeea-b34d-9c5b-75ab-9592d167d11d} {Suffrage:Voter ID:server-36a8cd05-eb25-d102-ba2d-fb1cdd102525 Address:36a8cd05-eb25-d102-ba2d-fb1cdd102525}]"
testLoggerAdapter verbose: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:11.203Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: added peer, starting replication: peer=server-36a8cd05-eb25-d102-ba2d-fb1cdd102525
    testing.go:149: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:11.203Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: added peer, starting replication: peer=server-36a8cd05-eb25-d102-ba2d-fb1cdd102525
testLoggerAdapter verbose: server-36a8cd05-eb25-d102-ba2d-fb1cdd102525: 2024-01-08T00:31:11.203Z [WARN]  server-36a8cd05-eb25-d102-ba2d-fb1cdd102525: failed to get previous log: previous-index=3 last-index=0 error="log not found"
    testing.go:149: server-36a8cd05-eb25-d102-ba2d-fb1cdd102525: 2024-01-08T00:31:11.203Z [WARN]  server-36a8cd05-eb25-d102-ba2d-fb1cdd102525: failed to get previous log: previous-index=3 last-index=0 error="log not found"
testLoggerAdapter verbose: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:11.203Z [WARN]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: appendEntries rejected, sending older logs: peer="{Voter server-36a8cd05-eb25-d102-ba2d-fb1cdd102525 36a8cd05-eb25-d102-ba2d-fb1cdd102525}" next=1
    testing.go:149: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:11.203Z [WARN]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: appendEntries rejected, sending older logs: peer="{Voter server-36a8cd05-eb25-d102-ba2d-fb1cdd102525 36a8cd05-eb25-d102-ba2d-fb1cdd102525}" next=1
testLoggerAdapter verbose: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:11.203Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: pipelining replication: peer="{Voter server-36a8cd05-eb25-d102-ba2d-fb1cdd102525 36a8cd05-eb25-d102-ba2d-fb1cdd102525}"
    testing.go:149: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:11.203Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: pipelining replication: peer="{Voter server-36a8cd05-eb25-d102-ba2d-fb1cdd102525 36a8cd05-eb25-d102-ba2d-fb1cdd102525}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:11.208Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:11.208Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:11.314Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:11.208895789 +0000 UTC m=+17.255784943 (1 nodes), 2.474µs from start of poll, 279.713244ms from cluster start. Timeout at 2024-01-08 00:31:11.314468917 +0000 UTC m=+17.361358107, 105.573164ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:11.314Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:11.208895789 +0000 UTC m=+17.255784943 (1 nodes), 2.474µs from start of poll, 279.713244ms from cluster start. Timeout at 2024-01-08 00:31:11.314468917 +0000 UTC m=+17.361358107, 105.573164ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:11.314Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:11.314Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:11.420Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:11.314641551 +0000 UTC m=+17.361530644 (1 nodes), 2.044µs from start of poll, 385.458945ms from cluster start. Timeout at 2024-01-08 00:31:11.42021097 +0000 UTC m=+17.467100141, 105.569497ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:11.420Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:11.314641551 +0000 UTC m=+17.361530644 (1 nodes), 2.044µs from start of poll, 385.458945ms from cluster start. Timeout at 2024-01-08 00:31:11.42021097 +0000 UTC m=+17.467100141, 105.569497ms after stability
testLoggerAdapter verbose: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:11.420Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: updating configuration: command=RemoveServer server-id=783def06-61e2-6284-e103-d42777c4e599 server-addr= servers="[{Suffrage:Voter ID:3a19eeea-b34d-9c5b-75ab-9592d167d11d Address:3a19eeea-b34d-9c5b-75ab-9592d167d11d} {Suffrage:Voter ID:server-36a8cd05-eb25-d102-ba2d-fb1cdd102525 Address:36a8cd05-eb25-d102-ba2d-fb1cdd102525}]"
    testing.go:149: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:11.420Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: updating configuration: command=RemoveServer server-id=783def06-61e2-6284-e103-d42777c4e599 server-addr= servers="[{Suffrage:Voter ID:3a19eeea-b34d-9c5b-75ab-9592d167d11d Address:3a19eeea-b34d-9c5b-75ab-9592d167d11d} {Suffrage:Voter ID:server-36a8cd05-eb25-d102-ba2d-fb1cdd102525 Address:36a8cd05-eb25-d102-ba2d-fb1cdd102525}]"
testLoggerAdapter verbose: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:11.420Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: removed peer, stopping replication: peer=783def06-61e2-6284-e103-d42777c4e599 last-index=4
    testing.go:149: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:11.420Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: removed peer, stopping replication: peer=783def06-61e2-6284-e103-d42777c4e599 last-index=4
testLoggerAdapter verbose: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:11.420Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: aborting pipeline replication: peer="{Voter 783def06-61e2-6284-e103-d42777c4e599 783def06-61e2-6284-e103-d42777c4e599}"
    testing.go:149: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:11.420Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: aborting pipeline replication: peer="{Voter 783def06-61e2-6284-e103-d42777c4e599 783def06-61e2-6284-e103-d42777c4e599}"
testLoggerAdapter verbose: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:11.420Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: aborting pipeline replication: peer="{Voter server-36a8cd05-eb25-d102-ba2d-fb1cdd102525 36a8cd05-eb25-d102-ba2d-fb1cdd102525}"
    testing.go:149: 3a19eeea-b34d-9c5b-75ab-9592d167d11d: 2024-01-08T00:31:11.420Z [INFO]  3a19eeea-b34d-9c5b-75ab-9592d167d11d: aborting pipeline replication: peer="{Voter server-36a8cd05-eb25-d102-ba2d-fb1cdd102525 36a8cd05-eb25-d102-ba2d-fb1cdd102525}"
--- PASS: TestRaft_ProtocolVersion_Upgrade_2_3 (0.49s)
=== RUN   TestRaft_LeadershipTransferInProgress
--- PASS: TestRaft_LeadershipTransferInProgress (0.00s)
=== RUN   TestRaft_LeadershipTransferPickServer
--- PASS: TestRaft_LeadershipTransferPickServer (0.00s)
=== RUN   TestRaft_LeadershipTransfer
testLoggerAdapter verbose: server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: 2024-01-08T00:31:11.422Z [INFO]  server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3dbabc23-a375-d2ba-b0f8-902fa23b8958 Address:3dbabc23-a375-d2ba-b0f8-902fa23b8958} {Suffrage:Voter ID:server-031f68ac-1e78-0df1-54e4-4f634840d4d0 Address:031f68ac-1e78-0df1-54e4-4f634840d4d0} {Suffrage:Voter ID:server-52bb4c20-c73a-5f29-2c36-f49961f116a9 Address:52bb4c20-c73a-5f29-2c36-f49961f116a9}]"
    testing.go:149: server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: 2024-01-08T00:31:11.422Z [INFO]  server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3dbabc23-a375-d2ba-b0f8-902fa23b8958 Address:3dbabc23-a375-d2ba-b0f8-902fa23b8958} {Suffrage:Voter ID:server-031f68ac-1e78-0df1-54e4-4f634840d4d0 Address:031f68ac-1e78-0df1-54e4-4f634840d4d0} {Suffrage:Voter ID:server-52bb4c20-c73a-5f29-2c36-f49961f116a9 Address:52bb4c20-c73a-5f29-2c36-f49961f116a9}]"
testLoggerAdapter verbose: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.422Z [INFO]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3dbabc23-a375-d2ba-b0f8-902fa23b8958 Address:3dbabc23-a375-d2ba-b0f8-902fa23b8958} {Suffrage:Voter ID:server-031f68ac-1e78-0df1-54e4-4f634840d4d0 Address:031f68ac-1e78-0df1-54e4-4f634840d4d0} {Suffrage:Voter ID:server-52bb4c20-c73a-5f29-2c36-f49961f116a9 Address:52bb4c20-c73a-5f29-2c36-f49961f116a9}]"
    testing.go:149: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.422Z [INFO]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3dbabc23-a375-d2ba-b0f8-902fa23b8958 Address:3dbabc23-a375-d2ba-b0f8-902fa23b8958} {Suffrage:Voter ID:server-031f68ac-1e78-0df1-54e4-4f634840d4d0 Address:031f68ac-1e78-0df1-54e4-4f634840d4d0} {Suffrage:Voter ID:server-52bb4c20-c73a-5f29-2c36-f49961f116a9 Address:52bb4c20-c73a-5f29-2c36-f49961f116a9}]"
testLoggerAdapter verbose: server-52bb4c20-c73a-5f29-2c36-f49961f116a9: 2024-01-08T00:31:11.422Z [INFO]  server-52bb4c20-c73a-5f29-2c36-f49961f116a9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3dbabc23-a375-d2ba-b0f8-902fa23b8958 Address:3dbabc23-a375-d2ba-b0f8-902fa23b8958} {Suffrage:Voter ID:server-031f68ac-1e78-0df1-54e4-4f634840d4d0 Address:031f68ac-1e78-0df1-54e4-4f634840d4d0} {Suffrage:Voter ID:server-52bb4c20-c73a-5f29-2c36-f49961f116a9 Address:52bb4c20-c73a-5f29-2c36-f49961f116a9}]"
    testing.go:149: server-52bb4c20-c73a-5f29-2c36-f49961f116a9: 2024-01-08T00:31:11.422Z [INFO]  server-52bb4c20-c73a-5f29-2c36-f49961f116a9: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-3dbabc23-a375-d2ba-b0f8-902fa23b8958 Address:3dbabc23-a375-d2ba-b0f8-902fa23b8958} {Suffrage:Voter ID:server-031f68ac-1e78-0df1-54e4-4f634840d4d0 Address:031f68ac-1e78-0df1-54e4-4f634840d4d0} {Suffrage:Voter ID:server-52bb4c20-c73a-5f29-2c36-f49961f116a9 Address:52bb4c20-c73a-5f29-2c36-f49961f116a9}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:11.422Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:11.422Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: 2024-01-08T00:31:11.422Z [INFO]  server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: entering follower state: follower="Node at 3dbabc23-a375-d2ba-b0f8-902fa23b8958 [Follower]" leader=
    testing.go:149: server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: 2024-01-08T00:31:11.422Z [INFO]  server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: entering follower state: follower="Node at 3dbabc23-a375-d2ba-b0f8-902fa23b8958 [Follower]" leader=
testLoggerAdapter verbose: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.423Z [INFO]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: entering follower state: follower="Node at 031f68ac-1e78-0df1-54e4-4f634840d4d0 [Follower]" leader=
    testing.go:149: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.423Z [INFO]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: entering follower state: follower="Node at 031f68ac-1e78-0df1-54e4-4f634840d4d0 [Follower]" leader=
testLoggerAdapter verbose: server-52bb4c20-c73a-5f29-2c36-f49961f116a9: 2024-01-08T00:31:11.423Z [INFO]  server-52bb4c20-c73a-5f29-2c36-f49961f116a9: entering follower state: follower="Node at 52bb4c20-c73a-5f29-2c36-f49961f116a9 [Follower]" leader=
    testing.go:149: server-52bb4c20-c73a-5f29-2c36-f49961f116a9: 2024-01-08T00:31:11.423Z [INFO]  server-52bb4c20-c73a-5f29-2c36-f49961f116a9: entering follower state: follower="Node at 52bb4c20-c73a-5f29-2c36-f49961f116a9 [Follower]" leader=
testLoggerAdapter verbose: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.476Z [WARN]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.476Z [WARN]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.476Z [INFO]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: entering candidate state: node="Node at 031f68ac-1e78-0df1-54e4-4f634840d4d0 [Candidate]" term=2
    testing.go:149: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.476Z [INFO]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: entering candidate state: node="Node at 031f68ac-1e78-0df1-54e4-4f634840d4d0 [Candidate]" term=2
testLoggerAdapter verbose: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.476Z [INFO]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: election won: tally=2
    testing.go:149: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.476Z [INFO]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: election won: tally=2
testLoggerAdapter verbose: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.476Z [INFO]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: entering leader state: leader="Node at 031f68ac-1e78-0df1-54e4-4f634840d4d0 [Leader]"
    testing.go:149: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.476Z [INFO]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: entering leader state: leader="Node at 031f68ac-1e78-0df1-54e4-4f634840d4d0 [Leader]"
testLoggerAdapter verbose: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.476Z [INFO]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: added peer, starting replication: peer=server-3dbabc23-a375-d2ba-b0f8-902fa23b8958
    testing.go:149: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.476Z [INFO]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: added peer, starting replication: peer=server-3dbabc23-a375-d2ba-b0f8-902fa23b8958
testLoggerAdapter verbose: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.476Z [INFO]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: added peer, starting replication: peer=server-52bb4c20-c73a-5f29-2c36-f49961f116a9
    testing.go:149: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.476Z [INFO]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: added peer, starting replication: peer=server-52bb4c20-c73a-5f29-2c36-f49961f116a9
testLoggerAdapter verbose: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.477Z [INFO]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: pipelining replication: peer="{Voter server-52bb4c20-c73a-5f29-2c36-f49961f116a9 52bb4c20-c73a-5f29-2c36-f49961f116a9}"
    testing.go:149: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.477Z [INFO]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: pipelining replication: peer="{Voter server-52bb4c20-c73a-5f29-2c36-f49961f116a9 52bb4c20-c73a-5f29-2c36-f49961f116a9}"
testLoggerAdapter verbose: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.477Z [INFO]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: pipelining replication: peer="{Voter server-3dbabc23-a375-d2ba-b0f8-902fa23b8958 3dbabc23-a375-d2ba-b0f8-902fa23b8958}"
    testing.go:149: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.477Z [INFO]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: pipelining replication: peer="{Voter server-3dbabc23-a375-d2ba-b0f8-902fa23b8958 3dbabc23-a375-d2ba-b0f8-902fa23b8958}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:11.582Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:11.477420469 +0000 UTC m=+17.524309579 (1 nodes), 54.502469ms from start of poll, 55.074981ms from cluster start. Timeout at 2024-01-08 00:31:11.582750866 +0000 UTC m=+17.629640071, 105.330492ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:11.582Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:11.477420469 +0000 UTC m=+17.524309579 (1 nodes), 54.502469ms from start of poll, 55.074981ms from cluster start. Timeout at 2024-01-08 00:31:11.582750866 +0000 UTC m=+17.629640071, 105.330492ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:11.582Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:11.582Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:11.688Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:11.582907337 +0000 UTC m=+17.629796433 (1 nodes), 2.27µs from start of poll, 160.561835ms from cluster start. Timeout at 2024-01-08 00:31:11.688642576 +0000 UTC m=+17.735531675, 105.735242ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:11.688Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:11.582907337 +0000 UTC m=+17.629796433 (1 nodes), 2.27µs from start of poll, 160.561835ms from cluster start. Timeout at 2024-01-08 00:31:11.688642576 +0000 UTC m=+17.735531675, 105.735242ms after stability
testLoggerAdapter verbose: server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: 2024-01-08T00:31:11.688Z [INFO]  server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: entering candidate state: node="Node at 3dbabc23-a375-d2ba-b0f8-902fa23b8958 [Candidate]" term=3
    testing.go:149: server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: 2024-01-08T00:31:11.688Z [INFO]  server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: entering candidate state: node="Node at 3dbabc23-a375-d2ba-b0f8-902fa23b8958 [Candidate]" term=3
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:11.688Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:11.688Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: 2024-01-08T00:31:11.689Z [INFO]  server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: election won: tally=2
    testing.go:149: server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: 2024-01-08T00:31:11.689Z [INFO]  server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: election won: tally=2
testLoggerAdapter verbose: server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: 2024-01-08T00:31:11.689Z [INFO]  server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: entering leader state: leader="Node at 3dbabc23-a375-d2ba-b0f8-902fa23b8958 [Leader]"
testLoggerAdapter verbose: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.689Z [INFO]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: entering follower state: follower="Node at 031f68ac-1e78-0df1-54e4-4f634840d4d0 [Follower]" leader=
    testing.go:149: server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: 2024-01-08T00:31:11.689Z [INFO]  server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: entering leader state: leader="Node at 3dbabc23-a375-d2ba-b0f8-902fa23b8958 [Leader]"
testLoggerAdapter verbose: server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: 2024-01-08T00:31:11.689Z [INFO]  server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: added peer, starting replication: peer=server-031f68ac-1e78-0df1-54e4-4f634840d4d0
    testing.go:149: server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: 2024-01-08T00:31:11.689Z [INFO]  server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: added peer, starting replication: peer=server-031f68ac-1e78-0df1-54e4-4f634840d4d0
testLoggerAdapter verbose: server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: 2024-01-08T00:31:11.689Z [INFO]  server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: added peer, starting replication: peer=server-52bb4c20-c73a-5f29-2c36-f49961f116a9
    testing.go:149: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.689Z [INFO]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: entering follower state: follower="Node at 031f68ac-1e78-0df1-54e4-4f634840d4d0 [Follower]" leader=
    testing.go:149: server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: 2024-01-08T00:31:11.689Z [INFO]  server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: added peer, starting replication: peer=server-52bb4c20-c73a-5f29-2c36-f49961f116a9
testLoggerAdapter verbose: server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: 2024-01-08T00:31:11.689Z [INFO]  server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: pipelining replication: peer="{Voter server-031f68ac-1e78-0df1-54e4-4f634840d4d0 031f68ac-1e78-0df1-54e4-4f634840d4d0}"
    testing.go:149: server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: 2024-01-08T00:31:11.689Z [INFO]  server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: pipelining replication: peer="{Voter server-031f68ac-1e78-0df1-54e4-4f634840d4d0 031f68ac-1e78-0df1-54e4-4f634840d4d0}"
testLoggerAdapter verbose: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.689Z [INFO]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: aborting pipeline replication: peer="{Voter server-3dbabc23-a375-d2ba-b0f8-902fa23b8958 3dbabc23-a375-d2ba-b0f8-902fa23b8958}"
testLoggerAdapter verbose: server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: 2024-01-08T00:31:11.689Z [INFO]  server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: pipelining replication: peer="{Voter server-52bb4c20-c73a-5f29-2c36-f49961f116a9 52bb4c20-c73a-5f29-2c36-f49961f116a9}"
    testing.go:149: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.689Z [INFO]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: aborting pipeline replication: peer="{Voter server-3dbabc23-a375-d2ba-b0f8-902fa23b8958 3dbabc23-a375-d2ba-b0f8-902fa23b8958}"
testLoggerAdapter verbose: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.689Z [INFO]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: aborting pipeline replication: peer="{Voter server-52bb4c20-c73a-5f29-2c36-f49961f116a9 52bb4c20-c73a-5f29-2c36-f49961f116a9}"
    testing.go:149: server-031f68ac-1e78-0df1-54e4-4f634840d4d0: 2024-01-08T00:31:11.689Z [INFO]  server-031f68ac-1e78-0df1-54e4-4f634840d4d0: aborting pipeline replication: peer="{Voter server-52bb4c20-c73a-5f29-2c36-f49961f116a9 52bb4c20-c73a-5f29-2c36-f49961f116a9}"
    testing.go:149: server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: 2024-01-08T00:31:11.689Z [INFO]  server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: pipelining replication: peer="{Voter server-52bb4c20-c73a-5f29-2c36-f49961f116a9 52bb4c20-c73a-5f29-2c36-f49961f116a9}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:11.794Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:11.689150918 +0000 UTC m=+17.736040039 (1 nodes), 119.765µs from start of poll, 266.805441ms from cluster start. Timeout at 2024-01-08 00:31:11.794660931 +0000 UTC m=+17.841550033, 105.509994ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:11.794Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:11.689150918 +0000 UTC m=+17.736040039 (1 nodes), 119.765µs from start of poll, 266.805441ms from cluster start. Timeout at 2024-01-08 00:31:11.794660931 +0000 UTC m=+17.841550033, 105.509994ms after stability
testLoggerAdapter verbose: server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: 2024-01-08T00:31:11.794Z [INFO]  server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: aborting pipeline replication: peer="{Voter server-52bb4c20-c73a-5f29-2c36-f49961f116a9 52bb4c20-c73a-5f29-2c36-f49961f116a9}"
    testing.go:149: server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: 2024-01-08T00:31:11.794Z [INFO]  server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: aborting pipeline replication: peer="{Voter server-52bb4c20-c73a-5f29-2c36-f49961f116a9 52bb4c20-c73a-5f29-2c36-f49961f116a9}"
testLoggerAdapter verbose: server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: 2024-01-08T00:31:11.794Z [INFO]  server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: aborting pipeline replication: peer="{Voter server-031f68ac-1e78-0df1-54e4-4f634840d4d0 031f68ac-1e78-0df1-54e4-4f634840d4d0}"
    testing.go:149: server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: 2024-01-08T00:31:11.794Z [INFO]  server-3dbabc23-a375-d2ba-b0f8-902fa23b8958: aborting pipeline replication: peer="{Voter server-031f68ac-1e78-0df1-54e4-4f634840d4d0 031f68ac-1e78-0df1-54e4-4f634840d4d0}"
--- PASS: TestRaft_LeadershipTransfer (0.37s)
=== RUN   TestRaft_LeadershipTransferWithOneNode
testLoggerAdapter verbose: server-e3816327-22ab-a511-1bb4-bd211d644cc7: 2024-01-08T00:31:11.796Z [INFO]  server-e3816327-22ab-a511-1bb4-bd211d644cc7: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e3816327-22ab-a511-1bb4-bd211d644cc7 Address:e3816327-22ab-a511-1bb4-bd211d644cc7}]"
    testing.go:149: server-e3816327-22ab-a511-1bb4-bd211d644cc7: 2024-01-08T00:31:11.796Z [INFO]  server-e3816327-22ab-a511-1bb4-bd211d644cc7: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-e3816327-22ab-a511-1bb4-bd211d644cc7 Address:e3816327-22ab-a511-1bb4-bd211d644cc7}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:11.796Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:11.796Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-e3816327-22ab-a511-1bb4-bd211d644cc7: 2024-01-08T00:31:11.796Z [INFO]  server-e3816327-22ab-a511-1bb4-bd211d644cc7: entering follower state: follower="Node at e3816327-22ab-a511-1bb4-bd211d644cc7 [Follower]" leader=
    testing.go:149: server-e3816327-22ab-a511-1bb4-bd211d644cc7: 2024-01-08T00:31:11.796Z [INFO]  server-e3816327-22ab-a511-1bb4-bd211d644cc7: entering follower state: follower="Node at e3816327-22ab-a511-1bb4-bd211d644cc7 [Follower]" leader=
testLoggerAdapter verbose: server-e3816327-22ab-a511-1bb4-bd211d644cc7: 2024-01-08T00:31:11.873Z [WARN]  server-e3816327-22ab-a511-1bb4-bd211d644cc7: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-e3816327-22ab-a511-1bb4-bd211d644cc7: 2024-01-08T00:31:11.873Z [WARN]  server-e3816327-22ab-a511-1bb4-bd211d644cc7: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-e3816327-22ab-a511-1bb4-bd211d644cc7: 2024-01-08T00:31:11.873Z [INFO]  server-e3816327-22ab-a511-1bb4-bd211d644cc7: entering candidate state: node="Node at e3816327-22ab-a511-1bb4-bd211d644cc7 [Candidate]" term=2
    testing.go:149: server-e3816327-22ab-a511-1bb4-bd211d644cc7: 2024-01-08T00:31:11.873Z [INFO]  server-e3816327-22ab-a511-1bb4-bd211d644cc7: entering candidate state: node="Node at e3816327-22ab-a511-1bb4-bd211d644cc7 [Candidate]" term=2
testLoggerAdapter verbose: server-e3816327-22ab-a511-1bb4-bd211d644cc7: 2024-01-08T00:31:11.873Z [INFO]  server-e3816327-22ab-a511-1bb4-bd211d644cc7: election won: tally=1
    testing.go:149: server-e3816327-22ab-a511-1bb4-bd211d644cc7: 2024-01-08T00:31:11.873Z [INFO]  server-e3816327-22ab-a511-1bb4-bd211d644cc7: election won: tally=1
testLoggerAdapter verbose: server-e3816327-22ab-a511-1bb4-bd211d644cc7: 2024-01-08T00:31:11.873Z [INFO]  server-e3816327-22ab-a511-1bb4-bd211d644cc7: entering leader state: leader="Node at e3816327-22ab-a511-1bb4-bd211d644cc7 [Leader]"
    testing.go:149: server-e3816327-22ab-a511-1bb4-bd211d644cc7: 2024-01-08T00:31:11.873Z [INFO]  server-e3816327-22ab-a511-1bb4-bd211d644cc7: entering leader state: leader="Node at e3816327-22ab-a511-1bb4-bd211d644cc7 [Leader]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:11.978Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:11.873408787 +0000 UTC m=+17.920297987 (1 nodes), 77.166729ms from start of poll, 77.424167ms from cluster start. Timeout at 2024-01-08 00:31:11.978719871 +0000 UTC m=+18.025609160, 105.311173ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:11.978Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:11.873408787 +0000 UTC m=+17.920297987 (1 nodes), 77.166729ms from start of poll, 77.424167ms from cluster start. Timeout at 2024-01-08 00:31:11.978719871 +0000 UTC m=+18.025609160, 105.311173ms after stability
--- PASS: TestRaft_LeadershipTransferWithOneNode (0.18s)
=== RUN   TestRaft_LeadershipTransferWithSevenNodes
testLoggerAdapter verbose: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:11.980Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-164b7adb-6988-0e3c-a0e4-f6962a583cb3 Address:164b7adb-6988-0e3c-a0e4-f6962a583cb3} {Suffrage:Voter ID:server-98bf8875-a96c-1ab1-046c-136db591cd9d Address:98bf8875-a96c-1ab1-046c-136db591cd9d} {Suffrage:Voter ID:server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8 Address:28ff6f15-37c2-bfde-bfce-97bb80b2cac8} {Suffrage:Voter ID:server-d6eecb40-44a2-80ed-0099-f25daa7cb905 Address:d6eecb40-44a2-80ed-0099-f25daa7cb905} {Suffrage:Voter ID:server-674c7145-0e20-c026-98a5-1bbf859cf227 Address:674c7145-0e20-c026-98a5-1bbf859cf227} {Suffrage:Voter ID:server-e0ed200a-903b-96cd-1145-6098b46ee1cd Address:e0ed200a-903b-96cd-1145-6098b46ee1cd} {Suffrage:Voter ID:server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6 Address:7f4593f0-d11d-5c51-be28-fb05ca53dfd6}]"
    testing.go:149: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:11.980Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-164b7adb-6988-0e3c-a0e4-f6962a583cb3 Address:164b7adb-6988-0e3c-a0e4-f6962a583cb3} {Suffrage:Voter ID:server-98bf8875-a96c-1ab1-046c-136db591cd9d Address:98bf8875-a96c-1ab1-046c-136db591cd9d} {Suffrage:Voter ID:server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8 Address:28ff6f15-37c2-bfde-bfce-97bb80b2cac8} {Suffrage:Voter ID:server-d6eecb40-44a2-80ed-0099-f25daa7cb905 Address:d6eecb40-44a2-80ed-0099-f25daa7cb905} {Suffrage:Voter ID:server-674c7145-0e20-c026-98a5-1bbf859cf227 Address:674c7145-0e20-c026-98a5-1bbf859cf227} {Suffrage:Voter ID:server-e0ed200a-903b-96cd-1145-6098b46ee1cd Address:e0ed200a-903b-96cd-1145-6098b46ee1cd} {Suffrage:Voter ID:server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6 Address:7f4593f0-d11d-5c51-be28-fb05ca53dfd6}]"
testLoggerAdapter verbose: server-98bf8875-a96c-1ab1-046c-136db591cd9d: 2024-01-08T00:31:11.981Z [INFO]  server-98bf8875-a96c-1ab1-046c-136db591cd9d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-164b7adb-6988-0e3c-a0e4-f6962a583cb3 Address:164b7adb-6988-0e3c-a0e4-f6962a583cb3} {Suffrage:Voter ID:server-98bf8875-a96c-1ab1-046c-136db591cd9d Address:98bf8875-a96c-1ab1-046c-136db591cd9d} {Suffrage:Voter ID:server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8 Address:28ff6f15-37c2-bfde-bfce-97bb80b2cac8} {Suffrage:Voter ID:server-d6eecb40-44a2-80ed-0099-f25daa7cb905 Address:d6eecb40-44a2-80ed-0099-f25daa7cb905} {Suffrage:Voter ID:server-674c7145-0e20-c026-98a5-1bbf859cf227 Address:674c7145-0e20-c026-98a5-1bbf859cf227} {Suffrage:Voter ID:server-e0ed200a-903b-96cd-1145-6098b46ee1cd Address:e0ed200a-903b-96cd-1145-6098b46ee1cd} {Suffrage:Voter ID:server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6 Address:7f4593f0-d11d-5c51-be28-fb05ca53dfd6}]"
    testing.go:149: server-98bf8875-a96c-1ab1-046c-136db591cd9d: 2024-01-08T00:31:11.981Z [INFO]  server-98bf8875-a96c-1ab1-046c-136db591cd9d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-164b7adb-6988-0e3c-a0e4-f6962a583cb3 Address:164b7adb-6988-0e3c-a0e4-f6962a583cb3} {Suffrage:Voter ID:server-98bf8875-a96c-1ab1-046c-136db591cd9d Address:98bf8875-a96c-1ab1-046c-136db591cd9d} {Suffrage:Voter ID:server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8 Address:28ff6f15-37c2-bfde-bfce-97bb80b2cac8} {Suffrage:Voter ID:server-d6eecb40-44a2-80ed-0099-f25daa7cb905 Address:d6eecb40-44a2-80ed-0099-f25daa7cb905} {Suffrage:Voter ID:server-674c7145-0e20-c026-98a5-1bbf859cf227 Address:674c7145-0e20-c026-98a5-1bbf859cf227} {Suffrage:Voter ID:server-e0ed200a-903b-96cd-1145-6098b46ee1cd Address:e0ed200a-903b-96cd-1145-6098b46ee1cd} {Suffrage:Voter ID:server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6 Address:7f4593f0-d11d-5c51-be28-fb05ca53dfd6}]"
testLoggerAdapter verbose: server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8: 2024-01-08T00:31:11.981Z [INFO]  server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-164b7adb-6988-0e3c-a0e4-f6962a583cb3 Address:164b7adb-6988-0e3c-a0e4-f6962a583cb3} {Suffrage:Voter ID:server-98bf8875-a96c-1ab1-046c-136db591cd9d Address:98bf8875-a96c-1ab1-046c-136db591cd9d} {Suffrage:Voter ID:server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8 Address:28ff6f15-37c2-bfde-bfce-97bb80b2cac8} {Suffrage:Voter ID:server-d6eecb40-44a2-80ed-0099-f25daa7cb905 Address:d6eecb40-44a2-80ed-0099-f25daa7cb905} {Suffrage:Voter ID:server-674c7145-0e20-c026-98a5-1bbf859cf227 Address:674c7145-0e20-c026-98a5-1bbf859cf227} {Suffrage:Voter ID:server-e0ed200a-903b-96cd-1145-6098b46ee1cd Address:e0ed200a-903b-96cd-1145-6098b46ee1cd} {Suffrage:Voter ID:server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6 Address:7f4593f0-d11d-5c51-be28-fb05ca53dfd6}]"
    testing.go:149: server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8: 2024-01-08T00:31:11.981Z [INFO]  server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-164b7adb-6988-0e3c-a0e4-f6962a583cb3 Address:164b7adb-6988-0e3c-a0e4-f6962a583cb3} {Suffrage:Voter ID:server-98bf8875-a96c-1ab1-046c-136db591cd9d Address:98bf8875-a96c-1ab1-046c-136db591cd9d} {Suffrage:Voter ID:server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8 Address:28ff6f15-37c2-bfde-bfce-97bb80b2cac8} {Suffrage:Voter ID:server-d6eecb40-44a2-80ed-0099-f25daa7cb905 Address:d6eecb40-44a2-80ed-0099-f25daa7cb905} {Suffrage:Voter ID:server-674c7145-0e20-c026-98a5-1bbf859cf227 Address:674c7145-0e20-c026-98a5-1bbf859cf227} {Suffrage:Voter ID:server-e0ed200a-903b-96cd-1145-6098b46ee1cd Address:e0ed200a-903b-96cd-1145-6098b46ee1cd} {Suffrage:Voter ID:server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6 Address:7f4593f0-d11d-5c51-be28-fb05ca53dfd6}]"
testLoggerAdapter verbose: server-d6eecb40-44a2-80ed-0099-f25daa7cb905: 2024-01-08T00:31:11.981Z [INFO]  server-d6eecb40-44a2-80ed-0099-f25daa7cb905: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-164b7adb-6988-0e3c-a0e4-f6962a583cb3 Address:164b7adb-6988-0e3c-a0e4-f6962a583cb3} {Suffrage:Voter ID:server-98bf8875-a96c-1ab1-046c-136db591cd9d Address:98bf8875-a96c-1ab1-046c-136db591cd9d} {Suffrage:Voter ID:server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8 Address:28ff6f15-37c2-bfde-bfce-97bb80b2cac8} {Suffrage:Voter ID:server-d6eecb40-44a2-80ed-0099-f25daa7cb905 Address:d6eecb40-44a2-80ed-0099-f25daa7cb905} {Suffrage:Voter ID:server-674c7145-0e20-c026-98a5-1bbf859cf227 Address:674c7145-0e20-c026-98a5-1bbf859cf227} {Suffrage:Voter ID:server-e0ed200a-903b-96cd-1145-6098b46ee1cd Address:e0ed200a-903b-96cd-1145-6098b46ee1cd} {Suffrage:Voter ID:server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6 Address:7f4593f0-d11d-5c51-be28-fb05ca53dfd6}]"
    testing.go:149: server-d6eecb40-44a2-80ed-0099-f25daa7cb905: 2024-01-08T00:31:11.981Z [INFO]  server-d6eecb40-44a2-80ed-0099-f25daa7cb905: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-164b7adb-6988-0e3c-a0e4-f6962a583cb3 Address:164b7adb-6988-0e3c-a0e4-f6962a583cb3} {Suffrage:Voter ID:server-98bf8875-a96c-1ab1-046c-136db591cd9d Address:98bf8875-a96c-1ab1-046c-136db591cd9d} {Suffrage:Voter ID:server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8 Address:28ff6f15-37c2-bfde-bfce-97bb80b2cac8} {Suffrage:Voter ID:server-d6eecb40-44a2-80ed-0099-f25daa7cb905 Address:d6eecb40-44a2-80ed-0099-f25daa7cb905} {Suffrage:Voter ID:server-674c7145-0e20-c026-98a5-1bbf859cf227 Address:674c7145-0e20-c026-98a5-1bbf859cf227} {Suffrage:Voter ID:server-e0ed200a-903b-96cd-1145-6098b46ee1cd Address:e0ed200a-903b-96cd-1145-6098b46ee1cd} {Suffrage:Voter ID:server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6 Address:7f4593f0-d11d-5c51-be28-fb05ca53dfd6}]"
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:11.981Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-164b7adb-6988-0e3c-a0e4-f6962a583cb3 Address:164b7adb-6988-0e3c-a0e4-f6962a583cb3} {Suffrage:Voter ID:server-98bf8875-a96c-1ab1-046c-136db591cd9d Address:98bf8875-a96c-1ab1-046c-136db591cd9d} {Suffrage:Voter ID:server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8 Address:28ff6f15-37c2-bfde-bfce-97bb80b2cac8} {Suffrage:Voter ID:server-d6eecb40-44a2-80ed-0099-f25daa7cb905 Address:d6eecb40-44a2-80ed-0099-f25daa7cb905} {Suffrage:Voter ID:server-674c7145-0e20-c026-98a5-1bbf859cf227 Address:674c7145-0e20-c026-98a5-1bbf859cf227} {Suffrage:Voter ID:server-e0ed200a-903b-96cd-1145-6098b46ee1cd Address:e0ed200a-903b-96cd-1145-6098b46ee1cd} {Suffrage:Voter ID:server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6 Address:7f4593f0-d11d-5c51-be28-fb05ca53dfd6}]"
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:11.981Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-164b7adb-6988-0e3c-a0e4-f6962a583cb3 Address:164b7adb-6988-0e3c-a0e4-f6962a583cb3} {Suffrage:Voter ID:server-98bf8875-a96c-1ab1-046c-136db591cd9d Address:98bf8875-a96c-1ab1-046c-136db591cd9d} {Suffrage:Voter ID:server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8 Address:28ff6f15-37c2-bfde-bfce-97bb80b2cac8} {Suffrage:Voter ID:server-d6eecb40-44a2-80ed-0099-f25daa7cb905 Address:d6eecb40-44a2-80ed-0099-f25daa7cb905} {Suffrage:Voter ID:server-674c7145-0e20-c026-98a5-1bbf859cf227 Address:674c7145-0e20-c026-98a5-1bbf859cf227} {Suffrage:Voter ID:server-e0ed200a-903b-96cd-1145-6098b46ee1cd Address:e0ed200a-903b-96cd-1145-6098b46ee1cd} {Suffrage:Voter ID:server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6 Address:7f4593f0-d11d-5c51-be28-fb05ca53dfd6}]"
testLoggerAdapter verbose: server-e0ed200a-903b-96cd-1145-6098b46ee1cd: 2024-01-08T00:31:11.981Z [INFO]  server-e0ed200a-903b-96cd-1145-6098b46ee1cd: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-164b7adb-6988-0e3c-a0e4-f6962a583cb3 Address:164b7adb-6988-0e3c-a0e4-f6962a583cb3} {Suffrage:Voter ID:server-98bf8875-a96c-1ab1-046c-136db591cd9d Address:98bf8875-a96c-1ab1-046c-136db591cd9d} {Suffrage:Voter ID:server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8 Address:28ff6f15-37c2-bfde-bfce-97bb80b2cac8} {Suffrage:Voter ID:server-d6eecb40-44a2-80ed-0099-f25daa7cb905 Address:d6eecb40-44a2-80ed-0099-f25daa7cb905} {Suffrage:Voter ID:server-674c7145-0e20-c026-98a5-1bbf859cf227 Address:674c7145-0e20-c026-98a5-1bbf859cf227} {Suffrage:Voter ID:server-e0ed200a-903b-96cd-1145-6098b46ee1cd Address:e0ed200a-903b-96cd-1145-6098b46ee1cd} {Suffrage:Voter ID:server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6 Address:7f4593f0-d11d-5c51-be28-fb05ca53dfd6}]"
    testing.go:149: server-e0ed200a-903b-96cd-1145-6098b46ee1cd: 2024-01-08T00:31:11.981Z [INFO]  server-e0ed200a-903b-96cd-1145-6098b46ee1cd: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-164b7adb-6988-0e3c-a0e4-f6962a583cb3 Address:164b7adb-6988-0e3c-a0e4-f6962a583cb3} {Suffrage:Voter ID:server-98bf8875-a96c-1ab1-046c-136db591cd9d Address:98bf8875-a96c-1ab1-046c-136db591cd9d} {Suffrage:Voter ID:server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8 Address:28ff6f15-37c2-bfde-bfce-97bb80b2cac8} {Suffrage:Voter ID:server-d6eecb40-44a2-80ed-0099-f25daa7cb905 Address:d6eecb40-44a2-80ed-0099-f25daa7cb905} {Suffrage:Voter ID:server-674c7145-0e20-c026-98a5-1bbf859cf227 Address:674c7145-0e20-c026-98a5-1bbf859cf227} {Suffrage:Voter ID:server-e0ed200a-903b-96cd-1145-6098b46ee1cd Address:e0ed200a-903b-96cd-1145-6098b46ee1cd} {Suffrage:Voter ID:server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6 Address:7f4593f0-d11d-5c51-be28-fb05ca53dfd6}]"
testLoggerAdapter verbose: server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6: 2024-01-08T00:31:11.982Z [INFO]  server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-164b7adb-6988-0e3c-a0e4-f6962a583cb3 Address:164b7adb-6988-0e3c-a0e4-f6962a583cb3} {Suffrage:Voter ID:server-98bf8875-a96c-1ab1-046c-136db591cd9d Address:98bf8875-a96c-1ab1-046c-136db591cd9d} {Suffrage:Voter ID:server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8 Address:28ff6f15-37c2-bfde-bfce-97bb80b2cac8} {Suffrage:Voter ID:server-d6eecb40-44a2-80ed-0099-f25daa7cb905 Address:d6eecb40-44a2-80ed-0099-f25daa7cb905} {Suffrage:Voter ID:server-674c7145-0e20-c026-98a5-1bbf859cf227 Address:674c7145-0e20-c026-98a5-1bbf859cf227} {Suffrage:Voter ID:server-e0ed200a-903b-96cd-1145-6098b46ee1cd Address:e0ed200a-903b-96cd-1145-6098b46ee1cd} {Suffrage:Voter ID:server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6 Address:7f4593f0-d11d-5c51-be28-fb05ca53dfd6}]"
    testing.go:149: server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6: 2024-01-08T00:31:11.982Z [INFO]  server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-164b7adb-6988-0e3c-a0e4-f6962a583cb3 Address:164b7adb-6988-0e3c-a0e4-f6962a583cb3} {Suffrage:Voter ID:server-98bf8875-a96c-1ab1-046c-136db591cd9d Address:98bf8875-a96c-1ab1-046c-136db591cd9d} {Suffrage:Voter ID:server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8 Address:28ff6f15-37c2-bfde-bfce-97bb80b2cac8} {Suffrage:Voter ID:server-d6eecb40-44a2-80ed-0099-f25daa7cb905 Address:d6eecb40-44a2-80ed-0099-f25daa7cb905} {Suffrage:Voter ID:server-674c7145-0e20-c026-98a5-1bbf859cf227 Address:674c7145-0e20-c026-98a5-1bbf859cf227} {Suffrage:Voter ID:server-e0ed200a-903b-96cd-1145-6098b46ee1cd Address:e0ed200a-903b-96cd-1145-6098b46ee1cd} {Suffrage:Voter ID:server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6 Address:7f4593f0-d11d-5c51-be28-fb05ca53dfd6}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:11.982Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:11.982Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:11.982Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: entering follower state: follower="Node at 164b7adb-6988-0e3c-a0e4-f6962a583cb3 [Follower]" leader=
    testing.go:149: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:11.982Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: entering follower state: follower="Node at 164b7adb-6988-0e3c-a0e4-f6962a583cb3 [Follower]" leader=
testLoggerAdapter verbose: server-98bf8875-a96c-1ab1-046c-136db591cd9d: 2024-01-08T00:31:11.982Z [INFO]  server-98bf8875-a96c-1ab1-046c-136db591cd9d: entering follower state: follower="Node at 98bf8875-a96c-1ab1-046c-136db591cd9d [Follower]" leader=
    testing.go:149: server-98bf8875-a96c-1ab1-046c-136db591cd9d: 2024-01-08T00:31:11.982Z [INFO]  server-98bf8875-a96c-1ab1-046c-136db591cd9d: entering follower state: follower="Node at 98bf8875-a96c-1ab1-046c-136db591cd9d [Follower]" leader=
testLoggerAdapter verbose: server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8: 2024-01-08T00:31:11.982Z [INFO]  server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8: entering follower state: follower="Node at 28ff6f15-37c2-bfde-bfce-97bb80b2cac8 [Follower]" leader=
    testing.go:149: server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8: 2024-01-08T00:31:11.982Z [INFO]  server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8: entering follower state: follower="Node at 28ff6f15-37c2-bfde-bfce-97bb80b2cac8 [Follower]" leader=
testLoggerAdapter verbose: server-d6eecb40-44a2-80ed-0099-f25daa7cb905: 2024-01-08T00:31:11.982Z [INFO]  server-d6eecb40-44a2-80ed-0099-f25daa7cb905: entering follower state: follower="Node at d6eecb40-44a2-80ed-0099-f25daa7cb905 [Follower]" leader=
    testing.go:149: server-d6eecb40-44a2-80ed-0099-f25daa7cb905: 2024-01-08T00:31:11.982Z [INFO]  server-d6eecb40-44a2-80ed-0099-f25daa7cb905: entering follower state: follower="Node at d6eecb40-44a2-80ed-0099-f25daa7cb905 [Follower]" leader=
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:11.982Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: entering follower state: follower="Node at 674c7145-0e20-c026-98a5-1bbf859cf227 [Follower]" leader=
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:11.982Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: entering follower state: follower="Node at 674c7145-0e20-c026-98a5-1bbf859cf227 [Follower]" leader=
testLoggerAdapter verbose: server-e0ed200a-903b-96cd-1145-6098b46ee1cd: 2024-01-08T00:31:11.982Z [INFO]  server-e0ed200a-903b-96cd-1145-6098b46ee1cd: entering follower state: follower="Node at e0ed200a-903b-96cd-1145-6098b46ee1cd [Follower]" leader=
    testing.go:149: server-e0ed200a-903b-96cd-1145-6098b46ee1cd: 2024-01-08T00:31:11.982Z [INFO]  server-e0ed200a-903b-96cd-1145-6098b46ee1cd: entering follower state: follower="Node at e0ed200a-903b-96cd-1145-6098b46ee1cd [Follower]" leader=
testLoggerAdapter verbose: server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6: 2024-01-08T00:31:11.982Z [INFO]  server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6: entering follower state: follower="Node at 7f4593f0-d11d-5c51-be28-fb05ca53dfd6 [Follower]" leader=
    testing.go:149: server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6: 2024-01-08T00:31:11.982Z [INFO]  server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6: entering follower state: follower="Node at 7f4593f0-d11d-5c51-be28-fb05ca53dfd6 [Follower]" leader=
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.039Z [WARN]  server-674c7145-0e20-c026-98a5-1bbf859cf227: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.039Z [WARN]  server-674c7145-0e20-c026-98a5-1bbf859cf227: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.039Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: entering candidate state: node="Node at 674c7145-0e20-c026-98a5-1bbf859cf227 [Candidate]" term=2
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.039Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: entering candidate state: node="Node at 674c7145-0e20-c026-98a5-1bbf859cf227 [Candidate]" term=2
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.040Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: election won: tally=4
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.040Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: election won: tally=4
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.040Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: entering leader state: leader="Node at 674c7145-0e20-c026-98a5-1bbf859cf227 [Leader]"
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.040Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: entering leader state: leader="Node at 674c7145-0e20-c026-98a5-1bbf859cf227 [Leader]"
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.040Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: added peer, starting replication: peer=server-164b7adb-6988-0e3c-a0e4-f6962a583cb3
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.040Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: added peer, starting replication: peer=server-164b7adb-6988-0e3c-a0e4-f6962a583cb3
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.040Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: added peer, starting replication: peer=server-98bf8875-a96c-1ab1-046c-136db591cd9d
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.040Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: added peer, starting replication: peer=server-98bf8875-a96c-1ab1-046c-136db591cd9d
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.040Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: added peer, starting replication: peer=server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.040Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: added peer, starting replication: peer=server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.040Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: added peer, starting replication: peer=server-d6eecb40-44a2-80ed-0099-f25daa7cb905
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.040Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: added peer, starting replication: peer=server-d6eecb40-44a2-80ed-0099-f25daa7cb905
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.040Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: added peer, starting replication: peer=server-e0ed200a-903b-96cd-1145-6098b46ee1cd
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.040Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: added peer, starting replication: peer=server-e0ed200a-903b-96cd-1145-6098b46ee1cd
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.040Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: added peer, starting replication: peer=server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.040Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: added peer, starting replication: peer=server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.040Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: pipelining replication: peer="{Voter server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6 7f4593f0-d11d-5c51-be28-fb05ca53dfd6}"
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.040Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: pipelining replication: peer="{Voter server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6 7f4593f0-d11d-5c51-be28-fb05ca53dfd6}"
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.040Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: pipelining replication: peer="{Voter server-164b7adb-6988-0e3c-a0e4-f6962a583cb3 164b7adb-6988-0e3c-a0e4-f6962a583cb3}"
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.040Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: pipelining replication: peer="{Voter server-164b7adb-6988-0e3c-a0e4-f6962a583cb3 164b7adb-6988-0e3c-a0e4-f6962a583cb3}"
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.040Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: pipelining replication: peer="{Voter server-98bf8875-a96c-1ab1-046c-136db591cd9d 98bf8875-a96c-1ab1-046c-136db591cd9d}"
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.040Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: pipelining replication: peer="{Voter server-98bf8875-a96c-1ab1-046c-136db591cd9d 98bf8875-a96c-1ab1-046c-136db591cd9d}"
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.041Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: pipelining replication: peer="{Voter server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8 28ff6f15-37c2-bfde-bfce-97bb80b2cac8}"
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.041Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: pipelining replication: peer="{Voter server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8 28ff6f15-37c2-bfde-bfce-97bb80b2cac8}"
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.041Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: pipelining replication: peer="{Voter server-d6eecb40-44a2-80ed-0099-f25daa7cb905 d6eecb40-44a2-80ed-0099-f25daa7cb905}"
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.041Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: pipelining replication: peer="{Voter server-d6eecb40-44a2-80ed-0099-f25daa7cb905 d6eecb40-44a2-80ed-0099-f25daa7cb905}"
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.041Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: pipelining replication: peer="{Voter server-e0ed200a-903b-96cd-1145-6098b46ee1cd e0ed200a-903b-96cd-1145-6098b46ee1cd}"
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.041Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: pipelining replication: peer="{Voter server-e0ed200a-903b-96cd-1145-6098b46ee1cd e0ed200a-903b-96cd-1145-6098b46ee1cd}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:12.146Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:12.04082743 +0000 UTC m=+18.087716628 (1 nodes), 58.589005ms from start of poll, 60.255995ms from cluster start. Timeout at 2024-01-08 00:31:12.146464813 +0000 UTC m=+18.193353987, 105.637359ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:12.146Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:12.04082743 +0000 UTC m=+18.087716628 (1 nodes), 58.589005ms from start of poll, 60.255995ms from cluster start. Timeout at 2024-01-08 00:31:12.146464813 +0000 UTC m=+18.193353987, 105.637359ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:12.146Z [INFO]  starting stability test: raft-state=Follower
    testing.go:149: cluster: 2024-01-08T00:31:12.146Z [INFO]  starting stability test: raft-state=Follower
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:12.252Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:12.146620129 +0000 UTC m=+18.193509341 (6 nodes), 3.66µs from start of poll, 166.048708ms from cluster start. Timeout at 2024-01-08 00:31:12.252323392 +0000 UTC m=+18.299212528, 105.703187ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:12.252Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:12.146620129 +0000 UTC m=+18.193509341 (6 nodes), 3.66µs from start of poll, 166.048708ms from cluster start. Timeout at 2024-01-08 00:31:12.252323392 +0000 UTC m=+18.299212528, 105.703187ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:12.252Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:12.252Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:12.359Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:12.253128115 +0000 UTC m=+18.300021103 (1 nodes), 11.874µs from start of poll, 272.56047ms from cluster start. Timeout at 2024-01-08 00:31:12.35916677 +0000 UTC m=+18.406055824, 106.034721ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:12.359Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:12.253128115 +0000 UTC m=+18.300021103 (1 nodes), 11.874µs from start of poll, 272.56047ms from cluster start. Timeout at 2024-01-08 00:31:12.35916677 +0000 UTC m=+18.406055824, 106.034721ms after stability
testLoggerAdapter verbose: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.359Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: entering candidate state: node="Node at 164b7adb-6988-0e3c-a0e4-f6962a583cb3 [Candidate]" term=3
    testing.go:149: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.359Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: entering candidate state: node="Node at 164b7adb-6988-0e3c-a0e4-f6962a583cb3 [Candidate]" term=3
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:12.359Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:12.359Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.359Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: election won: tally=4
    testing.go:149: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.359Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: election won: tally=4
testLoggerAdapter verbose: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.359Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: entering leader state: leader="Node at 164b7adb-6988-0e3c-a0e4-f6962a583cb3 [Leader]"
    testing.go:149: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.359Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: entering leader state: leader="Node at 164b7adb-6988-0e3c-a0e4-f6962a583cb3 [Leader]"
testLoggerAdapter verbose: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.359Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: added peer, starting replication: peer=server-98bf8875-a96c-1ab1-046c-136db591cd9d
    testing.go:149: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.359Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: added peer, starting replication: peer=server-98bf8875-a96c-1ab1-046c-136db591cd9d
testLoggerAdapter verbose: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.359Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: added peer, starting replication: peer=server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8
    testing.go:149: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.359Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: added peer, starting replication: peer=server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8
testLoggerAdapter verbose: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.359Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: added peer, starting replication: peer=server-d6eecb40-44a2-80ed-0099-f25daa7cb905
    testing.go:149: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.359Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: added peer, starting replication: peer=server-d6eecb40-44a2-80ed-0099-f25daa7cb905
testLoggerAdapter verbose: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.359Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: added peer, starting replication: peer=server-674c7145-0e20-c026-98a5-1bbf859cf227
    testing.go:149: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.359Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: added peer, starting replication: peer=server-674c7145-0e20-c026-98a5-1bbf859cf227
testLoggerAdapter verbose: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.359Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: added peer, starting replication: peer=server-e0ed200a-903b-96cd-1145-6098b46ee1cd
    testing.go:149: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.359Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: added peer, starting replication: peer=server-e0ed200a-903b-96cd-1145-6098b46ee1cd
testLoggerAdapter verbose: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.359Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: added peer, starting replication: peer=server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6
    testing.go:149: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.359Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: added peer, starting replication: peer=server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6
testLoggerAdapter verbose: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.359Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: pipelining replication: peer="{Voter server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6 7f4593f0-d11d-5c51-be28-fb05ca53dfd6}"
    testing.go:149: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.359Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: pipelining replication: peer="{Voter server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6 7f4593f0-d11d-5c51-be28-fb05ca53dfd6}"
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.359Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: entering follower state: follower="Node at 674c7145-0e20-c026-98a5-1bbf859cf227 [Follower]" leader=
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.359Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: entering follower state: follower="Node at 674c7145-0e20-c026-98a5-1bbf859cf227 [Follower]" leader=
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.359Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: aborting pipeline replication: peer="{Voter server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8 28ff6f15-37c2-bfde-bfce-97bb80b2cac8}"
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.359Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: aborting pipeline replication: peer="{Voter server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8 28ff6f15-37c2-bfde-bfce-97bb80b2cac8}"
testLoggerAdapter verbose: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.360Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: pipelining replication: peer="{Voter server-98bf8875-a96c-1ab1-046c-136db591cd9d 98bf8875-a96c-1ab1-046c-136db591cd9d}"
    testing.go:149: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.360Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: pipelining replication: peer="{Voter server-98bf8875-a96c-1ab1-046c-136db591cd9d 98bf8875-a96c-1ab1-046c-136db591cd9d}"
testLoggerAdapter verbose: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.360Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: pipelining replication: peer="{Voter server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8 28ff6f15-37c2-bfde-bfce-97bb80b2cac8}"
    testing.go:149: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.360Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: pipelining replication: peer="{Voter server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8 28ff6f15-37c2-bfde-bfce-97bb80b2cac8}"
testLoggerAdapter verbose: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.360Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: pipelining replication: peer="{Voter server-d6eecb40-44a2-80ed-0099-f25daa7cb905 d6eecb40-44a2-80ed-0099-f25daa7cb905}"
    testing.go:149: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.360Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: pipelining replication: peer="{Voter server-d6eecb40-44a2-80ed-0099-f25daa7cb905 d6eecb40-44a2-80ed-0099-f25daa7cb905}"
testLoggerAdapter verbose: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.360Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: pipelining replication: peer="{Voter server-674c7145-0e20-c026-98a5-1bbf859cf227 674c7145-0e20-c026-98a5-1bbf859cf227}"
    testing.go:149: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.360Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: pipelining replication: peer="{Voter server-674c7145-0e20-c026-98a5-1bbf859cf227 674c7145-0e20-c026-98a5-1bbf859cf227}"
testLoggerAdapter verbose: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.360Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: pipelining replication: peer="{Voter server-e0ed200a-903b-96cd-1145-6098b46ee1cd e0ed200a-903b-96cd-1145-6098b46ee1cd}"
    testing.go:149: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.360Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: pipelining replication: peer="{Voter server-e0ed200a-903b-96cd-1145-6098b46ee1cd e0ed200a-903b-96cd-1145-6098b46ee1cd}"
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.360Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: aborting pipeline replication: peer="{Voter server-d6eecb40-44a2-80ed-0099-f25daa7cb905 d6eecb40-44a2-80ed-0099-f25daa7cb905}"
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.360Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: aborting pipeline replication: peer="{Voter server-d6eecb40-44a2-80ed-0099-f25daa7cb905 d6eecb40-44a2-80ed-0099-f25daa7cb905}"
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.360Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: aborting pipeline replication: peer="{Voter server-e0ed200a-903b-96cd-1145-6098b46ee1cd e0ed200a-903b-96cd-1145-6098b46ee1cd}"
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.360Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: aborting pipeline replication: peer="{Voter server-e0ed200a-903b-96cd-1145-6098b46ee1cd e0ed200a-903b-96cd-1145-6098b46ee1cd}"
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.360Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: aborting pipeline replication: peer="{Voter server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6 7f4593f0-d11d-5c51-be28-fb05ca53dfd6}"
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.360Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: aborting pipeline replication: peer="{Voter server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6 7f4593f0-d11d-5c51-be28-fb05ca53dfd6}"
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.360Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: aborting pipeline replication: peer="{Voter server-164b7adb-6988-0e3c-a0e4-f6962a583cb3 164b7adb-6988-0e3c-a0e4-f6962a583cb3}"
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.360Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: aborting pipeline replication: peer="{Voter server-164b7adb-6988-0e3c-a0e4-f6962a583cb3 164b7adb-6988-0e3c-a0e4-f6962a583cb3}"
testLoggerAdapter verbose: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.360Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: aborting pipeline replication: peer="{Voter server-98bf8875-a96c-1ab1-046c-136db591cd9d 98bf8875-a96c-1ab1-046c-136db591cd9d}"
    testing.go:149: server-674c7145-0e20-c026-98a5-1bbf859cf227: 2024-01-08T00:31:12.360Z [INFO]  server-674c7145-0e20-c026-98a5-1bbf859cf227: aborting pipeline replication: peer="{Voter server-98bf8875-a96c-1ab1-046c-136db591cd9d 98bf8875-a96c-1ab1-046c-136db591cd9d}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:12.465Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:12.3600618 +0000 UTC m=+18.406950852 (1 nodes), 546.604µs from start of poll, 379.490219ms from cluster start. Timeout at 2024-01-08 00:31:12.465098955 +0000 UTC m=+18.511988001, 105.037149ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:12.465Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:12.3600618 +0000 UTC m=+18.406950852 (1 nodes), 546.604µs from start of poll, 379.490219ms from cluster start. Timeout at 2024-01-08 00:31:12.465098955 +0000 UTC m=+18.511988001, 105.037149ms after stability
testLoggerAdapter verbose: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.465Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: aborting pipeline replication: peer="{Voter server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8 28ff6f15-37c2-bfde-bfce-97bb80b2cac8}"
    testing.go:149: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.465Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: aborting pipeline replication: peer="{Voter server-28ff6f15-37c2-bfde-bfce-97bb80b2cac8 28ff6f15-37c2-bfde-bfce-97bb80b2cac8}"
testLoggerAdapter verbose: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.465Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: aborting pipeline replication: peer="{Voter server-d6eecb40-44a2-80ed-0099-f25daa7cb905 d6eecb40-44a2-80ed-0099-f25daa7cb905}"
    testing.go:149: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.465Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: aborting pipeline replication: peer="{Voter server-d6eecb40-44a2-80ed-0099-f25daa7cb905 d6eecb40-44a2-80ed-0099-f25daa7cb905}"
testLoggerAdapter verbose: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.465Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: aborting pipeline replication: peer="{Voter server-674c7145-0e20-c026-98a5-1bbf859cf227 674c7145-0e20-c026-98a5-1bbf859cf227}"
    testing.go:149: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.465Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: aborting pipeline replication: peer="{Voter server-674c7145-0e20-c026-98a5-1bbf859cf227 674c7145-0e20-c026-98a5-1bbf859cf227}"
testLoggerAdapter verbose: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.465Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: aborting pipeline replication: peer="{Voter server-e0ed200a-903b-96cd-1145-6098b46ee1cd e0ed200a-903b-96cd-1145-6098b46ee1cd}"
    testing.go:149: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.465Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: aborting pipeline replication: peer="{Voter server-e0ed200a-903b-96cd-1145-6098b46ee1cd e0ed200a-903b-96cd-1145-6098b46ee1cd}"
testLoggerAdapter verbose: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.465Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: aborting pipeline replication: peer="{Voter server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6 7f4593f0-d11d-5c51-be28-fb05ca53dfd6}"
    testing.go:149: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.465Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: aborting pipeline replication: peer="{Voter server-7f4593f0-d11d-5c51-be28-fb05ca53dfd6 7f4593f0-d11d-5c51-be28-fb05ca53dfd6}"
testLoggerAdapter verbose: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.465Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: aborting pipeline replication: peer="{Voter server-98bf8875-a96c-1ab1-046c-136db591cd9d 98bf8875-a96c-1ab1-046c-136db591cd9d}"
    testing.go:149: server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: 2024-01-08T00:31:12.465Z [INFO]  server-164b7adb-6988-0e3c-a0e4-f6962a583cb3: aborting pipeline replication: peer="{Voter server-98bf8875-a96c-1ab1-046c-136db591cd9d 98bf8875-a96c-1ab1-046c-136db591cd9d}"
--- PASS: TestRaft_LeadershipTransferWithSevenNodes (0.49s)
=== RUN   TestRaft_LeadershipTransferToInvalidID
testLoggerAdapter verbose: server-605a4112-d239-6ef2-fea3-50b782c01518: 2024-01-08T00:31:12.469Z [INFO]  server-605a4112-d239-6ef2-fea3-50b782c01518: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-605a4112-d239-6ef2-fea3-50b782c01518 Address:605a4112-d239-6ef2-fea3-50b782c01518} {Suffrage:Voter ID:server-346fd114-a0e4-1121-32e7-78f6ad1aa53e Address:346fd114-a0e4-1121-32e7-78f6ad1aa53e} {Suffrage:Voter ID:server-6df57732-58e0-859c-a87d-724d9f58ace2 Address:6df57732-58e0-859c-a87d-724d9f58ace2}]"
    testing.go:149: server-605a4112-d239-6ef2-fea3-50b782c01518: 2024-01-08T00:31:12.469Z [INFO]  server-605a4112-d239-6ef2-fea3-50b782c01518: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-605a4112-d239-6ef2-fea3-50b782c01518 Address:605a4112-d239-6ef2-fea3-50b782c01518} {Suffrage:Voter ID:server-346fd114-a0e4-1121-32e7-78f6ad1aa53e Address:346fd114-a0e4-1121-32e7-78f6ad1aa53e} {Suffrage:Voter ID:server-6df57732-58e0-859c-a87d-724d9f58ace2 Address:6df57732-58e0-859c-a87d-724d9f58ace2}]"
testLoggerAdapter verbose: server-346fd114-a0e4-1121-32e7-78f6ad1aa53e: 2024-01-08T00:31:12.469Z [INFO]  server-346fd114-a0e4-1121-32e7-78f6ad1aa53e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-605a4112-d239-6ef2-fea3-50b782c01518 Address:605a4112-d239-6ef2-fea3-50b782c01518} {Suffrage:Voter ID:server-346fd114-a0e4-1121-32e7-78f6ad1aa53e Address:346fd114-a0e4-1121-32e7-78f6ad1aa53e} {Suffrage:Voter ID:server-6df57732-58e0-859c-a87d-724d9f58ace2 Address:6df57732-58e0-859c-a87d-724d9f58ace2}]"
    testing.go:149: server-346fd114-a0e4-1121-32e7-78f6ad1aa53e: 2024-01-08T00:31:12.469Z [INFO]  server-346fd114-a0e4-1121-32e7-78f6ad1aa53e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-605a4112-d239-6ef2-fea3-50b782c01518 Address:605a4112-d239-6ef2-fea3-50b782c01518} {Suffrage:Voter ID:server-346fd114-a0e4-1121-32e7-78f6ad1aa53e Address:346fd114-a0e4-1121-32e7-78f6ad1aa53e} {Suffrage:Voter ID:server-6df57732-58e0-859c-a87d-724d9f58ace2 Address:6df57732-58e0-859c-a87d-724d9f58ace2}]"
testLoggerAdapter verbose: server-6df57732-58e0-859c-a87d-724d9f58ace2: 2024-01-08T00:31:12.469Z [INFO]  server-6df57732-58e0-859c-a87d-724d9f58ace2: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-605a4112-d239-6ef2-fea3-50b782c01518 Address:605a4112-d239-6ef2-fea3-50b782c01518} {Suffrage:Voter ID:server-346fd114-a0e4-1121-32e7-78f6ad1aa53e Address:346fd114-a0e4-1121-32e7-78f6ad1aa53e} {Suffrage:Voter ID:server-6df57732-58e0-859c-a87d-724d9f58ace2 Address:6df57732-58e0-859c-a87d-724d9f58ace2}]"
    testing.go:149: server-6df57732-58e0-859c-a87d-724d9f58ace2: 2024-01-08T00:31:12.469Z [INFO]  server-6df57732-58e0-859c-a87d-724d9f58ace2: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-605a4112-d239-6ef2-fea3-50b782c01518 Address:605a4112-d239-6ef2-fea3-50b782c01518} {Suffrage:Voter ID:server-346fd114-a0e4-1121-32e7-78f6ad1aa53e Address:346fd114-a0e4-1121-32e7-78f6ad1aa53e} {Suffrage:Voter ID:server-6df57732-58e0-859c-a87d-724d9f58ace2 Address:6df57732-58e0-859c-a87d-724d9f58ace2}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:12.469Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:12.469Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-605a4112-d239-6ef2-fea3-50b782c01518: 2024-01-08T00:31:12.469Z [INFO]  server-605a4112-d239-6ef2-fea3-50b782c01518: entering follower state: follower="Node at 605a4112-d239-6ef2-fea3-50b782c01518 [Follower]" leader=
    testing.go:149: server-605a4112-d239-6ef2-fea3-50b782c01518: 2024-01-08T00:31:12.469Z [INFO]  server-605a4112-d239-6ef2-fea3-50b782c01518: entering follower state: follower="Node at 605a4112-d239-6ef2-fea3-50b782c01518 [Follower]" leader=
testLoggerAdapter verbose: server-346fd114-a0e4-1121-32e7-78f6ad1aa53e: 2024-01-08T00:31:12.469Z [INFO]  server-346fd114-a0e4-1121-32e7-78f6ad1aa53e: entering follower state: follower="Node at 346fd114-a0e4-1121-32e7-78f6ad1aa53e [Follower]" leader=
    testing.go:149: server-346fd114-a0e4-1121-32e7-78f6ad1aa53e: 2024-01-08T00:31:12.469Z [INFO]  server-346fd114-a0e4-1121-32e7-78f6ad1aa53e: entering follower state: follower="Node at 346fd114-a0e4-1121-32e7-78f6ad1aa53e [Follower]" leader=
testLoggerAdapter verbose: server-6df57732-58e0-859c-a87d-724d9f58ace2: 2024-01-08T00:31:12.469Z [INFO]  server-6df57732-58e0-859c-a87d-724d9f58ace2: entering follower state: follower="Node at 6df57732-58e0-859c-a87d-724d9f58ace2 [Follower]" leader=
    testing.go:149: server-6df57732-58e0-859c-a87d-724d9f58ace2: 2024-01-08T00:31:12.469Z [INFO]  server-6df57732-58e0-859c-a87d-724d9f58ace2: entering follower state: follower="Node at 6df57732-58e0-859c-a87d-724d9f58ace2 [Follower]" leader=
testLoggerAdapter verbose: server-6df57732-58e0-859c-a87d-724d9f58ace2: 2024-01-08T00:31:12.533Z [WARN]  server-6df57732-58e0-859c-a87d-724d9f58ace2: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-6df57732-58e0-859c-a87d-724d9f58ace2: 2024-01-08T00:31:12.533Z [WARN]  server-6df57732-58e0-859c-a87d-724d9f58ace2: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-6df57732-58e0-859c-a87d-724d9f58ace2: 2024-01-08T00:31:12.533Z [INFO]  server-6df57732-58e0-859c-a87d-724d9f58ace2: entering candidate state: node="Node at 6df57732-58e0-859c-a87d-724d9f58ace2 [Candidate]" term=2
    testing.go:149: server-6df57732-58e0-859c-a87d-724d9f58ace2: 2024-01-08T00:31:12.533Z [INFO]  server-6df57732-58e0-859c-a87d-724d9f58ace2: entering candidate state: node="Node at 6df57732-58e0-859c-a87d-724d9f58ace2 [Candidate]" term=2
testLoggerAdapter verbose: server-6df57732-58e0-859c-a87d-724d9f58ace2: 2024-01-08T00:31:12.533Z [INFO]  server-6df57732-58e0-859c-a87d-724d9f58ace2: election won: tally=2
    testing.go:149: server-6df57732-58e0-859c-a87d-724d9f58ace2: 2024-01-08T00:31:12.533Z [INFO]  server-6df57732-58e0-859c-a87d-724d9f58ace2: election won: tally=2
testLoggerAdapter verbose: server-6df57732-58e0-859c-a87d-724d9f58ace2: 2024-01-08T00:31:12.533Z [INFO]  server-6df57732-58e0-859c-a87d-724d9f58ace2: entering leader state: leader="Node at 6df57732-58e0-859c-a87d-724d9f58ace2 [Leader]"
    testing.go:149: server-6df57732-58e0-859c-a87d-724d9f58ace2: 2024-01-08T00:31:12.533Z [INFO]  server-6df57732-58e0-859c-a87d-724d9f58ace2: entering leader state: leader="Node at 6df57732-58e0-859c-a87d-724d9f58ace2 [Leader]"
testLoggerAdapter verbose: server-6df57732-58e0-859c-a87d-724d9f58ace2: 2024-01-08T00:31:12.533Z [INFO]  server-6df57732-58e0-859c-a87d-724d9f58ace2: added peer, starting replication: peer=server-605a4112-d239-6ef2-fea3-50b782c01518
    testing.go:149: server-6df57732-58e0-859c-a87d-724d9f58ace2: 2024-01-08T00:31:12.533Z [INFO]  server-6df57732-58e0-859c-a87d-724d9f58ace2: added peer, starting replication: peer=server-605a4112-d239-6ef2-fea3-50b782c01518
testLoggerAdapter verbose: server-6df57732-58e0-859c-a87d-724d9f58ace2: 2024-01-08T00:31:12.533Z [INFO]  server-6df57732-58e0-859c-a87d-724d9f58ace2: added peer, starting replication: peer=server-346fd114-a0e4-1121-32e7-78f6ad1aa53e
    testing.go:149: server-6df57732-58e0-859c-a87d-724d9f58ace2: 2024-01-08T00:31:12.533Z [INFO]  server-6df57732-58e0-859c-a87d-724d9f58ace2: added peer, starting replication: peer=server-346fd114-a0e4-1121-32e7-78f6ad1aa53e
testLoggerAdapter verbose: server-6df57732-58e0-859c-a87d-724d9f58ace2: 2024-01-08T00:31:12.533Z [INFO]  server-6df57732-58e0-859c-a87d-724d9f58ace2: pipelining replication: peer="{Voter server-346fd114-a0e4-1121-32e7-78f6ad1aa53e 346fd114-a0e4-1121-32e7-78f6ad1aa53e}"
    testing.go:149: server-6df57732-58e0-859c-a87d-724d9f58ace2: 2024-01-08T00:31:12.533Z [INFO]  server-6df57732-58e0-859c-a87d-724d9f58ace2: pipelining replication: peer="{Voter server-346fd114-a0e4-1121-32e7-78f6ad1aa53e 346fd114-a0e4-1121-32e7-78f6ad1aa53e}"
testLoggerAdapter verbose: server-6df57732-58e0-859c-a87d-724d9f58ace2: 2024-01-08T00:31:12.533Z [INFO]  server-6df57732-58e0-859c-a87d-724d9f58ace2: pipelining replication: peer="{Voter server-605a4112-d239-6ef2-fea3-50b782c01518 605a4112-d239-6ef2-fea3-50b782c01518}"
    testing.go:149: server-6df57732-58e0-859c-a87d-724d9f58ace2: 2024-01-08T00:31:12.533Z [INFO]  server-6df57732-58e0-859c-a87d-724d9f58ace2: pipelining replication: peer="{Voter server-605a4112-d239-6ef2-fea3-50b782c01518 605a4112-d239-6ef2-fea3-50b782c01518}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:12.639Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:12.533924166 +0000 UTC m=+18.580813371 (1 nodes), 64.052627ms from start of poll, 64.469859ms from cluster start. Timeout at 2024-01-08 00:31:12.639776791 +0000 UTC m=+18.686665913, 105.852542ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:12.639Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:12.533924166 +0000 UTC m=+18.580813371 (1 nodes), 64.052627ms from start of poll, 64.469859ms from cluster start. Timeout at 2024-01-08 00:31:12.639776791 +0000 UTC m=+18.686665913, 105.852542ms after stability
testLoggerAdapter verbose: server-6df57732-58e0-859c-a87d-724d9f58ace2: 2024-01-08T00:31:12.639Z [INFO]  server-6df57732-58e0-859c-a87d-724d9f58ace2: aborting pipeline replication: peer="{Voter server-346fd114-a0e4-1121-32e7-78f6ad1aa53e 346fd114-a0e4-1121-32e7-78f6ad1aa53e}"
    testing.go:149: server-6df57732-58e0-859c-a87d-724d9f58ace2: 2024-01-08T00:31:12.639Z [INFO]  server-6df57732-58e0-859c-a87d-724d9f58ace2: aborting pipeline replication: peer="{Voter server-346fd114-a0e4-1121-32e7-78f6ad1aa53e 346fd114-a0e4-1121-32e7-78f6ad1aa53e}"
testLoggerAdapter verbose: server-6df57732-58e0-859c-a87d-724d9f58ace2: 2024-01-08T00:31:12.640Z [INFO]  server-6df57732-58e0-859c-a87d-724d9f58ace2: aborting pipeline replication: peer="{Voter server-605a4112-d239-6ef2-fea3-50b782c01518 605a4112-d239-6ef2-fea3-50b782c01518}"
    testing.go:149: server-6df57732-58e0-859c-a87d-724d9f58ace2: 2024-01-08T00:31:12.640Z [INFO]  server-6df57732-58e0-859c-a87d-724d9f58ace2: aborting pipeline replication: peer="{Voter server-605a4112-d239-6ef2-fea3-50b782c01518 605a4112-d239-6ef2-fea3-50b782c01518}"
--- PASS: TestRaft_LeadershipTransferToInvalidID (0.17s)
=== RUN   TestRaft_LeadershipTransferToInvalidAddress
testLoggerAdapter verbose: server-0dfcde5d-8416-36fa-d55e-16254f3730e3: 2024-01-08T00:31:12.641Z [INFO]  server-0dfcde5d-8416-36fa-d55e-16254f3730e3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0dfcde5d-8416-36fa-d55e-16254f3730e3 Address:0dfcde5d-8416-36fa-d55e-16254f3730e3} {Suffrage:Voter ID:server-e0621db4-42b8-f994-878f-4853232c2e7f Address:e0621db4-42b8-f994-878f-4853232c2e7f} {Suffrage:Voter ID:server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98 Address:675b4019-5b1d-d0c3-4ac5-dbc50ec31d98}]"
    testing.go:149: server-0dfcde5d-8416-36fa-d55e-16254f3730e3: 2024-01-08T00:31:12.641Z [INFO]  server-0dfcde5d-8416-36fa-d55e-16254f3730e3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0dfcde5d-8416-36fa-d55e-16254f3730e3 Address:0dfcde5d-8416-36fa-d55e-16254f3730e3} {Suffrage:Voter ID:server-e0621db4-42b8-f994-878f-4853232c2e7f Address:e0621db4-42b8-f994-878f-4853232c2e7f} {Suffrage:Voter ID:server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98 Address:675b4019-5b1d-d0c3-4ac5-dbc50ec31d98}]"
testLoggerAdapter verbose: server-e0621db4-42b8-f994-878f-4853232c2e7f: 2024-01-08T00:31:12.641Z [INFO]  server-e0621db4-42b8-f994-878f-4853232c2e7f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0dfcde5d-8416-36fa-d55e-16254f3730e3 Address:0dfcde5d-8416-36fa-d55e-16254f3730e3} {Suffrage:Voter ID:server-e0621db4-42b8-f994-878f-4853232c2e7f Address:e0621db4-42b8-f994-878f-4853232c2e7f} {Suffrage:Voter ID:server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98 Address:675b4019-5b1d-d0c3-4ac5-dbc50ec31d98}]"
    testing.go:149: server-e0621db4-42b8-f994-878f-4853232c2e7f: 2024-01-08T00:31:12.641Z [INFO]  server-e0621db4-42b8-f994-878f-4853232c2e7f: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0dfcde5d-8416-36fa-d55e-16254f3730e3 Address:0dfcde5d-8416-36fa-d55e-16254f3730e3} {Suffrage:Voter ID:server-e0621db4-42b8-f994-878f-4853232c2e7f Address:e0621db4-42b8-f994-878f-4853232c2e7f} {Suffrage:Voter ID:server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98 Address:675b4019-5b1d-d0c3-4ac5-dbc50ec31d98}]"
testLoggerAdapter verbose: server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: 2024-01-08T00:31:12.641Z [INFO]  server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0dfcde5d-8416-36fa-d55e-16254f3730e3 Address:0dfcde5d-8416-36fa-d55e-16254f3730e3} {Suffrage:Voter ID:server-e0621db4-42b8-f994-878f-4853232c2e7f Address:e0621db4-42b8-f994-878f-4853232c2e7f} {Suffrage:Voter ID:server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98 Address:675b4019-5b1d-d0c3-4ac5-dbc50ec31d98}]"
    testing.go:149: server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: 2024-01-08T00:31:12.641Z [INFO]  server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0dfcde5d-8416-36fa-d55e-16254f3730e3 Address:0dfcde5d-8416-36fa-d55e-16254f3730e3} {Suffrage:Voter ID:server-e0621db4-42b8-f994-878f-4853232c2e7f Address:e0621db4-42b8-f994-878f-4853232c2e7f} {Suffrage:Voter ID:server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98 Address:675b4019-5b1d-d0c3-4ac5-dbc50ec31d98}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:12.642Z [INFO]  starting stability test: raft-state=Follower
    testing.go:149: cluster: 2024-01-08T00:31:12.642Z [INFO]  starting stability test: raft-state=Follower
testLoggerAdapter verbose: server-0dfcde5d-8416-36fa-d55e-16254f3730e3: 2024-01-08T00:31:12.642Z [INFO]  server-0dfcde5d-8416-36fa-d55e-16254f3730e3: entering follower state: follower="Node at 0dfcde5d-8416-36fa-d55e-16254f3730e3 [Follower]" leader=
    testing.go:149: server-0dfcde5d-8416-36fa-d55e-16254f3730e3: 2024-01-08T00:31:12.642Z [INFO]  server-0dfcde5d-8416-36fa-d55e-16254f3730e3: entering follower state: follower="Node at 0dfcde5d-8416-36fa-d55e-16254f3730e3 [Follower]" leader=
testLoggerAdapter verbose: server-e0621db4-42b8-f994-878f-4853232c2e7f: 2024-01-08T00:31:12.642Z [INFO]  server-e0621db4-42b8-f994-878f-4853232c2e7f: entering follower state: follower="Node at e0621db4-42b8-f994-878f-4853232c2e7f [Follower]" leader=
    testing.go:149: server-e0621db4-42b8-f994-878f-4853232c2e7f: 2024-01-08T00:31:12.642Z [INFO]  server-e0621db4-42b8-f994-878f-4853232c2e7f: entering follower state: follower="Node at e0621db4-42b8-f994-878f-4853232c2e7f [Follower]" leader=
testLoggerAdapter verbose: server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: 2024-01-08T00:31:12.642Z [INFO]  server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: entering follower state: follower="Node at 675b4019-5b1d-d0c3-4ac5-dbc50ec31d98 [Follower]" leader=
    testing.go:149: server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: 2024-01-08T00:31:12.642Z [INFO]  server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: entering follower state: follower="Node at 675b4019-5b1d-d0c3-4ac5-dbc50ec31d98 [Follower]" leader=
testLoggerAdapter verbose: server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: 2024-01-08T00:31:12.714Z [WARN]  server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: 2024-01-08T00:31:12.714Z [WARN]  server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: 2024-01-08T00:31:12.714Z [INFO]  server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: entering candidate state: node="Node at 675b4019-5b1d-d0c3-4ac5-dbc50ec31d98 [Candidate]" term=2
    testing.go:149: server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: 2024-01-08T00:31:12.714Z [INFO]  server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: entering candidate state: node="Node at 675b4019-5b1d-d0c3-4ac5-dbc50ec31d98 [Candidate]" term=2
testLoggerAdapter verbose: server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: 2024-01-08T00:31:12.715Z [INFO]  server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: election won: tally=2
    testing.go:149: server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: 2024-01-08T00:31:12.715Z [INFO]  server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: election won: tally=2
testLoggerAdapter verbose: server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: 2024-01-08T00:31:12.715Z [INFO]  server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: entering leader state: leader="Node at 675b4019-5b1d-d0c3-4ac5-dbc50ec31d98 [Leader]"
    testing.go:149: server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: 2024-01-08T00:31:12.715Z [INFO]  server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: entering leader state: leader="Node at 675b4019-5b1d-d0c3-4ac5-dbc50ec31d98 [Leader]"
testLoggerAdapter verbose: server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: 2024-01-08T00:31:12.715Z [INFO]  server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: added peer, starting replication: peer=server-0dfcde5d-8416-36fa-d55e-16254f3730e3
    testing.go:149: server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: 2024-01-08T00:31:12.715Z [INFO]  server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: added peer, starting replication: peer=server-0dfcde5d-8416-36fa-d55e-16254f3730e3
testLoggerAdapter verbose: server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: 2024-01-08T00:31:12.715Z [INFO]  server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: added peer, starting replication: peer=server-e0621db4-42b8-f994-878f-4853232c2e7f
    testing.go:149: server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: 2024-01-08T00:31:12.715Z [INFO]  server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: added peer, starting replication: peer=server-e0621db4-42b8-f994-878f-4853232c2e7f
testLoggerAdapter verbose: server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: 2024-01-08T00:31:12.715Z [INFO]  server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: pipelining replication: peer="{Voter server-e0621db4-42b8-f994-878f-4853232c2e7f e0621db4-42b8-f994-878f-4853232c2e7f}"
    testing.go:149: server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: 2024-01-08T00:31:12.715Z [INFO]  server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: pipelining replication: peer="{Voter server-e0621db4-42b8-f994-878f-4853232c2e7f e0621db4-42b8-f994-878f-4853232c2e7f}"
testLoggerAdapter verbose: server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: 2024-01-08T00:31:12.715Z [INFO]  server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: pipelining replication: peer="{Voter server-0dfcde5d-8416-36fa-d55e-16254f3730e3 0dfcde5d-8416-36fa-d55e-16254f3730e3}"
    testing.go:149: server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: 2024-01-08T00:31:12.715Z [INFO]  server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: pipelining replication: peer="{Voter server-0dfcde5d-8416-36fa-d55e-16254f3730e3 0dfcde5d-8416-36fa-d55e-16254f3730e3}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:12.820Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:12.715472135 +0000 UTC m=+18.762361186 (2 nodes), 73.434247ms from start of poll, 73.949801ms from cluster start. Timeout at 2024-01-08 00:31:12.820779602 +0000 UTC m=+18.867668713, 105.307527ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:12.820Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:12.715472135 +0000 UTC m=+18.762361186 (2 nodes), 73.434247ms from start of poll, 73.949801ms from cluster start. Timeout at 2024-01-08 00:31:12.820779602 +0000 UTC m=+18.867668713, 105.307527ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:12.820Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:12.820Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:12.926Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:12.820999375 +0000 UTC m=+18.867888495 (1 nodes), 2.259µs from start of poll, 179.47711ms from cluster start. Timeout at 2024-01-08 00:31:12.926825553 +0000 UTC m=+18.973714595, 105.8261ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:12.926Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:12.820999375 +0000 UTC m=+18.867888495 (1 nodes), 2.259µs from start of poll, 179.47711ms from cluster start. Timeout at 2024-01-08 00:31:12.926825553 +0000 UTC m=+18.973714595, 105.8261ms after stability
testLoggerAdapter verbose: server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: 2024-01-08T00:31:12.927Z [INFO]  server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: aborting pipeline replication: peer="{Voter server-e0621db4-42b8-f994-878f-4853232c2e7f e0621db4-42b8-f994-878f-4853232c2e7f}"
    testing.go:149: server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: 2024-01-08T00:31:12.927Z [INFO]  server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: aborting pipeline replication: peer="{Voter server-e0621db4-42b8-f994-878f-4853232c2e7f e0621db4-42b8-f994-878f-4853232c2e7f}"
testLoggerAdapter verbose: server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: 2024-01-08T00:31:12.927Z [INFO]  server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: aborting pipeline replication: peer="{Voter server-0dfcde5d-8416-36fa-d55e-16254f3730e3 0dfcde5d-8416-36fa-d55e-16254f3730e3}"
    testing.go:149: server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: 2024-01-08T00:31:12.927Z [INFO]  server-675b4019-5b1d-d0c3-4ac5-dbc50ec31d98: aborting pipeline replication: peer="{Voter server-0dfcde5d-8416-36fa-d55e-16254f3730e3 0dfcde5d-8416-36fa-d55e-16254f3730e3}"
--- PASS: TestRaft_LeadershipTransferToInvalidAddress (0.29s)
=== RUN   TestRaft_LeadershipTransferToBehindServer
testLoggerAdapter verbose: server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: 2024-01-08T00:31:12.928Z [INFO]  server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923 Address:f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923} {Suffrage:Voter ID:server-73e1560a-14e4-b7ce-2078-a5c45305953d Address:73e1560a-14e4-b7ce-2078-a5c45305953d} {Suffrage:Voter ID:server-120b2497-465a-c4b2-0183-8928dbc276bb Address:120b2497-465a-c4b2-0183-8928dbc276bb}]"
    testing.go:149: server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: 2024-01-08T00:31:12.928Z [INFO]  server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923 Address:f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923} {Suffrage:Voter ID:server-73e1560a-14e4-b7ce-2078-a5c45305953d Address:73e1560a-14e4-b7ce-2078-a5c45305953d} {Suffrage:Voter ID:server-120b2497-465a-c4b2-0183-8928dbc276bb Address:120b2497-465a-c4b2-0183-8928dbc276bb}]"
testLoggerAdapter verbose: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:12.928Z [INFO]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923 Address:f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923} {Suffrage:Voter ID:server-73e1560a-14e4-b7ce-2078-a5c45305953d Address:73e1560a-14e4-b7ce-2078-a5c45305953d} {Suffrage:Voter ID:server-120b2497-465a-c4b2-0183-8928dbc276bb Address:120b2497-465a-c4b2-0183-8928dbc276bb}]"
    testing.go:149: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:12.928Z [INFO]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923 Address:f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923} {Suffrage:Voter ID:server-73e1560a-14e4-b7ce-2078-a5c45305953d Address:73e1560a-14e4-b7ce-2078-a5c45305953d} {Suffrage:Voter ID:server-120b2497-465a-c4b2-0183-8928dbc276bb Address:120b2497-465a-c4b2-0183-8928dbc276bb}]"
testLoggerAdapter verbose: server-120b2497-465a-c4b2-0183-8928dbc276bb: 2024-01-08T00:31:12.928Z [INFO]  server-120b2497-465a-c4b2-0183-8928dbc276bb: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923 Address:f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923} {Suffrage:Voter ID:server-73e1560a-14e4-b7ce-2078-a5c45305953d Address:73e1560a-14e4-b7ce-2078-a5c45305953d} {Suffrage:Voter ID:server-120b2497-465a-c4b2-0183-8928dbc276bb Address:120b2497-465a-c4b2-0183-8928dbc276bb}]"
    testing.go:149: server-120b2497-465a-c4b2-0183-8928dbc276bb: 2024-01-08T00:31:12.928Z [INFO]  server-120b2497-465a-c4b2-0183-8928dbc276bb: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923 Address:f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923} {Suffrage:Voter ID:server-73e1560a-14e4-b7ce-2078-a5c45305953d Address:73e1560a-14e4-b7ce-2078-a5c45305953d} {Suffrage:Voter ID:server-120b2497-465a-c4b2-0183-8928dbc276bb Address:120b2497-465a-c4b2-0183-8928dbc276bb}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:12.928Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:12.928Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: 2024-01-08T00:31:12.928Z [INFO]  server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: entering follower state: follower="Node at f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923 [Follower]" leader=
    testing.go:149: server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: 2024-01-08T00:31:12.928Z [INFO]  server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: entering follower state: follower="Node at f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923 [Follower]" leader=
testLoggerAdapter verbose: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:12.928Z [INFO]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: entering follower state: follower="Node at 73e1560a-14e4-b7ce-2078-a5c45305953d [Follower]" leader=
    testing.go:149: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:12.928Z [INFO]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: entering follower state: follower="Node at 73e1560a-14e4-b7ce-2078-a5c45305953d [Follower]" leader=
testLoggerAdapter verbose: server-120b2497-465a-c4b2-0183-8928dbc276bb: 2024-01-08T00:31:12.928Z [INFO]  server-120b2497-465a-c4b2-0183-8928dbc276bb: entering follower state: follower="Node at 120b2497-465a-c4b2-0183-8928dbc276bb [Follower]" leader=
    testing.go:149: server-120b2497-465a-c4b2-0183-8928dbc276bb: 2024-01-08T00:31:12.928Z [INFO]  server-120b2497-465a-c4b2-0183-8928dbc276bb: entering follower state: follower="Node at 120b2497-465a-c4b2-0183-8928dbc276bb [Follower]" leader=
testLoggerAdapter verbose: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:12.980Z [WARN]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:12.980Z [WARN]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:12.980Z [INFO]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: entering candidate state: node="Node at 73e1560a-14e4-b7ce-2078-a5c45305953d [Candidate]" term=2
    testing.go:149: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:12.980Z [INFO]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: entering candidate state: node="Node at 73e1560a-14e4-b7ce-2078-a5c45305953d [Candidate]" term=2
testLoggerAdapter verbose: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:12.980Z [INFO]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: election won: tally=2
    testing.go:149: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:12.980Z [INFO]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: election won: tally=2
testLoggerAdapter verbose: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:12.980Z [INFO]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: entering leader state: leader="Node at 73e1560a-14e4-b7ce-2078-a5c45305953d [Leader]"
    testing.go:149: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:12.980Z [INFO]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: entering leader state: leader="Node at 73e1560a-14e4-b7ce-2078-a5c45305953d [Leader]"
testLoggerAdapter verbose: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:12.980Z [INFO]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: added peer, starting replication: peer=server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923
    testing.go:149: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:12.980Z [INFO]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: added peer, starting replication: peer=server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923
testLoggerAdapter verbose: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:12.980Z [INFO]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: added peer, starting replication: peer=server-120b2497-465a-c4b2-0183-8928dbc276bb
    testing.go:149: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:12.980Z [INFO]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: added peer, starting replication: peer=server-120b2497-465a-c4b2-0183-8928dbc276bb
testLoggerAdapter verbose: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:12.981Z [INFO]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: pipelining replication: peer="{Voter server-120b2497-465a-c4b2-0183-8928dbc276bb 120b2497-465a-c4b2-0183-8928dbc276bb}"
    testing.go:149: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:12.981Z [INFO]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: pipelining replication: peer="{Voter server-120b2497-465a-c4b2-0183-8928dbc276bb 120b2497-465a-c4b2-0183-8928dbc276bb}"
testLoggerAdapter verbose: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:12.981Z [INFO]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: pipelining replication: peer="{Voter server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923 f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923}"
    testing.go:149: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:12.981Z [INFO]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: pipelining replication: peer="{Voter server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923 f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:13.087Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:12.981258051 +0000 UTC m=+19.028147108 (1 nodes), 52.566353ms from start of poll, 53.048673ms from cluster start. Timeout at 2024-01-08 00:31:13.087154202 +0000 UTC m=+19.134043318, 105.89621ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:13.087Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:12.981258051 +0000 UTC m=+19.028147108 (1 nodes), 52.566353ms from start of poll, 53.048673ms from cluster start. Timeout at 2024-01-08 00:31:13.087154202 +0000 UTC m=+19.134043318, 105.89621ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:13.087Z [INFO]  starting stability test: raft-state=Follower
    testing.go:149: cluster: 2024-01-08T00:31:13.087Z [INFO]  starting stability test: raft-state=Follower
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:13.192Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:13.087309318 +0000 UTC m=+19.134198445 (2 nodes), 2.454µs from start of poll, 159.10001ms from cluster start. Timeout at 2024-01-08 00:31:13.192347804 +0000 UTC m=+19.239236927, 105.038482ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:13.192Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:13.087309318 +0000 UTC m=+19.134198445 (2 nodes), 2.454µs from start of poll, 159.10001ms from cluster start. Timeout at 2024-01-08 00:31:13.192347804 +0000 UTC m=+19.239236927, 105.038482ms after stability
testLoggerAdapter verbose: server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: 2024-01-08T00:31:13.221Z [INFO]  server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: entering candidate state: node="Node at f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923 [Candidate]" term=3
    testing.go:149: server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: 2024-01-08T00:31:13.221Z [INFO]  server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: entering candidate state: node="Node at f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923 [Candidate]" term=3
testLoggerAdapter verbose: server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: 2024-01-08T00:31:13.221Z [INFO]  server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: election won: tally=2
    testing.go:149: server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: 2024-01-08T00:31:13.221Z [INFO]  server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: election won: tally=2
testLoggerAdapter verbose: server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: 2024-01-08T00:31:13.221Z [INFO]  server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: entering leader state: leader="Node at f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923 [Leader]"
    testing.go:149: server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: 2024-01-08T00:31:13.221Z [INFO]  server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: entering leader state: leader="Node at f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923 [Leader]"
testLoggerAdapter verbose: server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: 2024-01-08T00:31:13.221Z [INFO]  server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: added peer, starting replication: peer=server-73e1560a-14e4-b7ce-2078-a5c45305953d
    testing.go:149: server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: 2024-01-08T00:31:13.221Z [INFO]  server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: added peer, starting replication: peer=server-73e1560a-14e4-b7ce-2078-a5c45305953d
testLoggerAdapter verbose: server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: 2024-01-08T00:31:13.221Z [INFO]  server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: added peer, starting replication: peer=server-120b2497-465a-c4b2-0183-8928dbc276bb
    testing.go:149: server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: 2024-01-08T00:31:13.221Z [INFO]  server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: added peer, starting replication: peer=server-120b2497-465a-c4b2-0183-8928dbc276bb
testLoggerAdapter verbose: server-120b2497-465a-c4b2-0183-8928dbc276bb: 2024-01-08T00:31:13.221Z [WARN]  server-120b2497-465a-c4b2-0183-8928dbc276bb: failed to get previous log: previous-index=1002 last-index=822 error="log not found"
    testing.go:149: server-120b2497-465a-c4b2-0183-8928dbc276bb: 2024-01-08T00:31:13.221Z [WARN]  server-120b2497-465a-c4b2-0183-8928dbc276bb: failed to get previous log: previous-index=1002 last-index=822 error="log not found"
testLoggerAdapter verbose: server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: 2024-01-08T00:31:13.221Z [WARN]  server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: appendEntries rejected, sending older logs: peer="{Voter server-120b2497-465a-c4b2-0183-8928dbc276bb 120b2497-465a-c4b2-0183-8928dbc276bb}" next=823
    testing.go:149: server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: 2024-01-08T00:31:13.221Z [WARN]  server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: appendEntries rejected, sending older logs: peer="{Voter server-120b2497-465a-c4b2-0183-8928dbc276bb 120b2497-465a-c4b2-0183-8928dbc276bb}" next=823
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:13.221Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:13.221Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: 2024-01-08T00:31:13.221Z [INFO]  server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: pipelining replication: peer="{Voter server-120b2497-465a-c4b2-0183-8928dbc276bb 120b2497-465a-c4b2-0183-8928dbc276bb}"
    testing.go:149: server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: 2024-01-08T00:31:13.221Z [INFO]  server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: pipelining replication: peer="{Voter server-120b2497-465a-c4b2-0183-8928dbc276bb 120b2497-465a-c4b2-0183-8928dbc276bb}"
testLoggerAdapter verbose: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:13.222Z [INFO]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: entering follower state: follower="Node at 73e1560a-14e4-b7ce-2078-a5c45305953d [Follower]" leader=
    testing.go:149: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:13.222Z [INFO]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: entering follower state: follower="Node at 73e1560a-14e4-b7ce-2078-a5c45305953d [Follower]" leader=
testLoggerAdapter verbose: server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: 2024-01-08T00:31:13.222Z [INFO]  server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: pipelining replication: peer="{Voter server-73e1560a-14e4-b7ce-2078-a5c45305953d 73e1560a-14e4-b7ce-2078-a5c45305953d}"
    testing.go:149: server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: 2024-01-08T00:31:13.222Z [INFO]  server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: pipelining replication: peer="{Voter server-73e1560a-14e4-b7ce-2078-a5c45305953d 73e1560a-14e4-b7ce-2078-a5c45305953d}"
testLoggerAdapter verbose: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:13.223Z [INFO]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: aborting pipeline replication: peer="{Voter server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923 f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923}"
    testing.go:149: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:13.223Z [INFO]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: aborting pipeline replication: peer="{Voter server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923 f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923}"
testLoggerAdapter verbose: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:13.223Z [INFO]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: aborting pipeline replication: peer="{Voter server-120b2497-465a-c4b2-0183-8928dbc276bb 120b2497-465a-c4b2-0183-8928dbc276bb}"
    testing.go:149: server-73e1560a-14e4-b7ce-2078-a5c45305953d: 2024-01-08T00:31:13.223Z [INFO]  server-73e1560a-14e4-b7ce-2078-a5c45305953d: aborting pipeline replication: peer="{Voter server-120b2497-465a-c4b2-0183-8928dbc276bb 120b2497-465a-c4b2-0183-8928dbc276bb}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:13.329Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:13.223024714 +0000 UTC m=+19.269913933 (1 nodes), 1.290673ms from start of poll, 294.815498ms from cluster start. Timeout at 2024-01-08 00:31:13.329202411 +0000 UTC m=+19.376091552, 106.177619ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:13.329Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:13.223024714 +0000 UTC m=+19.269913933 (1 nodes), 1.290673ms from start of poll, 294.815498ms from cluster start. Timeout at 2024-01-08 00:31:13.329202411 +0000 UTC m=+19.376091552, 106.177619ms after stability
testLoggerAdapter verbose: server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: 2024-01-08T00:31:13.329Z [INFO]  server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: aborting pipeline replication: peer="{Voter server-73e1560a-14e4-b7ce-2078-a5c45305953d 73e1560a-14e4-b7ce-2078-a5c45305953d}"
    testing.go:149: server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: 2024-01-08T00:31:13.329Z [INFO]  server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: aborting pipeline replication: peer="{Voter server-73e1560a-14e4-b7ce-2078-a5c45305953d 73e1560a-14e4-b7ce-2078-a5c45305953d}"
testLoggerAdapter verbose: server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: 2024-01-08T00:31:13.329Z [INFO]  server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: aborting pipeline replication: peer="{Voter server-120b2497-465a-c4b2-0183-8928dbc276bb 120b2497-465a-c4b2-0183-8928dbc276bb}"
    testing.go:149: server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: 2024-01-08T00:31:13.329Z [INFO]  server-f70c3fd0-bdc9-2c25-b3ed-e1fe3fa08923: aborting pipeline replication: peer="{Voter server-120b2497-465a-c4b2-0183-8928dbc276bb 120b2497-465a-c4b2-0183-8928dbc276bb}"
--- PASS: TestRaft_LeadershipTransferToBehindServer (0.40s)
=== RUN   TestRaft_LeadershipTransferToItself
testLoggerAdapter verbose: server-9f315898-dfb9-f03d-4e0a-5ad8dc1a928e: 2024-01-08T00:31:13.330Z [INFO]  server-9f315898-dfb9-f03d-4e0a-5ad8dc1a928e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9f315898-dfb9-f03d-4e0a-5ad8dc1a928e Address:9f315898-dfb9-f03d-4e0a-5ad8dc1a928e} {Suffrage:Voter ID:server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6 Address:6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6} {Suffrage:Voter ID:server-eabf3825-ed24-b923-ae5c-dbc482dc87b0 Address:eabf3825-ed24-b923-ae5c-dbc482dc87b0}]"
    testing.go:149: server-9f315898-dfb9-f03d-4e0a-5ad8dc1a928e: 2024-01-08T00:31:13.330Z [INFO]  server-9f315898-dfb9-f03d-4e0a-5ad8dc1a928e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9f315898-dfb9-f03d-4e0a-5ad8dc1a928e Address:9f315898-dfb9-f03d-4e0a-5ad8dc1a928e} {Suffrage:Voter ID:server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6 Address:6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6} {Suffrage:Voter ID:server-eabf3825-ed24-b923-ae5c-dbc482dc87b0 Address:eabf3825-ed24-b923-ae5c-dbc482dc87b0}]"
testLoggerAdapter verbose: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.330Z [INFO]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9f315898-dfb9-f03d-4e0a-5ad8dc1a928e Address:9f315898-dfb9-f03d-4e0a-5ad8dc1a928e} {Suffrage:Voter ID:server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6 Address:6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6} {Suffrage:Voter ID:server-eabf3825-ed24-b923-ae5c-dbc482dc87b0 Address:eabf3825-ed24-b923-ae5c-dbc482dc87b0}]"
    testing.go:149: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.330Z [INFO]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9f315898-dfb9-f03d-4e0a-5ad8dc1a928e Address:9f315898-dfb9-f03d-4e0a-5ad8dc1a928e} {Suffrage:Voter ID:server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6 Address:6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6} {Suffrage:Voter ID:server-eabf3825-ed24-b923-ae5c-dbc482dc87b0 Address:eabf3825-ed24-b923-ae5c-dbc482dc87b0}]"
testLoggerAdapter verbose: server-eabf3825-ed24-b923-ae5c-dbc482dc87b0: 2024-01-08T00:31:13.331Z [INFO]  server-eabf3825-ed24-b923-ae5c-dbc482dc87b0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9f315898-dfb9-f03d-4e0a-5ad8dc1a928e Address:9f315898-dfb9-f03d-4e0a-5ad8dc1a928e} {Suffrage:Voter ID:server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6 Address:6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6} {Suffrage:Voter ID:server-eabf3825-ed24-b923-ae5c-dbc482dc87b0 Address:eabf3825-ed24-b923-ae5c-dbc482dc87b0}]"
    testing.go:149: server-eabf3825-ed24-b923-ae5c-dbc482dc87b0: 2024-01-08T00:31:13.331Z [INFO]  server-eabf3825-ed24-b923-ae5c-dbc482dc87b0: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-9f315898-dfb9-f03d-4e0a-5ad8dc1a928e Address:9f315898-dfb9-f03d-4e0a-5ad8dc1a928e} {Suffrage:Voter ID:server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6 Address:6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6} {Suffrage:Voter ID:server-eabf3825-ed24-b923-ae5c-dbc482dc87b0 Address:eabf3825-ed24-b923-ae5c-dbc482dc87b0}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:13.331Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:13.331Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-9f315898-dfb9-f03d-4e0a-5ad8dc1a928e: 2024-01-08T00:31:13.331Z [INFO]  server-9f315898-dfb9-f03d-4e0a-5ad8dc1a928e: entering follower state: follower="Node at 9f315898-dfb9-f03d-4e0a-5ad8dc1a928e [Follower]" leader=
    testing.go:149: server-9f315898-dfb9-f03d-4e0a-5ad8dc1a928e: 2024-01-08T00:31:13.331Z [INFO]  server-9f315898-dfb9-f03d-4e0a-5ad8dc1a928e: entering follower state: follower="Node at 9f315898-dfb9-f03d-4e0a-5ad8dc1a928e [Follower]" leader=
testLoggerAdapter verbose: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.331Z [INFO]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: entering follower state: follower="Node at 6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6 [Follower]" leader=
    testing.go:149: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.331Z [INFO]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: entering follower state: follower="Node at 6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6 [Follower]" leader=
testLoggerAdapter verbose: server-eabf3825-ed24-b923-ae5c-dbc482dc87b0: 2024-01-08T00:31:13.331Z [INFO]  server-eabf3825-ed24-b923-ae5c-dbc482dc87b0: entering follower state: follower="Node at eabf3825-ed24-b923-ae5c-dbc482dc87b0 [Follower]" leader=
    testing.go:149: server-eabf3825-ed24-b923-ae5c-dbc482dc87b0: 2024-01-08T00:31:13.331Z [INFO]  server-eabf3825-ed24-b923-ae5c-dbc482dc87b0: entering follower state: follower="Node at eabf3825-ed24-b923-ae5c-dbc482dc87b0 [Follower]" leader=
testLoggerAdapter verbose: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.385Z [WARN]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.385Z [WARN]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.386Z [INFO]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: entering candidate state: node="Node at 6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6 [Candidate]" term=2
    testing.go:149: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.386Z [INFO]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: entering candidate state: node="Node at 6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6 [Candidate]" term=2
testLoggerAdapter verbose: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.386Z [INFO]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: election won: tally=2
    testing.go:149: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.386Z [INFO]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: election won: tally=2
testLoggerAdapter verbose: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.386Z [INFO]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: entering leader state: leader="Node at 6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6 [Leader]"
    testing.go:149: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.386Z [INFO]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: entering leader state: leader="Node at 6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6 [Leader]"
testLoggerAdapter verbose: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.386Z [INFO]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: added peer, starting replication: peer=server-9f315898-dfb9-f03d-4e0a-5ad8dc1a928e
    testing.go:149: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.386Z [INFO]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: added peer, starting replication: peer=server-9f315898-dfb9-f03d-4e0a-5ad8dc1a928e
testLoggerAdapter verbose: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.386Z [INFO]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: added peer, starting replication: peer=server-eabf3825-ed24-b923-ae5c-dbc482dc87b0
    testing.go:149: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.386Z [INFO]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: added peer, starting replication: peer=server-eabf3825-ed24-b923-ae5c-dbc482dc87b0
testLoggerAdapter verbose: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.386Z [INFO]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: pipelining replication: peer="{Voter server-eabf3825-ed24-b923-ae5c-dbc482dc87b0 eabf3825-ed24-b923-ae5c-dbc482dc87b0}"
    testing.go:149: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.386Z [INFO]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: pipelining replication: peer="{Voter server-eabf3825-ed24-b923-ae5c-dbc482dc87b0 eabf3825-ed24-b923-ae5c-dbc482dc87b0}"
testLoggerAdapter verbose: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.386Z [INFO]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: pipelining replication: peer="{Voter server-9f315898-dfb9-f03d-4e0a-5ad8dc1a928e 9f315898-dfb9-f03d-4e0a-5ad8dc1a928e}"
    testing.go:149: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.386Z [INFO]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: pipelining replication: peer="{Voter server-9f315898-dfb9-f03d-4e0a-5ad8dc1a928e 9f315898-dfb9-f03d-4e0a-5ad8dc1a928e}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:13.492Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:13.386677732 +0000 UTC m=+19.433566968 (1 nodes), 55.474344ms from start of poll, 55.98127ms from cluster start. Timeout at 2024-01-08 00:31:13.492774381 +0000 UTC m=+19.539663605, 106.096637ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:13.492Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:13.386677732 +0000 UTC m=+19.433566968 (1 nodes), 55.474344ms from start of poll, 55.98127ms from cluster start. Timeout at 2024-01-08 00:31:13.492774381 +0000 UTC m=+19.539663605, 106.096637ms after stability
testLoggerAdapter verbose: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.492Z [INFO]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: cannot transfer leadership to itself
    testing.go:149: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.492Z [INFO]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: cannot transfer leadership to itself
testLoggerAdapter verbose: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.492Z [INFO]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: aborting pipeline replication: peer="{Voter server-eabf3825-ed24-b923-ae5c-dbc482dc87b0 eabf3825-ed24-b923-ae5c-dbc482dc87b0}"
    testing.go:149: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.492Z [INFO]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: aborting pipeline replication: peer="{Voter server-eabf3825-ed24-b923-ae5c-dbc482dc87b0 eabf3825-ed24-b923-ae5c-dbc482dc87b0}"
testLoggerAdapter verbose: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.492Z [INFO]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: aborting pipeline replication: peer="{Voter server-9f315898-dfb9-f03d-4e0a-5ad8dc1a928e 9f315898-dfb9-f03d-4e0a-5ad8dc1a928e}"
    testing.go:149: server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: 2024-01-08T00:31:13.492Z [INFO]  server-6c84fc72-995f-88ff-4a04-4c8b5bb6bfa6: aborting pipeline replication: peer="{Voter server-9f315898-dfb9-f03d-4e0a-5ad8dc1a928e 9f315898-dfb9-f03d-4e0a-5ad8dc1a928e}"
--- PASS: TestRaft_LeadershipTransferToItself (0.16s)
=== RUN   TestRaft_LeadershipTransferLeaderRejectsClientRequests
    raft_test.go:2064: Disabled unreliable test
--- SKIP: TestRaft_LeadershipTransferLeaderRejectsClientRequests (0.00s)
=== RUN   TestRaft_LeadershipTransferLeaderReplicationTimeout
testLoggerAdapter verbose: server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: 2024-01-08T00:31:13.494Z [INFO]  server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1 Address:0a0f1ed7-2d77-7717-673e-e0ecee9910a1} {Suffrage:Voter ID:server-cf8aab85-5f7f-3026-60cc-5df9dc5aa44e Address:cf8aab85-5f7f-3026-60cc-5df9dc5aa44e} {Suffrage:Voter ID:server-259137cf-5d3d-38bf-3995-e1d74d8c4293 Address:259137cf-5d3d-38bf-3995-e1d74d8c4293}]"
    testing.go:149: server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: 2024-01-08T00:31:13.494Z [INFO]  server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1 Address:0a0f1ed7-2d77-7717-673e-e0ecee9910a1} {Suffrage:Voter ID:server-cf8aab85-5f7f-3026-60cc-5df9dc5aa44e Address:cf8aab85-5f7f-3026-60cc-5df9dc5aa44e} {Suffrage:Voter ID:server-259137cf-5d3d-38bf-3995-e1d74d8c4293 Address:259137cf-5d3d-38bf-3995-e1d74d8c4293}]"
testLoggerAdapter verbose: server-cf8aab85-5f7f-3026-60cc-5df9dc5aa44e: 2024-01-08T00:31:13.494Z [INFO]  server-cf8aab85-5f7f-3026-60cc-5df9dc5aa44e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1 Address:0a0f1ed7-2d77-7717-673e-e0ecee9910a1} {Suffrage:Voter ID:server-cf8aab85-5f7f-3026-60cc-5df9dc5aa44e Address:cf8aab85-5f7f-3026-60cc-5df9dc5aa44e} {Suffrage:Voter ID:server-259137cf-5d3d-38bf-3995-e1d74d8c4293 Address:259137cf-5d3d-38bf-3995-e1d74d8c4293}]"
    testing.go:149: server-cf8aab85-5f7f-3026-60cc-5df9dc5aa44e: 2024-01-08T00:31:13.494Z [INFO]  server-cf8aab85-5f7f-3026-60cc-5df9dc5aa44e: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1 Address:0a0f1ed7-2d77-7717-673e-e0ecee9910a1} {Suffrage:Voter ID:server-cf8aab85-5f7f-3026-60cc-5df9dc5aa44e Address:cf8aab85-5f7f-3026-60cc-5df9dc5aa44e} {Suffrage:Voter ID:server-259137cf-5d3d-38bf-3995-e1d74d8c4293 Address:259137cf-5d3d-38bf-3995-e1d74d8c4293}]"
testLoggerAdapter verbose: server-259137cf-5d3d-38bf-3995-e1d74d8c4293: 2024-01-08T00:31:13.494Z [INFO]  server-259137cf-5d3d-38bf-3995-e1d74d8c4293: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1 Address:0a0f1ed7-2d77-7717-673e-e0ecee9910a1} {Suffrage:Voter ID:server-cf8aab85-5f7f-3026-60cc-5df9dc5aa44e Address:cf8aab85-5f7f-3026-60cc-5df9dc5aa44e} {Suffrage:Voter ID:server-259137cf-5d3d-38bf-3995-e1d74d8c4293 Address:259137cf-5d3d-38bf-3995-e1d74d8c4293}]"
    testing.go:149: server-259137cf-5d3d-38bf-3995-e1d74d8c4293: 2024-01-08T00:31:13.494Z [INFO]  server-259137cf-5d3d-38bf-3995-e1d74d8c4293: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1 Address:0a0f1ed7-2d77-7717-673e-e0ecee9910a1} {Suffrage:Voter ID:server-cf8aab85-5f7f-3026-60cc-5df9dc5aa44e Address:cf8aab85-5f7f-3026-60cc-5df9dc5aa44e} {Suffrage:Voter ID:server-259137cf-5d3d-38bf-3995-e1d74d8c4293 Address:259137cf-5d3d-38bf-3995-e1d74d8c4293}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:13.494Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:13.494Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: 2024-01-08T00:31:13.494Z [INFO]  server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: entering follower state: follower="Node at 0a0f1ed7-2d77-7717-673e-e0ecee9910a1 [Follower]" leader=
    testing.go:149: server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: 2024-01-08T00:31:13.494Z [INFO]  server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: entering follower state: follower="Node at 0a0f1ed7-2d77-7717-673e-e0ecee9910a1 [Follower]" leader=
testLoggerAdapter verbose: server-cf8aab85-5f7f-3026-60cc-5df9dc5aa44e: 2024-01-08T00:31:13.494Z [INFO]  server-cf8aab85-5f7f-3026-60cc-5df9dc5aa44e: entering follower state: follower="Node at cf8aab85-5f7f-3026-60cc-5df9dc5aa44e [Follower]" leader=
    testing.go:149: server-cf8aab85-5f7f-3026-60cc-5df9dc5aa44e: 2024-01-08T00:31:13.494Z [INFO]  server-cf8aab85-5f7f-3026-60cc-5df9dc5aa44e: entering follower state: follower="Node at cf8aab85-5f7f-3026-60cc-5df9dc5aa44e [Follower]" leader=
testLoggerAdapter verbose: server-259137cf-5d3d-38bf-3995-e1d74d8c4293: 2024-01-08T00:31:13.494Z [INFO]  server-259137cf-5d3d-38bf-3995-e1d74d8c4293: entering follower state: follower="Node at 259137cf-5d3d-38bf-3995-e1d74d8c4293 [Follower]" leader=
    testing.go:149: server-259137cf-5d3d-38bf-3995-e1d74d8c4293: 2024-01-08T00:31:13.494Z [INFO]  server-259137cf-5d3d-38bf-3995-e1d74d8c4293: entering follower state: follower="Node at 259137cf-5d3d-38bf-3995-e1d74d8c4293 [Follower]" leader=
testLoggerAdapter verbose: server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: 2024-01-08T00:31:13.550Z [WARN]  server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: 2024-01-08T00:31:13.550Z [WARN]  server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: 2024-01-08T00:31:13.550Z [INFO]  server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: entering candidate state: node="Node at 0a0f1ed7-2d77-7717-673e-e0ecee9910a1 [Candidate]" term=2
    testing.go:149: server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: 2024-01-08T00:31:13.550Z [INFO]  server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: entering candidate state: node="Node at 0a0f1ed7-2d77-7717-673e-e0ecee9910a1 [Candidate]" term=2
testLoggerAdapter verbose: server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: 2024-01-08T00:31:13.550Z [INFO]  server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: election won: tally=2
    testing.go:149: server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: 2024-01-08T00:31:13.550Z [INFO]  server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: election won: tally=2
testLoggerAdapter verbose: server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: 2024-01-08T00:31:13.550Z [INFO]  server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: entering leader state: leader="Node at 0a0f1ed7-2d77-7717-673e-e0ecee9910a1 [Leader]"
    testing.go:149: server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: 2024-01-08T00:31:13.550Z [INFO]  server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: entering leader state: leader="Node at 0a0f1ed7-2d77-7717-673e-e0ecee9910a1 [Leader]"
testLoggerAdapter verbose: server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: 2024-01-08T00:31:13.550Z [INFO]  server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: added peer, starting replication: peer=server-cf8aab85-5f7f-3026-60cc-5df9dc5aa44e
    testing.go:149: server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: 2024-01-08T00:31:13.550Z [INFO]  server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: added peer, starting replication: peer=server-cf8aab85-5f7f-3026-60cc-5df9dc5aa44e
testLoggerAdapter verbose: server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: 2024-01-08T00:31:13.550Z [INFO]  server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: added peer, starting replication: peer=server-259137cf-5d3d-38bf-3995-e1d74d8c4293
    testing.go:149: server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: 2024-01-08T00:31:13.550Z [INFO]  server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: added peer, starting replication: peer=server-259137cf-5d3d-38bf-3995-e1d74d8c4293
testLoggerAdapter verbose: server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: 2024-01-08T00:31:13.551Z [INFO]  server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: pipelining replication: peer="{Voter server-259137cf-5d3d-38bf-3995-e1d74d8c4293 259137cf-5d3d-38bf-3995-e1d74d8c4293}"
    testing.go:149: server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: 2024-01-08T00:31:13.551Z [INFO]  server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: pipelining replication: peer="{Voter server-259137cf-5d3d-38bf-3995-e1d74d8c4293 259137cf-5d3d-38bf-3995-e1d74d8c4293}"
testLoggerAdapter verbose: server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: 2024-01-08T00:31:13.551Z [INFO]  server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: pipelining replication: peer="{Voter server-cf8aab85-5f7f-3026-60cc-5df9dc5aa44e cf8aab85-5f7f-3026-60cc-5df9dc5aa44e}"
    testing.go:149: server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: 2024-01-08T00:31:13.551Z [INFO]  server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: pipelining replication: peer="{Voter server-cf8aab85-5f7f-3026-60cc-5df9dc5aa44e cf8aab85-5f7f-3026-60cc-5df9dc5aa44e}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:13.657Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:13.551315809 +0000 UTC m=+19.598204972 (1 nodes), 56.943839ms from start of poll, 57.312961ms from cluster start. Timeout at 2024-01-08 00:31:13.657153156 +0000 UTC m=+19.704042244, 105.837272ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:13.657Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:13.551315809 +0000 UTC m=+19.598204972 (1 nodes), 56.943839ms from start of poll, 57.312961ms from cluster start. Timeout at 2024-01-08 00:31:13.657153156 +0000 UTC m=+19.704042244, 105.837272ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:13.657Z [INFO]  starting stability test: raft-state=Follower
    testing.go:149: cluster: 2024-01-08T00:31:13.657Z [INFO]  starting stability test: raft-state=Follower
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:13.762Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:13.657314862 +0000 UTC m=+19.704203974 (2 nodes), 2.551µs from start of poll, 163.311963ms from cluster start. Timeout at 2024-01-08 00:31:13.76292298 +0000 UTC m=+19.809812855, 105.608881ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:13.762Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:13.657314862 +0000 UTC m=+19.704203974 (2 nodes), 2.551µs from start of poll, 163.311963ms from cluster start. Timeout at 2024-01-08 00:31:13.76292298 +0000 UTC m=+19.809812855, 105.608881ms after stability
testLoggerAdapter verbose: server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: 2024-01-08T00:31:14.144Z [INFO]  server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: aborting pipeline replication: peer="{Voter server-259137cf-5d3d-38bf-3995-e1d74d8c4293 259137cf-5d3d-38bf-3995-e1d74d8c4293}"
    testing.go:149: server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: 2024-01-08T00:31:14.144Z [INFO]  server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: aborting pipeline replication: peer="{Voter server-259137cf-5d3d-38bf-3995-e1d74d8c4293 259137cf-5d3d-38bf-3995-e1d74d8c4293}"
testLoggerAdapter verbose: server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: 2024-01-08T00:31:14.144Z [INFO]  server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: aborting pipeline replication: peer="{Voter server-cf8aab85-5f7f-3026-60cc-5df9dc5aa44e cf8aab85-5f7f-3026-60cc-5df9dc5aa44e}"
    testing.go:149: server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: 2024-01-08T00:31:14.144Z [INFO]  server-0a0f1ed7-2d77-7717-673e-e0ecee9910a1: aborting pipeline replication: peer="{Voter server-cf8aab85-5f7f-3026-60cc-5df9dc5aa44e cf8aab85-5f7f-3026-60cc-5df9dc5aa44e}"
--- PASS: TestRaft_LeadershipTransferLeaderReplicationTimeout (0.65s)
=== RUN   TestRaft_LeadershipTransferStopRightAway
--- PASS: TestRaft_LeadershipTransferStopRightAway (0.00s)
=== RUN   TestRaft_GetConfigurationNoBootstrap
testLoggerAdapter verbose: server-583c32b5-8e2e-809d-cc03-feff3264cb4d: 2024-01-08T00:31:14.146Z [INFO]  server-583c32b5-8e2e-809d-cc03-feff3264cb4d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-583c32b5-8e2e-809d-cc03-feff3264cb4d Address:583c32b5-8e2e-809d-cc03-feff3264cb4d} {Suffrage:Voter ID:server-8942f1da-0549-78a5-321a-3384b25815a3 Address:8942f1da-0549-78a5-321a-3384b25815a3}]"
    testing.go:149: server-583c32b5-8e2e-809d-cc03-feff3264cb4d: 2024-01-08T00:31:14.146Z [INFO]  server-583c32b5-8e2e-809d-cc03-feff3264cb4d: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-583c32b5-8e2e-809d-cc03-feff3264cb4d Address:583c32b5-8e2e-809d-cc03-feff3264cb4d} {Suffrage:Voter ID:server-8942f1da-0549-78a5-321a-3384b25815a3 Address:8942f1da-0549-78a5-321a-3384b25815a3}]"
testLoggerAdapter verbose: server-583c32b5-8e2e-809d-cc03-feff3264cb4d: 2024-01-08T00:31:14.146Z [INFO]  server-583c32b5-8e2e-809d-cc03-feff3264cb4d: entering follower state: follower="Node at 583c32b5-8e2e-809d-cc03-feff3264cb4d [Follower]" leader=
    testing.go:149: server-583c32b5-8e2e-809d-cc03-feff3264cb4d: 2024-01-08T00:31:14.146Z [INFO]  server-583c32b5-8e2e-809d-cc03-feff3264cb4d: entering follower state: follower="Node at 583c32b5-8e2e-809d-cc03-feff3264cb4d [Follower]" leader=
testLoggerAdapter verbose: server-8942f1da-0549-78a5-321a-3384b25815a3: 2024-01-08T00:31:14.146Z [INFO]  server-8942f1da-0549-78a5-321a-3384b25815a3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-583c32b5-8e2e-809d-cc03-feff3264cb4d Address:583c32b5-8e2e-809d-cc03-feff3264cb4d} {Suffrage:Voter ID:server-8942f1da-0549-78a5-321a-3384b25815a3 Address:8942f1da-0549-78a5-321a-3384b25815a3}]"
    testing.go:149: server-8942f1da-0549-78a5-321a-3384b25815a3: 2024-01-08T00:31:14.146Z [INFO]  server-8942f1da-0549-78a5-321a-3384b25815a3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-583c32b5-8e2e-809d-cc03-feff3264cb4d Address:583c32b5-8e2e-809d-cc03-feff3264cb4d} {Suffrage:Voter ID:server-8942f1da-0549-78a5-321a-3384b25815a3 Address:8942f1da-0549-78a5-321a-3384b25815a3}]"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:14.146Z [INFO]  starting stability test: raft-state=Follower
    testing.go:149: cluster: 2024-01-08T00:31:14.146Z [INFO]  starting stability test: raft-state=Follower
testLoggerAdapter verbose: server-8942f1da-0549-78a5-321a-3384b25815a3: 2024-01-08T00:31:14.146Z [INFO]  server-8942f1da-0549-78a5-321a-3384b25815a3: entering follower state: follower="Node at 8942f1da-0549-78a5-321a-3384b25815a3 [Follower]" leader=
    testing.go:149: server-8942f1da-0549-78a5-321a-3384b25815a3: 2024-01-08T00:31:14.146Z [INFO]  server-8942f1da-0549-78a5-321a-3384b25815a3: entering follower state: follower="Node at 8942f1da-0549-78a5-321a-3384b25815a3 [Follower]" leader=
testLoggerAdapter verbose: server-8942f1da-0549-78a5-321a-3384b25815a3: 2024-01-08T00:31:14.224Z [WARN]  server-8942f1da-0549-78a5-321a-3384b25815a3: heartbeat timeout reached, starting election: last-leader=
    testing.go:149: server-8942f1da-0549-78a5-321a-3384b25815a3: 2024-01-08T00:31:14.224Z [WARN]  server-8942f1da-0549-78a5-321a-3384b25815a3: heartbeat timeout reached, starting election: last-leader=
testLoggerAdapter verbose: server-8942f1da-0549-78a5-321a-3384b25815a3: 2024-01-08T00:31:14.224Z [INFO]  server-8942f1da-0549-78a5-321a-3384b25815a3: entering candidate state: node="Node at 8942f1da-0549-78a5-321a-3384b25815a3 [Candidate]" term=2
    testing.go:149: server-8942f1da-0549-78a5-321a-3384b25815a3: 2024-01-08T00:31:14.224Z [INFO]  server-8942f1da-0549-78a5-321a-3384b25815a3: entering candidate state: node="Node at 8942f1da-0549-78a5-321a-3384b25815a3 [Candidate]" term=2
testLoggerAdapter verbose: server-8942f1da-0549-78a5-321a-3384b25815a3: 2024-01-08T00:31:14.224Z [INFO]  server-8942f1da-0549-78a5-321a-3384b25815a3: election won: tally=2
    testing.go:149: server-8942f1da-0549-78a5-321a-3384b25815a3: 2024-01-08T00:31:14.224Z [INFO]  server-8942f1da-0549-78a5-321a-3384b25815a3: election won: tally=2
testLoggerAdapter verbose: server-8942f1da-0549-78a5-321a-3384b25815a3: 2024-01-08T00:31:14.224Z [INFO]  server-8942f1da-0549-78a5-321a-3384b25815a3: entering leader state: leader="Node at 8942f1da-0549-78a5-321a-3384b25815a3 [Leader]"
    testing.go:149: server-8942f1da-0549-78a5-321a-3384b25815a3: 2024-01-08T00:31:14.224Z [INFO]  server-8942f1da-0549-78a5-321a-3384b25815a3: entering leader state: leader="Node at 8942f1da-0549-78a5-321a-3384b25815a3 [Leader]"
testLoggerAdapter verbose: server-8942f1da-0549-78a5-321a-3384b25815a3: 2024-01-08T00:31:14.224Z [INFO]  server-8942f1da-0549-78a5-321a-3384b25815a3: added peer, starting replication: peer=server-583c32b5-8e2e-809d-cc03-feff3264cb4d
    testing.go:149: server-8942f1da-0549-78a5-321a-3384b25815a3: 2024-01-08T00:31:14.224Z [INFO]  server-8942f1da-0549-78a5-321a-3384b25815a3: added peer, starting replication: peer=server-583c32b5-8e2e-809d-cc03-feff3264cb4d
testLoggerAdapter verbose: server-8942f1da-0549-78a5-321a-3384b25815a3: 2024-01-08T00:31:14.224Z [INFO]  server-8942f1da-0549-78a5-321a-3384b25815a3: pipelining replication: peer="{Voter server-583c32b5-8e2e-809d-cc03-feff3264cb4d 583c32b5-8e2e-809d-cc03-feff3264cb4d}"
    testing.go:149: server-8942f1da-0549-78a5-321a-3384b25815a3: 2024-01-08T00:31:14.224Z [INFO]  server-8942f1da-0549-78a5-321a-3384b25815a3: pipelining replication: peer="{Voter server-583c32b5-8e2e-809d-cc03-feff3264cb4d 583c32b5-8e2e-809d-cc03-feff3264cb4d}"
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:14.330Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:14.22477102 +0000 UTC m=+20.271660107 (1 nodes), 78.189388ms from start of poll, 78.662146ms from cluster start. Timeout at 2024-01-08 00:31:14.330707984 +0000 UTC m=+20.377597221, 105.937114ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:14.330Z [INFO]  stable state for Follower reached at 2024-01-08 00:31:14.22477102 +0000 UTC m=+20.271660107 (1 nodes), 78.189388ms from start of poll, 78.662146ms from cluster start. Timeout at 2024-01-08 00:31:14.330707984 +0000 UTC m=+20.377597221, 105.937114ms after stability
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:14.330Z [INFO]  starting stability test: raft-state=Leader
    testing.go:149: cluster: 2024-01-08T00:31:14.330Z [INFO]  starting stability test: raft-state=Leader
testLoggerAdapter verbose: cluster: 2024-01-08T00:31:14.436Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:14.330922676 +0000 UTC m=+20.377811867 (1 nodes), 2.64µs from start of poll, 184.813906ms from cluster start. Timeout at 2024-01-08 00:31:14.436783734 +0000 UTC m=+20.483672847, 105.86098ms after stability
    testing.go:149: cluster: 2024-01-08T00:31:14.436Z [INFO]  stable state for Leader reached at 2024-01-08 00:31:14.330922676 +0000 UTC m=+20.377811867 (1 nodes), 2.64µs from start of poll, 184.813906ms from cluster start. Timeout at 2024-01-08 00:31:14.436783734 +0000 UTC m=+20.483672847, 105.86098ms after stability
testLoggerAdapter verbose: server-8942f1da-0549-78a5-321a-3384b25815a3: 2024-01-08T00:31:14.448Z [INFO]  server-8942f1da-0549-78a5-321a-3384b25815a3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-583c32b5-8e2e-809d-cc03-feff3264cb4d Address:583c32b5-8e2e-809d-cc03-feff3264cb4d} {Suffrage:Voter ID:server-8942f1da-0549-78a5-321a-3384b25815a3 Address:8942f1da-0549-78a5-321a-3384b25815a3}]"
    testing.go:149: server-8942f1da-0549-78a5-321a-3384b25815a3: 2024-01-08T00:31:14.448Z [INFO]  server-8942f1da-0549-78a5-321a-3384b25815a3: initial configuration: index=1 servers="[{Suffrage:Voter ID:server-583c32b5-8e2e-809d-cc03-feff3264cb4d Address:583c32b5-8e2e-809d-cc03-feff3264cb4d} {Suffrage:Voter ID:server-8942f1da-0549-78a5-321a-3384b25815a3 Address:8942f1da-0549-78a5-321a-3384b25815a3}]"
testLoggerAdapter verbose: server-8942f1da-0549-78a5-321a-3384b25815a3: 2024-01-08T00:31:14.448Z [INFO]  server-8942f1da-0549-78a5-321a-3384b25815a3: aborting pipeline replication: peer="{Voter server-583c32b5-8e2e-809d-cc03-feff3264cb4d 583c32b5-8e2e-809d-cc03-feff3264cb4d}"
    testing.go:149: server-8942f1da-0549-78a5-321a-3384b25815a3: 2024-01-08T00:31:14.448Z [INFO]  server-8942f1da-0549-78a5-321a-3384b25815a3: aborting pipeline replication: peer="{Voter server-583c32b5-8e2e-809d-cc03-feff3264cb4d 583c32b5-8e2e-809d-cc03-feff3264cb4d}"
--- PASS: TestRaft_GetConfigurationNoBootstrap (0.30s)
=== RUN   TestTCPTransport_BadAddr
--- PASS: TestTCPTransport_BadAddr (0.00s)
=== RUN   TestTCPTransport_WithAdvertise
--- PASS: TestTCPTransport_WithAdvertise (0.00s)
=== RUN   TestTransport_StartStop
--- PASS: TestTransport_StartStop (0.00s)
=== RUN   TestTransport_AppendEntries
--- PASS: TestTransport_AppendEntries (0.00s)
=== RUN   TestTransport_AppendEntriesPipeline
--- PASS: TestTransport_AppendEntriesPipeline (0.00s)
=== RUN   TestTransport_RequestVote
--- PASS: TestTransport_RequestVote (0.00s)
=== RUN   TestTransport_InstallSnapshot
--- PASS: TestTransport_InstallSnapshot (0.00s)
=== RUN   TestTransport_EncodeDecode
--- PASS: TestTransport_EncodeDecode (0.00s)
=== RUN   TestRandomTimeout
--- PASS: TestRandomTimeout (0.00s)
=== RUN   TestNewSeed
--- PASS: TestNewSeed (0.00s)
=== RUN   TestRandomTimeout_NoTime
--- PASS: TestRandomTimeout_NoTime (0.00s)
=== RUN   TestMin
--- PASS: TestMin (0.00s)
=== RUN   TestMax
--- PASS: TestMax (0.00s)
=== RUN   TestGenerateUUID
--- PASS: TestGenerateUUID (0.01s)
=== RUN   TestBackoff
--- PASS: TestBackoff (0.00s)
PASS
ok  	github.com/hashicorp/raft	20.524s
?   	github.com/hashicorp/raft/bench	[no test files]
   create-stamp debian/debhelper-build-stamp
 fakeroot debian/rules binary
dh binary --buildsystem=golang --with=golang
   dh_testroot -O--buildsystem=golang
   dh_prep -O--buildsystem=golang
   dh_auto_install --destdir=debian/golang-github-hashicorp-raft-dev/ -O--buildsystem=golang
   dh_installdocs -O--buildsystem=golang
   dh_installchangelogs -O--buildsystem=golang
   dh_installinit -O--buildsystem=golang
   dh_installsystemduser -O--buildsystem=golang
   dh_perl -O--buildsystem=golang
   dh_link -O--buildsystem=golang
   dh_strip_nondeterminism -O--buildsystem=golang
   dh_compress -O--buildsystem=golang
   dh_fixperms -O--buildsystem=golang
   dh_missing -O--buildsystem=golang
   dh_installdeb -O--buildsystem=golang
   dh_golang -O--buildsystem=golang
   dh_gencontrol -O--buildsystem=golang
dpkg-gencontrol: warning: Depends field of package golang-github-hashicorp-raft-dev: substitution variable ${shlibs:Depends} used, but is not defined
   dh_md5sums -O--buildsystem=golang
   dh_builddeb -O--buildsystem=golang
dpkg-deb: building package 'golang-github-hashicorp-raft-dev' in '../golang-github-hashicorp-raft-dev_1.1.2-1_all.deb'.
 dpkg-genbuildinfo --build=binary -O../golang-github-hashicorp-raft_1.1.2-1_i386.buildinfo
 dpkg-genchanges --build=binary -O../golang-github-hashicorp-raft_1.1.2-1_i386.changes
dpkg-genchanges: info: binary-only upload (no source code included)
 dpkg-source --after-build .
dpkg-buildpackage: info: binary-only upload (no source included)
dpkg-genchanges: info: including full source code in upload
I: copying local configuration
I: user script /srv/workspace/pbuilder/15535/tmp/hooks/B01_cleanup starting
I: user script /srv/workspace/pbuilder/15535/tmp/hooks/B01_cleanup finished
I: unmounting dev/ptmx filesystem
I: unmounting dev/pts filesystem
I: unmounting dev/shm filesystem
I: unmounting proc filesystem
I: unmounting sys filesystem
I: cleaning the build env 
I: removing directory /srv/workspace/pbuilder/15535 and its subdirectories
I: Current time: Mon Jan  8 14:31:23 +14 2024
I: pbuilder-time-stamp: 1704673883