==> Building on ceruledge ==> Checking for remote environment... ==> Syncing package to remote host... sending incremental file list created directory packages/solr ./ PKGBUILD 7,307 100% 0.00kB/s 0:00:00 7,307 100% 0.00kB/s 0:00:00 (xfr#1, to-chk=25/27) allow-using-system-gradle.patch 1,429 100% 1.36MB/s 0:00:00 1,429 100% 1.36MB/s 0:00:00 (xfr#2, to-chk=24/27) skip-checks-for-git-repo.patch 2,131 100% 2.03MB/s 0:00:00 2,131 100% 2.03MB/s 0:00:00 (xfr#3, to-chk=23/27) skip-failing-test.patch 1,047 100% 1022.46kB/s 0:00:00 1,047 100% 1022.46kB/s 0:00:00 (xfr#4, to-chk=22/27) solr-9.2.1-1.log 779 100% 760.74kB/s 0:00:00 779 100% 760.74kB/s 0:00:00 (xfr#5, to-chk=21/27) solr.service 962 100% 939.45kB/s 0:00:00 962 100% 939.45kB/s 0:00:00 (xfr#6, to-chk=20/27) solr.sysusers 32 100% 31.25kB/s 0:00:00 32 100% 31.25kB/s 0:00:00 (xfr#7, to-chk=19/27) solr.tmpfiles 161 100% 157.23kB/s 0:00:00 161 100% 157.23kB/s 0:00:00 (xfr#8, to-chk=18/27) support-reproducible-builds.patch 1,668 100% 1.59MB/s 0:00:00 1,668 100% 1.59MB/s 0:00:00 (xfr#9, to-chk=17/27) keys/ keys/pgp/ keys/pgp/2085660D9C1FCCACC4A479A3BF160FF14992A24C.asc 3,078 100% 2.94MB/s 0:00:00 3,078 100% 2.94MB/s 0:00:00 (xfr#10, to-chk=14/27) keys/pgp/3558857D1F5754B78C7F8B5A71A45A3D0D8D0B93.asc 1,672 100% 1.59MB/s 0:00:00 1,672 100% 1.59MB/s 0:00:00 (xfr#11, to-chk=13/27) keys/pgp/38DA0C3CE8181703A08E4D57377C3BA26AD29C0A.asc 3,139 100% 2.99MB/s 0:00:00 3,139 100% 2.99MB/s 0:00:00 (xfr#12, to-chk=12/27) keys/pgp/50E3EE1C91C7E0CB4DFB007B369424FC98F3F6EC.asc 3,171 100% 1.51MB/s 0:00:00 3,171 100% 1.51MB/s 0:00:00 (xfr#13, to-chk=11/27) keys/pgp/7D8D90F8F64F23077AC87CF7CB77CB79928BB0EC.asc 2,448 100% 1.17MB/s 0:00:00 2,448 100% 1.17MB/s 0:00:00 (xfr#14, to-chk=10/27) keys/pgp/81D3EB0408B4E1EB10AF443BA4F4C886B29BC2F4.asc 3,163 100% 1.51MB/s 0:00:00 3,163 100% 1.51MB/s 0:00:00 (xfr#15, to-chk=9/27) keys/pgp/86EDB9C33B8517228E88A8F93E48C0C6EF362B9E.asc 3,090 100% 1.47MB/s 0:00:00 3,090 100% 1.47MB/s 0:00:00 (xfr#16, to-chk=8/27) keys/pgp/902CC51935C140BF820230961FD5295281436075.asc 3,167 100% 1.51MB/s 0:00:00 3,167 100% 1.51MB/s 0:00:00 (xfr#17, to-chk=7/27) keys/pgp/9722F25F650057E26C803B60A6D064D833B3A969.asc 3,159 100% 1.00MB/s 0:00:00 3,159 100% 1.00MB/s 0:00:00 (xfr#18, to-chk=6/27) keys/pgp/C3E7CBD9B9FE2B419EB14B89612B39A5BC981763.asc 7,770 100% 2.47MB/s 0:00:00 7,770 100% 2.47MB/s 0:00:00 (xfr#19, to-chk=5/27) keys/pgp/CFCE5FBB920C3C745CEEE084C38FF5EC3FCFDB3E.asc 3,090 100% 1005.86kB/s 0:00:00 3,090 100% 1005.86kB/s 0:00:00 (xfr#20, to-chk=4/27) keys/pgp/E58A6F4D5B2B48AC66D5E53BD4F181881A42F9E6.asc 3,155 100% 770.26kB/s 0:00:00 3,155 100% 770.26kB/s 0:00:00 (xfr#21, to-chk=3/27) keys/pgp/E6E21FFCDCEA14C95910EA65051A0FAF76BC6507.asc 3,094 100% 755.37kB/s 0:00:00 3,094 100% 755.37kB/s 0:00:00 (xfr#22, to-chk=2/27) keys/pgp/FBC25D7E1712025294FE66590A6AA179B9BBF45E.asc 3,163 100% 772.22kB/s 0:00:00 3,163 100% 772.22kB/s 0:00:00 (xfr#23, to-chk=1/27) keys/pgp/FDB3D11D716BB32ACF8C93AB919B21537AA80271.asc 3,167 100% 773.19kB/s 0:00:00 3,167 100% 773.19kB/s 0:00:00 (xfr#24, to-chk=0/27) sent 46,432 bytes received 527 bytes 93,918.00 bytes/sec total size is 64,459 speedup is 1.37 ==> Ensuring required PGP keys are present...  -> Checking for 2085660D9C1FCCACC4A479A3BF160FF14992A24C...  -> Checking for E58A6F4D5B2B48AC66D5E53BD4F181881A42F9E6...  -> Checking for 81D3EB0408B4E1EB10AF443BA4F4C886B29BC2F4...  -> Checking for 86EDB9C33B8517228E88A8F93E48C0C6EF362B9E...  -> Checking for 38DA0C3CE8181703A08E4D57377C3BA26AD29C0A...  -> Checking for 50E3EE1C91C7E0CB4DFB007B369424FC98F3F6EC...  -> Checking for 902CC51935C140BF820230961FD5295281436075...  -> Checking for 7D8D90F8F64F23077AC87CF7CB77CB79928BB0EC...  -> Checking for CFCE5FBB920C3C745CEEE084C38FF5EC3FCFDB3E...  -> Checking for FBC25D7E1712025294FE66590A6AA179B9BBF45E...  -> Checking for 9722F25F650057E26C803B60A6D064D833B3A969...  -> Checking for E6E21FFCDCEA14C95910EA65051A0FAF76BC6507...  -> Checking for 3558857D1F5754B78C7F8B5A71A45A3D0D8D0B93...  -> Checking for C3E7CBD9B9FE2B419EB14B89612B39A5BC981763...  -> Checking for FDB3D11D716BB32ACF8C93AB919B21537AA80271... ==> Running extra-riscv64-build -- -d /home/felix/packages/riscv64-pkg-cache:/var/cache/pacman/pkg -l root2 on remote host... [?25l:: Synchronizing package databases... core downloading... extra downloading... :: Starting full system upgrade... there is nothing to do [?25h==> Building in chroot for [extra] (riscv64)... ==> Synchronizing chroot copy [/var/lib/archbuild/extra-riscv64/root] -> [root2]...done ==> Making package: solr 9.2.1-1 (Tue Oct 3 06:41:46 2023) ==> Retrieving sources...  -> Downloading solr-9.2.1-src.tgz... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0 0 85.7M 0 7941 0 0 3628 0 6:53:04 0:00:02 6:53:02 3627 0 85.7M 0 249k 0 0 73018 0 0:20:31 0:00:03 0:20:28 73021 0 85.7M 0 343k 0 0 80207 0 0:18:41 0:00:04 0:18:37 80205 0 85.7M 0 515k 0 0 97k 0 0:14:57 0:00:05 0:14:52 101k 0 85.7M 0 828k 0 0 123k 0 0:11:49 0:00:06 0:11:43 155k 1 85.7M 1 1062k 0 0 143k 0 0:10:12 0:00:07 0:10:05 202k 1 85.7M 1 1343k 0 0 163k 0 0:08:55 0:00:08 0:08:47 233k 2 85.7M 2 1765k 0 0 190k 0 0:07:40 0:00:09 0:07:31 292k 2 85.7M 2 2140k 0 0 208k 0 0:07:00 0:00:10 0:06:50 325k 3 85.7M 3 2851k 0 0 253k 0 0:05:45 0:00:11 0:05:34 445k 4 85.7M 4 4046k 0 0 332k 0 0:04:24 0:00:12 0:04:12 625k 6 85.7M 6 5398k 0 0 408k 0 0:03:35 0:00:13 0:03:22 806k 7 85.7M 7 6656k 0 0 469k 0 0:03:06 0:00:14 0:02:52 993k 9 85.7M 9 7976k 0 0 525k 0 0:02:47 0:00:15 0:02:32 1187k 10 85.7M 10 9281k 0 0 573k 0 0:02:33 0:00:16 0:02:17 1299k 12 85.7M 12 10.8M 0 0 648k 0 0:02:15 0:00:17 0:01:58 1415k 15 85.7M 15 13.0M 0 0 736k 0 0:01:59 0:00:18 0:01:41 1610k 18 85.7M 18 15.8M 0 0 837k 0 0:01:44 0:00:19 0:01:25 1840k 21 85.7M 21 18.3M 0 0 926k 0 0:01:34 0:00:20 0:01:14 2125k 23 85.7M 23 20.3M 0 0 983k 0 0:01:29 0:00:21 0:01:08 2302k 26 85.7M 26 23.1M 0 0 1064k 0 0:01:22 0:00:22 0:01:00 2486k 29 85.7M 29 25.2M 0 0 1116k 0 0:01:18 0:00:23 0:00:55 2501k 32 85.7M 32 27.6M 0 0 1172k 0 0:01:14 0:00:24 0:00:50 2518k 34 85.7M 34 29.8M 0 0 1213k 0 0:01:12 0:00:25 0:00:47 2392k 35 85.7M 35 30.7M 0 0 1201k 0 0:01:13 0:00:26 0:00:47 2126k 38 85.7M 38 32.8M 0 0 1239k 0 0:01:10 0:00:27 0:00:43 2028k 41 85.7M 41 35.4M 0 0 1286k 0 0:01:08 0:00:28 0:00:40 2075k 43 85.7M 43 37.5M 0 0 1316k 0 0:01:06 0:00:29 0:00:37 2003k 45 85.7M 45 39.1M 0 0 1329k 0 0:01:06 0:00:30 0:00:36 1916k 48 85.7M 48 41.7M 0 0 1367k 0 0:01:04 0:00:31 0:00:33 2227k 51 85.7M 51 44.0M 0 0 1402k 0 0:01:02 0:00:32 0:00:30 2285k 53 85.7M 53 45.9M 0 0 1417k 0 0:01:01 0:00:33 0:00:28 2156k 54 85.7M 54 47.0M 0 0 1408k 0 0:01:02 0:00:34 0:00:28 1957k 56 85.7M 56 48.1M 0 0 1402k 0 0:01:02 0:00:35 0:00:27 1843k 58 85.7M 58 50.5M 0 0 1431k 0 0:01:01 0:00:36 0:00:25 1842k 61 85.7M 61 52.8M 0 0 1456k 0 0:01:00 0:00:37 0:00:23 1804k 64 85.7M 64 55.2M 0 0 1482k 0 0:00:59 0:00:38 0:00:21 1917k 67 85.7M 67 58.0M 0 0 1516k 0 0:00:57 0:00:39 0:00:18 2255k 71 85.7M 71 60.9M 0 0 1552k 0 0:00:56 0:00:40 0:00:16 2604k 74 85.7M 74 63.7M 0 0 1586k 0 0:00:55 0:00:41 0:00:14 2703k 77 85.7M 77 66.5M 0 0 1615k 0 0:00:54 0:00:42 0:00:12 2801k 81 85.7M 81 69.4M 0 0 1647k 0 0:00:53 0:00:43 0:00:10 2907k 84 85.7M 84 72.6M 0 0 1684k 0 0:00:52 0:00:44 0:00:08 3000k 88 85.7M 88 76.1M 0 0 1726k 0 0:00:50 0:00:45 0:00:05 3119k 92 85.7M 92 79.5M 0 0 1764k 0 0:00:49 0:00:46 0:00:03 3235k 97 85.7M 97 83.5M 0 0 1813k 0 0:00:48 0:00:47 0:00:01 3484k 100 85.7M 100 85.7M 0 0 1841k 0 0:00:47 0:00:47 --:--:-- 3700k  -> Downloading solr-9.2.1-src.tgz.asc... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 853 100 853 0 0 724 0 0:00:01 0:00:01 --:--:-- 725  -> Found solr.service  -> Found solr.sysusers  -> Found solr.tmpfiles  -> Found support-reproducible-builds.patch  -> Found skip-failing-test.patch  -> Found skip-checks-for-git-repo.patch  -> Found allow-using-system-gradle.patch ==> Validating source files with sha512sums... solr-9.2.1-src.tgz ... Passed solr-9.2.1-src.tgz.asc ... Skipped solr.service ... Passed solr.sysusers ... Passed solr.tmpfiles ... Passed support-reproducible-builds.patch ... Passed skip-failing-test.patch ... Passed skip-checks-for-git-repo.patch ... Passed allow-using-system-gradle.patch ... Passed ==> Validating source files with b2sums... solr-9.2.1-src.tgz ... Passed solr-9.2.1-src.tgz.asc ... Skipped solr.service ... Passed solr.sysusers ... Passed solr.tmpfiles ... Passed support-reproducible-builds.patch ... Passed skip-failing-test.patch ... Passed skip-checks-for-git-repo.patch ... Passed allow-using-system-gradle.patch ... Passed ==> Verifying source file signatures with gpg... solr-9.2.1-src.tgz ... Passed ==> Making package: solr 9.2.1-1 (Tue Oct 3 06:43:01 2023) ==> Checking runtime dependencies... ==> Installing missing dependencies... [?25lresolving dependencies... looking for conflicting packages... warning: dependency cycle detected: warning: harfbuzz will be installed before its freetype2 dependency Package (15) New Version Net Change extra/freetype2 2.13.2-1 1.51 MiB extra/giflib 5.2.1-2 0.22 MiB extra/graphite 1:1.3.14-3 0.17 MiB extra/harfbuzz 8.2.1-1 3.62 MiB extra/java-runtime-common 3-5 0.01 MiB extra/jbigkit 2.1-7 0.13 MiB extra/jre11-openjdk-headless 11.0.20.u8-2 138.33 MiB extra/lcms2 2.15-1 0.58 MiB extra/libjpeg-turbo 3.0.0-1 1.79 MiB extra/libnet 1:1.1.6-1 0.28 MiB extra/libpng 1.6.40-2 0.51 MiB extra/libtiff 4.6.0-1 4.95 MiB core/nspr 4.35-1 0.67 MiB core/nss 3.93-1 4.21 MiB extra/jre11-openjdk 11.0.20.u8-2 0.49 MiB Total Installed Size: 157.48 MiB :: Proceed with installation? [Y/n] checking keyring... checking package integrity... loading package files... checking for file conflicts... :: Processing package changes... installing java-runtime-common... For the complete set of Java binaries to be available in your PATH, you need to re-login or source /etc/profile.d/jre.sh Please note that this package does not support forcing JAVA_HOME as former package java-common did installing nspr... installing nss... installing libjpeg-turbo... Optional dependencies for libjpeg-turbo java-runtime>11: for TurboJPEG Java wrapper installing jbigkit... installing libtiff... Optional dependencies for libtiff freeglut: for using tiffgt installing lcms2... installing libnet... installing libpng... installing graphite... Optional dependencies for graphite graphite-docs: Documentation installing harfbuzz... Optional dependencies for harfbuzz harfbuzz-utils: utilities installing freetype2... installing jre11-openjdk-headless... Optional dependencies for jre11-openjdk-headless java-rhino: for some JavaScript support installing giflib... installing jre11-openjdk... when you use a non-reparenting window manager, set _JAVA_AWT_WM_NONREPARENTING=1 in /etc/profile.d/jre.sh Optional dependencies for jre11-openjdk alsa-lib: for basic sound support gtk2: for the Gtk+ 2 look and feel - desktop usage gtk3: for the Gtk+ 3 look and feel - desktop usage [?25h==> Checking buildtime dependencies... ==> Installing missing dependencies... [?25lresolving dependencies... looking for conflicting packages... Package (9) New Version Net Change Download Size extra/hicolor-icon-theme 0.17-3 0.05 MiB extra/java-environment-common 3-5 0.00 MiB extra/perl-error 0.17029-5 0.04 MiB extra/perl-mailtools 2.21-7 0.10 MiB extra/perl-timedate 2.33-5 0.08 MiB extra/git 2.42.0-1 24.79 MiB extra/gradle 8.3-1 138.08 MiB 121.81 MiB extra/groovy 4.0.12-1 23.99 MiB 21.18 MiB extra/jdk11-openjdk 11.0.20.u8-2 247.04 MiB Total Download Size: 142.99 MiB Total Installed Size: 434.17 MiB :: Proceed with installation? [Y/n] :: Retrieving packages... gradle-8.3-1-any downloading... groovy-4.0.12-1-any downloading... checking keyring... checking package integrity... loading package files... checking for file conflicts... :: Processing package changes... installing java-environment-common... installing hicolor-icon-theme... installing jdk11-openjdk... installing gradle... Optional dependencies for gradle gradle-doc: gradle documentation gradle-src: gradle sources installing perl-error... installing perl-timedate... installing perl-mailtools... installing git... Optional dependencies for git tk: gitk and git gui openssh: ssh transport and crypto perl-libwww: git svn perl-term-readkey: git svn and interactive.singlekey setting perl-io-socket-ssl: git send-email TLS support perl-authen-sasl: git send-email TLS support perl-mediawiki-api: git mediawiki support perl-datetime-format-iso8601: git mediawiki support perl-lwp-protocol-https: git mediawiki https support perl-cgi: gitweb (web interface) support python: git svn & git p4 subversion: git svn org.freedesktop.secrets: keyring credential helper libsecret: libsecret credential helper [installed] installing groovy... Optional dependencies for groovy groovy-docs: html and pdf documentation for Groovy :: Running post-transaction hooks... (1/1) Warn about old perl modules [?25h==> Retrieving sources...  -> Found solr-9.2.1-src.tgz  -> Found solr-9.2.1-src.tgz.asc  -> Found solr.service  -> Found solr.sysusers  -> Found solr.tmpfiles  -> Found support-reproducible-builds.patch  -> Found skip-failing-test.patch  -> Found skip-checks-for-git-repo.patch  -> Found allow-using-system-gradle.patch ==> WARNING: Skipping all source file integrity checks. ==> Extracting sources...  -> Extracting solr-9.2.1-src.tgz with bsdtar ==> Starting prepare()... patching file gradle/globals.gradle patching file gradle/java/jar-manifest.gradle patching file solr/modules/extraction/src/test/org/apache/solr/handler/extraction/TestXLSXResponseWriter.java patching file solr/distribution/source-release.gradle patching file gradle/validation/precommit.gradle patching file solr/distribution/build.gradle ==> Starting build()... Downloading gradle-wrapper.jar from https://raw.githubusercontent.com/gradle/gradle/v7.6.0/gradle/wrapper/gradle-wrapper.jar Generating gradle.properties Downloading https://services.gradle.org/distributions/gradle-7.6-bin.zip ...........10%............20%...........30%............40%............50%...........60%............70%............80%...........90%............100% Welcome to Gradle 7.6! Here are the highlights of this release: - Added support for Java 19. - Introduced `--rerun` flag for individual task rerun. - Improved dependency block for test suites to be strongly typed. - Added a pluggable system for Java toolchains provisioning. For more details see https://docs.gradle.org/7.6/release-notes.html Starting a Gradle Daemon (subsequent builds will be faster) > Task :buildSrc:compileJava > Task :buildSrc:compileGroovy NO-SOURCE > Task :buildSrc:processResources NO-SOURCE > Task :buildSrc:classes > Task :buildSrc:jar > Task :buildSrc:assemble > Task :buildSrc:compileTestJava NO-SOURCE > Task :buildSrc:compileTestGroovy NO-SOURCE > Task :buildSrc:processTestResources NO-SOURCE > Task :buildSrc:testClasses UP-TO-DATE > Task :buildSrc:test NO-SOURCE > Task :buildSrc:check UP-TO-DATE > Task :buildSrc:build > Task :mavenLocalClean UP-TO-DATE > Task :gitStatus > Task :errorProneSkipped WARNING: errorprone disabled (skipped on builds not running inside CI environments, pass -Pvalidation.errorprone=true to enable) > Task :solr-missing-doclet:compileJava > Task :solr-missing-doclet:processResources NO-SOURCE > Task :solr-missing-doclet:classes > Task :solr-missing-doclet:jar > Task :solr:core:processResources > Task :solr:core:generatePomFileForJarsPublication > Task :solr:prometheus-exporter:processResources NO-SOURCE > Task :solr:prometheus-exporter:generatePomFileForJarsPublication > Task :solr:solrj-streaming:processResources > Task :solr:solrj-streaming:generatePomFileForJarsPublication > Task :solr:solrj-zookeeper:processResources NO-SOURCE > Task :solr:solrj-zookeeper:generatePomFileForJarsPublication > Task :solr:test-framework:processResources > Task :solr:test-framework:generatePomFileForJarsPublication > Task :solr:modules:analysis-extras:processResources NO-SOURCE > Task :solr:modules:analysis-extras:generatePomFileForJarsPublication > Task :solr:modules:analytics:processResources NO-SOURCE > Task :solr:modules:analytics:generatePomFileForJarsPublication > Task :solr:modules:clustering:processResources NO-SOURCE > Task :solr:modules:clustering:generatePomFileForJarsPublication > Task :solr:modules:extraction:processResources > Task :solr:modules:extraction:generatePomFileForJarsPublication > Task :solr:modules:gcs-repository:processResources NO-SOURCE > Task :solr:solrj:compileJava > Task :solr:modules:gcs-repository:generatePomFileForJarsPublication > Task :solr:modules:hadoop-auth:processResources NO-SOURCE > Task :solr:modules:hadoop-auth:generatePomFileForJarsPublication > Task :solr:modules:hdfs:processResources NO-SOURCE > Task :solr:modules:hdfs:generatePomFileForJarsPublication > Task :solr:modules:jaegertracer-configurator:processResources NO-SOURCE > Task :solr:modules:jaegertracer-configurator:generatePomFileForJarsPublication > Task :solr:modules:jwt-auth:processResources NO-SOURCE > Task :solr:modules:jwt-auth:generatePomFileForJarsPublication > Task :solr:modules:langid:processResources > Task :solr:modules:langid:generatePomFileForJarsPublication > Task :solr:modules:ltr:processResources NO-SOURCE > Task :solr:modules:ltr:generatePomFileForJarsPublication > Task :solr:modules:opentelemetry:processResources NO-SOURCE > Task :solr:modules:opentelemetry:generatePomFileForJarsPublication > Task :solr:modules:s3-repository:processResources NO-SOURCE > Task :solr:modules:s3-repository:generatePomFileForJarsPublication > Task :solr:modules:scripting:processResources NO-SOURCE > Task :solr:modules:scripting:generatePomFileForJarsPublication > Task :solr:modules:sql:processResources > Task :solr:modules:sql:generatePomFileForJarsPublication > Task :solr:docker:createBodySnippetDockerfile > Task :solr:docker:createDockerfileLocal > Task :solr:docker:assemblePackaging > Task :solr:solrj:compileJava Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. > Task :solr:documentation:changesToHtml > Task :solr:solrj-streaming:compileJava Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. > Task :solr:solrj-zookeeper:compileJava Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. > Task :solr:solrj:renderJavadoc > Task :solr:core:compileJava > Task :solr:solrj-streaming:renderJavadoc > Task :solr:solrj-zookeeper:renderJavadoc > Task :solr:prometheus-exporter:compileJava Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. > Task :solr:prometheus-exporter:classes > Task :solr:prometheus-exporter:jar > Task :solr:prometheus-exporter:generateMetadataFileForJarsPublication SKIPPED > Task :solr:prometheus-exporter:renderJavadoc > Task :solr:prometheus-exporter:javadoc SKIPPED > Task :solr:prometheus-exporter:javadocJar > Task :solr:prometheus-exporter:sourcesJar > Task :solr:prometheus-exporter:publishJarsPublicationToBuildRepository > Task :solr:solrj:processResources NO-SOURCE > Task :solr:solrj:classes > Task :solr:solrj:jar > Task :solr:solrj:generateMetadataFileForJarsPublication SKIPPED > Task :solr:solrj:generatePomFileForJarsPublication > Task :solr:solrj:javadoc SKIPPED > Task :solr:solrj:javadocJar > Task :solr:solrj:sourcesJar > Task :solr:solrj:publishJarsPublicationToBuildRepository > Task :solr:solrj-streaming:classes > Task :solr:solrj-streaming:jar > Task :solr:solrj-streaming:generateMetadataFileForJarsPublication SKIPPED > Task :solr:solrj-streaming:javadoc SKIPPED > Task :solr:solrj-streaming:javadocJar > Task :solr:solrj-streaming:sourcesJar > Task :solr:solrj-streaming:publishJarsPublicationToBuildRepository > Task :solr:solrj-zookeeper:classes > Task :solr:solrj-zookeeper:jar > Task :solr:solrj-zookeeper:generateMetadataFileForJarsPublication SKIPPED > Task :solr:solrj-zookeeper:javadoc SKIPPED > Task :solr:solrj-zookeeper:javadocJar > Task :solr:solrj-zookeeper:sourcesJar > Task :solr:solrj-zookeeper:publishJarsPublicationToBuildRepository > Task :solr:documentation:copyDocumentationAssets > Task :solr:documentation:markdownToHtml > Task :solr:documentation:copyChangesToHtmlForMiniSite > Task :solr:documentation:copyMiniDocumentationAssets > Task :solr:documentation:createMiniDocumentationIndex > Task :solr:documentation:documentationMinimal > Task :solr:prometheus-exporter:assemblePackaging > Task :solr:core:compileJava Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. > Task :solr:core:classes > Task :solr:core:jar > Task :solr:core:generateMetadataFileForJarsPublication SKIPPED > Task :solr:test-framework:compileJava > Task :solr:core:renderJavadoc > Task :solr:test-framework:compileJava Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. > Task :solr:test-framework:classes > Task :solr:test-framework:jar > Task :solr:test-framework:generateMetadataFileForJarsPublication SKIPPED > Task :solr:test-framework:sourcesJar > Task :solr:modules:analysis-extras:compileJava Note: /build/solr/src/solr-9.2.1/solr/modules/analysis-extras/src/java/org/apache/solr/schema/ICUCollationField.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. > Task :solr:modules:analysis-extras:classes > Task :solr:modules:analysis-extras:jar > Task :solr:modules:analysis-extras:generateMetadataFileForJarsPublication SKIPPED > Task :solr:modules:analysis-extras:sourcesJar > Task :solr:modules:analytics:compileJava Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. > Task :solr:modules:analytics:classes > Task :solr:modules:analytics:jar > Task :solr:modules:analytics:generateMetadataFileForJarsPublication SKIPPED > Task :solr:modules:analytics:sourcesJar > Task :solr:modules:clustering:compileJava > Task :solr:modules:clustering:classes > Task :solr:modules:clustering:jar > Task :solr:modules:clustering:generateMetadataFileForJarsPublication SKIPPED > Task :solr:modules:clustering:sourcesJar > Task :solr:modules:extraction:compileJava Note: /build/solr/src/solr-9.2.1/solr/modules/extraction/src/java/org/apache/solr/handler/extraction/ExtractingDocumentLoader.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. > Task :solr:core:javadoc SKIPPED > Task :solr:core:javadocJar > Task :solr:core:sourcesJar > Task :solr:core:publishJarsPublicationToBuildRepository > Task :solr:modules:analysis-extras:renderJavadoc > Task :solr:modules:analysis-extras:javadoc SKIPPED > Task :solr:modules:analysis-extras:javadocJar > Task :solr:modules:analysis-extras:publishJarsPublicationToBuildRepository > Task :solr:test-framework:renderJavadoc > Task :solr:test-framework:javadoc SKIPPED > Task :solr:modules:analytics:renderJavadoc > Task :solr:test-framework:javadocJar > Task :solr:test-framework:publishJarsPublicationToBuildRepository > Task :solr:modules:clustering:renderJavadoc > Task :solr:modules:clustering:javadoc SKIPPED > Task :solr:modules:clustering:javadocJar > Task :solr:modules:clustering:publishJarsPublicationToBuildRepository > Task :solr:modules:extraction:classes > Task :solr:modules:extraction:jar > Task :solr:modules:extraction:generateMetadataFileForJarsPublication SKIPPED > Task :solr:modules:analytics:javadoc SKIPPED > Task :solr:modules:analytics:javadocJar > Task :solr:modules:analytics:publishJarsPublicationToBuildRepository > Task :solr:modules:extraction:renderJavadoc > Task :solr:modules:extraction:javadoc SKIPPED > Task :solr:modules:extraction:javadocJar > Task :solr:modules:extraction:sourcesJar > Task :solr:modules:extraction:publishJarsPublicationToBuildRepository > Task :solr:modules:gcs-repository:compileJava Note: /build/solr/src/solr-9.2.1/solr/modules/gcs-repository/src/java/org/apache/solr/gcs/GCSConfigParser.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. > Task :solr:modules:hadoop-auth:compileJava Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. > Task :solr:modules:hadoop-auth:classes > Task :solr:modules:hadoop-auth:jar > Task :solr:modules:hadoop-auth:generateMetadataFileForJarsPublication SKIPPED > Task :solr:modules:gcs-repository:classes > Task :solr:modules:gcs-repository:jar > Task :solr:modules:gcs-repository:generateMetadataFileForJarsPublication SKIPPED > Task :solr:modules:hadoop-auth:renderJavadoc > Task :solr:modules:hadoop-auth:javadoc SKIPPED > Task :solr:modules:hadoop-auth:javadocJar > Task :solr:modules:hadoop-auth:sourcesJar > Task :solr:modules:hadoop-auth:publishJarsPublicationToBuildRepository > Task :solr:modules:gcs-repository:renderJavadoc > Task :solr:modules:gcs-repository:javadoc SKIPPED > Task :solr:modules:gcs-repository:javadocJar > Task :solr:modules:gcs-repository:sourcesJar > Task :solr:modules:gcs-repository:publishJarsPublicationToBuildRepository > Task :solr:modules:jaegertracer-configurator:compileJava > Task :solr:modules:jaegertracer-configurator:classes > Task :solr:modules:jaegertracer-configurator:jar > Task :solr:modules:jaegertracer-configurator:generateMetadataFileForJarsPublication SKIPPED > Task :solr:modules:jaegertracer-configurator:renderJavadoc > Task :solr:modules:jaegertracer-configurator:javadoc SKIPPED > Task :solr:modules:hdfs:compileJava Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. > Task :solr:modules:jaegertracer-configurator:javadocJar > Task :solr:modules:jaegertracer-configurator:sourcesJar > Task :solr:modules:jaegertracer-configurator:publishJarsPublicationToBuildRepository > Task :solr:modules:jwt-auth:compileJava Note: /build/solr/src/solr-9.2.1/solr/modules/jwt-auth/src/java/org/apache/solr/security/jwt/JWTAuthPlugin.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. > Task :solr:modules:jwt-auth:classes > Task :solr:modules:jwt-auth:jar > Task :solr:modules:jwt-auth:generateMetadataFileForJarsPublication SKIPPED > Task :solr:modules:jwt-auth:renderJavadoc > Task :solr:modules:jwt-auth:javadoc SKIPPED > Task :solr:modules:jwt-auth:javadocJar > Task :solr:modules:jwt-auth:sourcesJar > Task :solr:modules:jwt-auth:publishJarsPublicationToBuildRepository > Task :solr:modules:langid:compileJava Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. > Task :solr:modules:langid:classes > Task :solr:modules:langid:jar > Task :solr:modules:langid:generateMetadataFileForJarsPublication SKIPPED > Task :solr:modules:hdfs:classes > Task :solr:modules:hdfs:jar > Task :solr:modules:hdfs:generateMetadataFileForJarsPublication SKIPPED > Task :solr:modules:langid:renderJavadoc > Task :solr:modules:langid:javadoc SKIPPED > Task :solr:modules:langid:javadocJar > Task :solr:modules:langid:sourcesJar > Task :solr:modules:langid:publishJarsPublicationToBuildRepository > Task :solr:modules:ltr:compileJava > Task :solr:modules:ltr:classes > Task :solr:modules:ltr:jar > Task :solr:modules:ltr:generateMetadataFileForJarsPublication SKIPPED > Task :solr:modules:hdfs:renderJavadoc > Task :solr:modules:hdfs:javadoc SKIPPED > Task :solr:modules:hdfs:javadocJar > Task :solr:modules:hdfs:sourcesJar > Task :solr:modules:hdfs:publishJarsPublicationToBuildRepository > Task :solr:modules:opentelemetry:compileJava > Task :solr:modules:opentelemetry:classes > Task :solr:modules:opentelemetry:jar > Task :solr:modules:opentelemetry:generateMetadataFileForJarsPublication SKIPPED > Task :solr:modules:ltr:renderJavadoc > Task :solr:modules:ltr:javadoc SKIPPED > Task :solr:modules:opentelemetry:renderJavadoc > Task :solr:modules:opentelemetry:javadoc SKIPPED > Task :solr:modules:ltr:javadocJar > Task :solr:modules:opentelemetry:javadocJar > Task :solr:modules:ltr:sourcesJar > Task :solr:modules:opentelemetry:sourcesJar > Task :solr:modules:opentelemetry:publishJarsPublicationToBuildRepository > Task :solr:modules:ltr:publishJarsPublicationToBuildRepository > Task :solr:modules:scripting:compileJava Note: /build/solr/src/solr-9.2.1/solr/modules/scripting/src/java/org/apache/solr/scripting/xslt/XSLTUpdateRequestHandler.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. > Task :solr:modules:scripting:classes > Task :solr:modules:scripting:jar > Task :solr:modules:scripting:generateMetadataFileForJarsPublication SKIPPED > Task :solr:modules:scripting:renderJavadoc > Task :solr:modules:scripting:javadoc SKIPPED > Task :solr:modules:scripting:javadocJar > Task :solr:modules:scripting:sourcesJar > Task :solr:modules:scripting:publishJarsPublicationToBuildRepository > Task :solr:modules:s3-repository:compileJava > Task :solr:modules:s3-repository:classes > Task :solr:modules:s3-repository:jar > Task :solr:modules:s3-repository:generateMetadataFileForJarsPublication SKIPPED > Task :solr:modules:s3-repository:renderJavadoc > Task :solr:modules:s3-repository:javadoc SKIPPED > Task :solr:modules:s3-repository:javadocJar > Task :solr:modules:s3-repository:sourcesJar > Task :solr:modules:s3-repository:publishJarsPublicationToBuildRepository > Task :solr:core:assemblePostJar > Task :solr:example:assemblePackaging > Task :solr:webapp:compileJava NO-SOURCE > Task :solr:webapp:processResources NO-SOURCE > Task :solr:webapp:classes UP-TO-DATE > Task :solr:modules:sql:compileJava Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. > Task :solr:modules:sql:classes > Task :solr:modules:sql:jar > Task :solr:modules:sql:generateMetadataFileForJarsPublication SKIPPED > Task :solr:webapp:war > Task :solr:server:assemblePackaging > Task :solr:modules:sql:renderJavadoc > Task :solr:modules:sql:javadoc SKIPPED > Task :solr:modules:sql:javadocJar > Task :solr:modules:sql:sourcesJar > Task :solr:modules:sql:publishJarsPublicationToBuildRepository > Task :mavenToLocalFolder Local maven artifacts (poms, jars) created at: /build/solr/src/solr-9.2.1/build/maven-local > Task :solr:modules:analytics:assemblePackaging > Task :solr:modules:clustering:assemblePackaging > Task :solr:modules:analysis-extras:assemblePackaging > Task :solr:modules:extraction:assemblePackaging > Task :solr:modules:hadoop-auth:assemblePackaging > Task :solr:modules:gcs-repository:assemblePackaging > Task :solr:modules:jaegertracer-configurator:assemblePackaging > Task :solr:modules:jwt-auth:assemblePackaging > Task :solr:modules:langid:assemblePackaging > Task :solr:modules:ltr:assemblePackaging > Task :solr:modules:opentelemetry:assemblePackaging > Task :solr:modules:hdfs:assemblePackaging > Task :solr:modules:scripting:assemblePackaging > Task :solr:modules:s3-repository:assemblePackaging > Task :solr:distribution:prepareGitRev > Task :solr:modules:sql:assemblePackaging > Task :solr:packaging:distTar > Task :solr:distribution:computeChecksums > Task :solr:distribution:assembleRelease BUILD SUCCESSFUL in 56m 5s 194 actionable tasks: 193 executed, 1 up-to-date install: creating directory '/build/solr/src/solr-9.2.1/dist' ==> Starting check()... > Task :buildSrc:compileJava UP-TO-DATE > Task :buildSrc:compileGroovy NO-SOURCE > Task :buildSrc:processResources NO-SOURCE > Task :buildSrc:classes UP-TO-DATE > Task :buildSrc:jar UP-TO-DATE > Task :buildSrc:assemble UP-TO-DATE > Task :buildSrc:compileTestJava NO-SOURCE > Task :buildSrc:compileTestGroovy NO-SOURCE > Task :buildSrc:processTestResources NO-SOURCE > Task :buildSrc:testClasses UP-TO-DATE > Task :buildSrc:test NO-SOURCE > Task :buildSrc:check UP-TO-DATE > Task :buildSrc:build UP-TO-DATE > Task :randomizationInfo Running tests with randomization seed: tests.seed=9CAA27B60427D55F > Task :errorProneSkipped WARNING: errorprone disabled (skipped on builds not running inside CI environments, pass -Pvalidation.errorprone=true to enable) > Task :solr:benchmark:processResources > Task :solr:benchmark:processTestResources > Task :solr:solrj:compileJava UP-TO-DATE > Task :gitStatus > Task :solr:solrj-streaming:compileJava UP-TO-DATE > Task :solr:core:processResources UP-TO-DATE > Task :solr:solrj:processResources NO-SOURCE > Task :solr:solrj:classes UP-TO-DATE > Task :solr:solrj-zookeeper:compileJava UP-TO-DATE > Task :solr:solrj:jar UP-TO-DATE > Task :solr:solrj-streaming:processResources UP-TO-DATE > Task :solr:solrj-streaming:classes UP-TO-DATE > Task :solr:solrj-streaming:jar UP-TO-DATE > Task :solr:solrj-zookeeper:processResources NO-SOURCE > Task :solr:solrj-zookeeper:classes UP-TO-DATE > Task :solr:solrj-zookeeper:jar UP-TO-DATE > Task :solr:core:compileJava UP-TO-DATE > Task :solr:core:classes UP-TO-DATE > Task :solr:test-framework:processResources UP-TO-DATE > Task :solr:core:jar UP-TO-DATE > Task :solr:test-framework:compileJava UP-TO-DATE > Task :solr:core:processTestResources > Task :solr:test-framework:classes UP-TO-DATE > Task :solr:test-framework:jar UP-TO-DATE > Task :solr:benchmark:compileJava > Task :solr:core:compileTestJava > Task :solr:benchmark:classes > Task :solr:benchmark:compileTestJava > Task :solr:benchmark:testClasses > Task :solr:benchmark:test > Task :solr:core:compileTestJava Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. > Task :solr:core:testClasses > Task :solr:modules:analysis-extras:compileJava UP-TO-DATE > Task :solr:modules:analysis-extras:processResources NO-SOURCE > Task :solr:modules:analysis-extras:classes UP-TO-DATE > Task :solr:modules:analysis-extras:jar UP-TO-DATE > Task :solr:core:test WARNING: Test org.apache.solr.TestRandomFaceting wrote 13,246,749 bytes of output. org.apache.solr.cloud.HttpPartitionOnCommitTest > test FAILED org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:45565/dql/qo: Underlying core creation failed while creating collection: c8n_1x3_commits at __randomizedtesting.SeedInfo.seed([9CAA27B60427D55F:14FE186CAADBB8A7]:0) at app//org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:745) at app//org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259) at app//org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240) at app//org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1193) at app//org.apache.solr.cloud.AbstractBasicDistributedZkTestBase.createCollection(AbstractBasicDistributedZkTestBase.java:1172) at app//org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:2013) at app//org.apache.solr.cloud.HttpPartitionOnCommitTest.oneShardTest(HttpPartitionOnCommitTest.java:131) at app//org.apache.solr.cloud.HttpPartitionOnCommitTest.test(HttpPartitionOnCommitTest.java:66) at java.base@11.0.20/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base@11.0.20/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base@11.0.20/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base@11.0.20/java.lang.reflect.Method.invoke(Method.java:566) at app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) at app//com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) at app//com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) at app//com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) at app//org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1165) at app//org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1136) at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) at app//org.junit.rules.RunRules.evaluate(RunRules.java:20) at app//org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:44) at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) at app//org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) at app//org.junit.rules.RunRules.evaluate(RunRules.java:20) at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) at app//com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) at app//com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) at app//com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) at app//org.junit.rules.RunRules.evaluate(RunRules.java:20) at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at app//org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at app//org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) at app//org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) at app//org.junit.rules.RunRules.evaluate(RunRules.java:20) at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) at java.base@11.0.20/java.lang.Thread.run(Thread.java:829) org.apache.solr.cloud.HttpPartitionOnCommitTest > test suite's output saved to /build/solr/src/solr-9.2.1/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.HttpPartitionOnCommitTest.txt, copied below: 2> 2143850 INFO (SUITE-HttpPartitionOnCommitTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/build/solr/src/solr-9.2.1/solr/server/solr/configsets/_default/conf' 2> 2143851 INFO (SUITE-HttpPartitionOnCommitTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom 2> 2143858 INFO (SUITE-HttpPartitionOnCommitTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.u.ErrorLogMuter Closing ErrorLogMuter-regex-100 after mutting 0 log messages 2> 2143858 INFO (SUITE-HttpPartitionOnCommitTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.u.ErrorLogMuter Creating ErrorLogMuter-regex-101 for ERROR logs matching regex: ignore_exception 2> 2143868 INFO (SUITE-HttpPartitionOnCommitTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.SolrTestCaseJ4 Created dataDir: /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/data-dir-37-001 2> 2143876 WARN (SUITE-HttpPartitionOnCommitTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=47 numCloses=47 2> 2143877 INFO (SUITE-HttpPartitionOnCommitTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true 2> 2143883 INFO (SUITE-HttpPartitionOnCommitTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl="https://issues.apache.org/jira/browse/SOLR-5776") 2> 2143902 INFO (SUITE-HttpPartitionOnCommitTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /dql/qo 2> 2144086 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.u.ErrorLogMuter Closing ErrorLogMuter-regex-101 after mutting 0 log messages 2> 2144086 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.u.ErrorLogMuter Creating ErrorLogMuter-regex-102 for ERROR logs matching regex: ignore_exception 2> 2144105 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER 2> 2144114 INFO (ZkTestServer Run Thread) [] o.a.s.c.ZkTestServer client port: 0.0.0.0/0.0.0.0:0 2> 2144114 INFO (ZkTestServer Run Thread) [] o.a.s.c.ZkTestServer Starting server 2> 2144128 WARN (ZkTestServer Run Thread) [] o.a.z.s.ServerCnxnFactory maxCnxns is not configured, using default value 0. 2> 2144216 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkTestServer start zk server on port: 38599 2> 2144261 WARN (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2144308 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 2144401 INFO (zkConnectionManagerCallback-3044-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 2144401 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 2144402 WARN (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2144456 WARN (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2144492 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 2144512 INFO (zkConnectionManagerCallback-3046-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 2144513 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 2144514 WARN (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2144545 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkTestServer put /build/solr/src/solr-9.2.1/solr/core/build/resources/test/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml 2> 2144589 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkTestServer put /build/solr/src/solr-9.2.1/solr/core/build/resources/test/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml 2> 2144629 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkTestServer put /build/solr/src/solr-9.2.1/solr/core/build/resources/test/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml 2> 2144685 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkTestServer put /build/solr/src/solr-9.2.1/solr/core/build/resources/test/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt 2> 2144717 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkTestServer put /build/solr/src/solr-9.2.1/solr/core/build/resources/test/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt 2> 2144743 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkTestServer put /build/solr/src/solr-9.2.1/solr/core/build/resources/test/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml 2> 2144785 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkTestServer put /build/solr/src/solr-9.2.1/solr/core/build/resources/test/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml 2> 2144829 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkTestServer put /build/solr/src/solr-9.2.1/solr/core/build/resources/test/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json 2> 2144881 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkTestServer put /build/solr/src/solr-9.2.1/solr/core/build/resources/test/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt 2> 2144929 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkTestServer put /build/solr/src/solr-9.2.1/solr/core/build/resources/test/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt 2> 2144978 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkTestServer put /build/solr/src/solr-9.2.1/solr/core/build/resources/test/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt 2> 2145045 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise 2> 2150424 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0) 2> 2150424 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ... 2> 2150426 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.e.j.s.Server jetty-10.0.13; built: 2022-12-07T20:13:20.134Z; git: 1c2636ea05c0ca8de1ffd6ca7f3a98ac084c766d; jvm 11.0.20+8 2> 2150512 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.e.j.s.s.DefaultSessionIdManager Session workerName=node0 2> 2150530 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@65e9aeb1{/dql/qo,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,AVAILABLE} 2> 2150533 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.e.j.s.AbstractConnector Started ServerConnector@370d36ea{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:39373} 2> 2150534 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.e.j.s.Server Started Server@b8bdf0d{STARTING}[10.0.13,sto=0] @2150881ms 2> 2150541 ERROR (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. 2> 2150542 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.s.CoreContainerProvider Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory 2> 2150543 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.s.CoreContainerProvider ___ _ Welcome to Apache Solr™ version 9.2.1 2> 2150543 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.s.CoreContainerProvider / __| ___| |_ _ Starting in cloud mode on port null 2> 2150544 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.s.CoreContainerProvider \__ \/ _ \ | '_| Install dir: /build/solr/src/solr-9.2.1/solr 2> 2150545 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.s.CoreContainerProvider |___/\___/_|_| Start time: 2023-10-03T08:24:28.549051Z 2> 2150553 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.s.CoreContainerProvider Solr Home: /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/control-001 (source: servlet config: solr.solr.home) 2> 2150554 WARN (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2150603 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 2150632 INFO (zkConnectionManagerCallback-3048-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 2150633 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 2150633 WARN (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2150772 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.NodeConfig Loading solr.xml from SolrHome (not found in ZooKeeper) 2> 2150773 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.SolrXmlConfig Loading solr.xml from /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/control-001/solr.xml 2> 2160982 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38599/solr 2> 2160983 WARN (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2160996 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 2161026 INFO (zkConnectionManagerCallback-3058-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 2161027 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 2161027 WARN (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2161157 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=false. Solr will be using Overseer based cluster state updates. 2> 2161193 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 2161205 INFO (zkConnectionManagerCallback-3060-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 2161206 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 2161754 WARN (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.c.ZkController Contents of zookeeper /security.json are world-readable; consider setting up ACLs as described in https://solr.apache.org/guide/solr/latest/deployment-guide/zookeeper-access-control.html 2> 2161817 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=false. Solr will be using Overseer based cluster state updates. 2> 2161861 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42333_dql%2Fqo 2> 2161874 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.c.Overseer Overseer (id=72080878792015876-127.0.0.1:42333_dql%2Fqo-n_0000000000) starting 2> 2162051 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42333_dql%2Fqo 2> 2162051 INFO (OverseerStateUpdate-72080878792015876-127.0.0.1:42333_dql%2Fqo-n_0000000000) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:42333_dql%2Fqo 2> 2162091 INFO (OverseerStateUpdate-72080878792015876-127.0.0.1:42333_dql%2Fqo-n_0000000000) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) 2> 2162134 WARN (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info 2> 2163707 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/control-001/cores 2> 2164310 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 2164429 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.e.JettySolrRunner Jetty properties: {hostContext=/dql/qo, solr.data.dir=/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/tempDir-001/control/data, hostPort=42333, coreRootDirectory=/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/control-001/cores} 2> 2164444 WARN (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2164456 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 15000ms for client to connect to ZooKeeper 2> 2164484 INFO (zkConnectionManagerCallback-3072-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 2164485 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 2164485 WARN (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2164503 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) 2> 2164517 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:38599/solr ready 2> 2164523 INFO (SocketProxy-Acceptor-42333) [] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=53658,localport=42333], receiveBufferSize: 65536 2> 2164588 INFO (SocketProxy-Acceptor-42333) [] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=39373,localport=40008], receiveBufferSize=65536 2> 2164660 INFO (OverseerThreadFactory-3065-thread-1) [] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection 2> 2164863 INFO (OverseerStateUpdate-72080878792015876-127.0.0.1:42333_dql%2Fqo-n_0000000000) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"control_collection_shard1_replica_n1", 2> "node_name":"127.0.0.1:42333_dql%2Fqo", 2> "base_url":"http://127.0.0.1:42333/dql/qo", 2> "collection":"control_collection", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 2164992 INFO (zkCallback-3059-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1]) 2> 2165026 INFO (SocketProxy-Acceptor-42333) [] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=53662,localport=42333], receiveBufferSize: 65536 2> 2165028 INFO (SocketProxy-Acceptor-42333) [] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=39373,localport=40012], receiveBufferSize=65536 2> 2165051 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo x:control_collection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 2165407 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 2165512 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Schema name=test 2> 2165693 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id 2> 2166102 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from configset conf1, trusted=true 2> 2166106 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/control-001/cores/control_collection_shard1_replica_n1/data/] 2> 2166144 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=34, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022208E12, noCFSRatio=0.0] 2> 2166166 WARN (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A, b=B}}} 2> 2166954 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 2166970 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 2166971 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 2167004 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 2167004 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 2167030 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.tests.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=919818910, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022208E12, noCFSRatio=0.1] 2> 2167052 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 2167058 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 2167070 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 2167071 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778721997534003200 2> 2167156 INFO (searcherExecutor-3074-thread-1-processing-127.0.0.1:42333_dql%2Fqo control_collection_shard1_replica_n1 control_collection shard1 core_node2) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 2167228 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, version=0} for registerTerm 2> 2167228 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/control_collection/leaders/shard1 2> 2167401 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 2167401 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 2167401 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42333/dql/qo/control_collection_shard1_replica_n1/ 2> 2167410 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 2167419 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:42333/dql/qo/control_collection_shard1_replica_n1/ has no replicas 2> 2167419 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/72080878792015876-core_node2-n_0000000000 2> 2167485 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42333/dql/qo/control_collection_shard1_replica_n1/ shard1 2> 2167524 INFO (zkCallback-3059-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1]) 2> 2167524 INFO (zkCallback-3059-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1]) 2> 2167540 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 2167582 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2533 2> 2167627 INFO (qtp1257576055-4703) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas 2> 2167675 INFO (zkCallback-3059-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1]) 2> 2167675 INFO (zkCallback-3059-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1]) 2> 2167676 INFO (zkCallback-3059-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1]) 2> 2167687 INFO (qtp1257576055-4703) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:42333_dql%252Fqo&wt=javabin&version=2} status=0 QTime=3075 2> 2167693 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.AbstractFullDistribZkTestBase Waiting to see 1 active replicas in collection: control_collection 2> 2167909 WARN (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2167924 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 15000ms for client to connect to ZooKeeper 2> 2167948 INFO (zkConnectionManagerCallback-3083-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 2167949 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 2167949 WARN (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2167984 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) 2> 2168010 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:38599/solr ready 2> 2168010 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false 2> 2168015 INFO (SocketProxy-Acceptor-42333) [] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=53672,localport=42333], receiveBufferSize: 65536 2> 2168018 INFO (SocketProxy-Acceptor-42333) [] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=39373,localport=40022], receiveBufferSize=65536 2> 2168140 INFO (OverseerThreadFactory-3065-thread-2) [] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1 2> 2168153 INFO (OverseerCollectionConfigSetProcessor-72080878792015876-127.0.0.1:42333_dql%2Fqo-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 2168452 WARN (OverseerThreadFactory-3065-thread-2) [] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores. 2> 2168480 INFO (qtp1257576055-4705) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas 2> 2168515 INFO (qtp1257576055-4705) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&wt=javabin&version=2} status=0 QTime=466 2> 2168531 INFO (watches-3080-thread-1) [] o.a.s.c.SolrCloudTestCase active slice count: 1 expected: 1 2> 2168532 INFO (watches-3080-thread-1) [] o.a.s.c.SolrCloudTestCase active replica count: 0 expected replica count: 0 2> 2168552 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.SolrCloudTestCase active slice count: 1 expected: 1 2> 2168552 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.SolrCloudTestCase active replica count: 0 expected replica count: 0 2> 2168563 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.SolrCloudTestCase active slice count: 1 expected: 1 2> 2168563 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.SolrCloudTestCase active replica count: 0 expected replica count: 0 2> 2168581 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.SolrCloudTestCase active slice count: 1 expected: 1 2> 2168581 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.SolrCloudTestCase active replica count: 0 expected replica count: 0 2> 2168581 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.AbstractFullDistribZkTestBase Creating jetty instances pullReplicaCount=0 numOtherReplicas=4 2> 2170186 INFO (OverseerCollectionConfigSetProcessor-72080878792015876-127.0.0.1:42333_dql%2Fqo-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 2173673 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-1-001 of type NRT for shard1 2> 2173709 INFO (closeThreadPool-3084-thread-1) [] o.a.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0) 2> 2173709 INFO (closeThreadPool-3084-thread-1) [] o.a.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ... 2> 2173710 INFO (closeThreadPool-3084-thread-1) [] o.e.j.s.Server jetty-10.0.13; built: 2022-12-07T20:13:20.134Z; git: 1c2636ea05c0ca8de1ffd6ca7f3a98ac084c766d; jvm 11.0.20+8 2> 2173741 INFO (closeThreadPool-3084-thread-1) [] o.e.j.s.s.DefaultSessionIdManager Session workerName=node0 2> 2173745 INFO (closeThreadPool-3084-thread-1) [] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@60c0e855{/dql/qo,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,AVAILABLE} 2> 2173750 INFO (closeThreadPool-3084-thread-1) [] o.e.j.s.AbstractConnector Started ServerConnector@2f06c358{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:43375} 2> 2173750 INFO (closeThreadPool-3084-thread-1) [] o.e.j.s.Server Started Server@422145a3{STARTING}[10.0.13,sto=0] @2174097ms 2> 2173752 ERROR (closeThreadPool-3084-thread-1) [] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. 2> 2173752 INFO (closeThreadPool-3084-thread-1) [] o.a.s.s.CoreContainerProvider Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory 2> 2173752 INFO (closeThreadPool-3084-thread-1) [] o.a.s.s.CoreContainerProvider ___ _ Welcome to Apache Solr™ version 9.2.1 2> 2173753 INFO (closeThreadPool-3084-thread-1) [] o.a.s.s.CoreContainerProvider / __| ___| |_ _ Starting in cloud mode on port null 2> 2173753 INFO (closeThreadPool-3084-thread-1) [] o.a.s.s.CoreContainerProvider \__ \/ _ \ | '_| Install dir: /build/solr/src/solr-9.2.1/solr 2> 2173753 INFO (closeThreadPool-3084-thread-1) [] o.a.s.s.CoreContainerProvider |___/\___/_|_| Start time: 2023-10-03T08:24:51.757775Z 2> 2173754 INFO (closeThreadPool-3084-thread-1) [] o.a.s.s.CoreContainerProvider Solr Home: /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-1-001 (source: servlet config: solr.solr.home) 2> 2173755 WARN (closeThreadPool-3084-thread-1) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2173813 INFO (closeThreadPool-3084-thread-1) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 2173836 INFO (zkConnectionManagerCallback-3086-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 2173836 INFO (closeThreadPool-3084-thread-1) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 2173847 WARN (closeThreadPool-3084-thread-1) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2174004 INFO (closeThreadPool-3084-thread-1) [] o.a.s.c.NodeConfig Loading solr.xml from SolrHome (not found in ZooKeeper) 2> 2174005 INFO (closeThreadPool-3084-thread-1) [] o.a.s.c.SolrXmlConfig Loading solr.xml from /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-1-001/solr.xml 2> 2178748 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-2-001 of type NRT for shard1 2> 2178769 INFO (closeThreadPool-3084-thread-1) [] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38599/solr 2> 2178772 WARN (closeThreadPool-3084-thread-1) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2178816 INFO (closeThreadPool-3084-thread-1) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 2178860 INFO (closeThreadPool-3084-thread-2) [] o.a.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0) 2> 2178861 INFO (closeThreadPool-3084-thread-2) [] o.a.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ... 2> 2178862 INFO (closeThreadPool-3084-thread-2) [] o.e.j.s.Server jetty-10.0.13; built: 2022-12-07T20:13:20.134Z; git: 1c2636ea05c0ca8de1ffd6ca7f3a98ac084c766d; jvm 11.0.20+8 2> 2178889 INFO (zkConnectionManagerCallback-3096-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 2178892 INFO (closeThreadPool-3084-thread-1) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 2178893 WARN (closeThreadPool-3084-thread-1) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2178993 INFO (closeThreadPool-3084-thread-2) [] o.e.j.s.s.DefaultSessionIdManager Session workerName=node0 2> 2179006 INFO (closeThreadPool-3084-thread-2) [] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6ef7c7e2{/dql/qo,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,AVAILABLE} 2> 2179023 INFO (closeThreadPool-3084-thread-2) [] o.e.j.s.AbstractConnector Started ServerConnector@59a173a7{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:44821} 2> 2179023 INFO (closeThreadPool-3084-thread-2) [] o.e.j.s.Server Started Server@7f9bcce2{STARTING}[10.0.13,sto=0] @2179370ms 2> 2179025 ERROR (closeThreadPool-3084-thread-2) [] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. 2> 2179056 INFO (closeThreadPool-3084-thread-2) [] o.a.s.s.CoreContainerProvider Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory 2> 2179056 INFO (closeThreadPool-3084-thread-2) [] o.a.s.s.CoreContainerProvider ___ _ Welcome to Apache Solr™ version 9.2.1 2> 2179057 INFO (closeThreadPool-3084-thread-2) [] o.a.s.s.CoreContainerProvider / __| ___| |_ _ Starting in cloud mode on port null 2> 2179057 INFO (closeThreadPool-3084-thread-2) [] o.a.s.s.CoreContainerProvider \__ \/ _ \ | '_| Install dir: /build/solr/src/solr-9.2.1/solr 2> 2179058 INFO (closeThreadPool-3084-thread-2) [] o.a.s.s.CoreContainerProvider |___/\___/_|_| Start time: 2023-10-03T08:24:57.061993Z 2> 2179058 INFO (closeThreadPool-3084-thread-2) [] o.a.s.s.CoreContainerProvider Solr Home: /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-2-001 (source: servlet config: solr.solr.home) 2> 2179060 WARN (closeThreadPool-3084-thread-2) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2179097 INFO (closeThreadPool-3084-thread-1) [] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=false. Solr will be using Overseer based cluster state updates. 2> 2179160 INFO (closeThreadPool-3084-thread-2) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 2179180 INFO (closeThreadPool-3084-thread-1) [n:127.0.0.1:39977_dql%2Fqo] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 2179234 INFO (zkConnectionManagerCallback-3098-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 2179240 INFO (closeThreadPool-3084-thread-2) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 2179243 WARN (closeThreadPool-3084-thread-2) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2179281 INFO (zkConnectionManagerCallback-3100-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 2179288 INFO (closeThreadPool-3084-thread-1) [n:127.0.0.1:39977_dql%2Fqo] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 2179400 INFO (closeThreadPool-3084-thread-2) [] o.a.s.c.NodeConfig Loading solr.xml from SolrHome (not found in ZooKeeper) 2> 2179401 INFO (closeThreadPool-3084-thread-2) [] o.a.s.c.SolrXmlConfig Loading solr.xml from /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-2-001/solr.xml 2> 2179715 WARN (closeThreadPool-3084-thread-1) [n:127.0.0.1:39977_dql%2Fqo] o.a.s.c.ZkController Contents of zookeeper /security.json are world-readable; consider setting up ACLs as described in https://solr.apache.org/guide/solr/latest/deployment-guide/zookeeper-access-control.html 2> 2179769 INFO (closeThreadPool-3084-thread-1) [n:127.0.0.1:39977_dql%2Fqo] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) 2> 2180117 INFO (closeThreadPool-3084-thread-1) [n:127.0.0.1:39977_dql%2Fqo] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=false. Solr will be using Overseer based cluster state updates. 2> 2180216 INFO (closeThreadPool-3084-thread-1) [n:127.0.0.1:39977_dql%2Fqo] o.a.s.c.ZkController Publish node=127.0.0.1:39977_dql%2Fqo as DOWN 2> 2180256 INFO (closeThreadPool-3084-thread-1) [n:127.0.0.1:39977_dql%2Fqo] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39977_dql%2Fqo 2> 2180289 INFO (zkCallback-3059-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2) 2> 2180309 INFO (zkCallback-3082-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2) 2> 2180313 INFO (zkCallback-3099-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2) 2> 2180408 WARN (closeThreadPool-3084-thread-1) [n:127.0.0.1:39977_dql%2Fqo] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info 2> 2182982 INFO (closeThreadPool-3084-thread-1) [n:127.0.0.1:39977_dql%2Fqo] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-1-001/cores 2> 2184298 INFO (closeThreadPool-3084-thread-1) [n:127.0.0.1:39977_dql%2Fqo] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 2184492 INFO (closeThreadPool-3084-thread-1) [n:127.0.0.1:39977_dql%2Fqo] o.a.s.e.JettySolrRunner Jetty properties: {hostContext=/dql/qo, solrconfig=solrconfig.xml, solr.data.dir=/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/tempDir-001/jetty1, hostPort=39977, coreRootDirectory=/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-1-001/cores} 2> 2184495 INFO (closeThreadPool-3084-thread-1) [] o.a.s.c.AbstractFullDistribZkTestBase waitForLiveNode: 127.0.0.1:39977_dql%2Fqo 2> 2185857 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-3-001 of type NRT for shard1 2> 2185882 INFO (closeThreadPool-3084-thread-1) [] o.a.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0) 2> 2185882 INFO (closeThreadPool-3084-thread-1) [] o.a.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ... 2> 2185883 INFO (closeThreadPool-3084-thread-1) [] o.e.j.s.Server jetty-10.0.13; built: 2022-12-07T20:13:20.134Z; git: 1c2636ea05c0ca8de1ffd6ca7f3a98ac084c766d; jvm 11.0.20+8 2> 2185900 INFO (closeThreadPool-3084-thread-1) [] o.e.j.s.s.DefaultSessionIdManager Session workerName=node0 2> 2185903 INFO (closeThreadPool-3084-thread-1) [] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@78e95412{/dql/qo,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,AVAILABLE} 2> 2185909 INFO (closeThreadPool-3084-thread-1) [] o.e.j.s.AbstractConnector Started ServerConnector@2794b6f5{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:36939} 2> 2185910 INFO (closeThreadPool-3084-thread-1) [] o.e.j.s.Server Started Server@3d77ebda{STARTING}[10.0.13,sto=0] @2186256ms 2> 2185911 ERROR (closeThreadPool-3084-thread-1) [] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. 2> 2185911 INFO (closeThreadPool-3084-thread-1) [] o.a.s.s.CoreContainerProvider Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory 2> 2185912 INFO (closeThreadPool-3084-thread-1) [] o.a.s.s.CoreContainerProvider ___ _ Welcome to Apache Solr™ version 9.2.1 2> 2185912 INFO (closeThreadPool-3084-thread-1) [] o.a.s.s.CoreContainerProvider / __| ___| |_ _ Starting in cloud mode on port null 2> 2185913 INFO (closeThreadPool-3084-thread-1) [] o.a.s.s.CoreContainerProvider \__ \/ _ \ | '_| Install dir: /build/solr/src/solr-9.2.1/solr 2> 2185913 INFO (closeThreadPool-3084-thread-1) [] o.a.s.s.CoreContainerProvider |___/\___/_|_| Start time: 2023-10-03T08:25:03.917265Z 2> 2185914 INFO (closeThreadPool-3084-thread-1) [] o.a.s.s.CoreContainerProvider Solr Home: /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-3-001 (source: servlet config: solr.solr.home) 2> 2185915 WARN (closeThreadPool-3084-thread-1) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2185954 INFO (closeThreadPool-3084-thread-1) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 2185988 INFO (zkConnectionManagerCallback-3109-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 2185996 INFO (closeThreadPool-3084-thread-1) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 2185996 WARN (closeThreadPool-3084-thread-1) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2186156 INFO (closeThreadPool-3084-thread-1) [] o.a.s.c.NodeConfig Loading solr.xml from SolrHome (not found in ZooKeeper) 2> 2186157 INFO (closeThreadPool-3084-thread-1) [] o.a.s.c.SolrXmlConfig Loading solr.xml from /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-3-001/solr.xml 2> 2188006 INFO (closeThreadPool-3084-thread-2) [] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38599/solr 2> 2188007 WARN (closeThreadPool-3084-thread-2) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2188055 INFO (closeThreadPool-3084-thread-2) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 2188080 INFO (zkConnectionManagerCallback-3119-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 2188081 INFO (closeThreadPool-3084-thread-2) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 2188081 WARN (closeThreadPool-3084-thread-2) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2188222 INFO (closeThreadPool-3084-thread-2) [] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=false. Solr will be using Overseer based cluster state updates. 2> 2188255 INFO (closeThreadPool-3084-thread-2) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 2188296 INFO (zkConnectionManagerCallback-3121-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 2188297 INFO (closeThreadPool-3084-thread-2) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 2188560 WARN (closeThreadPool-3084-thread-2) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.c.ZkController Contents of zookeeper /security.json are world-readable; consider setting up ACLs as described in https://solr.apache.org/guide/solr/latest/deployment-guide/zookeeper-access-control.html 2> 2188600 INFO (closeThreadPool-3084-thread-2) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2) 2> 2188716 INFO (closeThreadPool-3084-thread-2) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=false. Solr will be using Overseer based cluster state updates. 2> 2188800 INFO (closeThreadPool-3084-thread-2) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.c.ZkController Publish node=127.0.0.1:45565_dql%2Fqo as DOWN 2> 2188840 INFO (closeThreadPool-3084-thread-2) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45565_dql%2Fqo 2> 2188873 INFO (zkCallback-3120-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) 2> 2188876 INFO (zkCallback-3099-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) 2> 2188877 INFO (zkCallback-3059-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) 2> 2188877 INFO (zkCallback-3082-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) 2> 2188921 WARN (closeThreadPool-3084-thread-2) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info 2> 2190799 INFO (closeThreadPool-3084-thread-2) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-2-001/cores 2> 2191603 INFO (closeThreadPool-3084-thread-2) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 2191833 INFO (closeThreadPool-3084-thread-2) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.e.JettySolrRunner Jetty properties: {hostContext=/dql/qo, solrconfig=solrconfig.xml, solr.data.dir=/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/tempDir-001/jetty2, hostPort=45565, coreRootDirectory=/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-2-001/cores} 2> 2191853 INFO (closeThreadPool-3084-thread-2) [] o.a.s.c.AbstractFullDistribZkTestBase waitForLiveNode: 127.0.0.1:45565_dql%2Fqo 2> 2192534 INFO (closeThreadPool-3084-thread-1) [] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38599/solr 2> 2192536 WARN (closeThreadPool-3084-thread-1) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2192595 INFO (closeThreadPool-3084-thread-1) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 2192606 INFO (zkConnectionManagerCallback-3132-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 2192606 INFO (closeThreadPool-3084-thread-1) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 2192607 WARN (closeThreadPool-3084-thread-1) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2192738 INFO (closeThreadPool-3084-thread-1) [] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=false. Solr will be using Overseer based cluster state updates. 2> 2192758 INFO (closeThreadPool-3084-thread-1) [n:127.0.0.1:37307_dql%2Fqo] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 2192816 INFO (zkConnectionManagerCallback-3134-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 2192817 INFO (closeThreadPool-3084-thread-1) [n:127.0.0.1:37307_dql%2Fqo] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 2192905 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.AbstractFullDistribZkTestBase create jetty 4 in directory /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-4-001 of type NRT for shard1 2> 2192920 INFO (closeThreadPool-3084-thread-2) [] o.a.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0) 2> 2192920 INFO (closeThreadPool-3084-thread-2) [] o.a.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ... 2> 2192921 INFO (closeThreadPool-3084-thread-2) [] o.e.j.s.Server jetty-10.0.13; built: 2022-12-07T20:13:20.134Z; git: 1c2636ea05c0ca8de1ffd6ca7f3a98ac084c766d; jvm 11.0.20+8 2> 2192942 INFO (closeThreadPool-3084-thread-2) [] o.e.j.s.s.DefaultSessionIdManager Session workerName=node0 2> 2192946 WARN (closeThreadPool-3084-thread-1) [n:127.0.0.1:37307_dql%2Fqo] o.a.s.c.ZkController Contents of zookeeper /security.json are world-readable; consider setting up ACLs as described in https://solr.apache.org/guide/solr/latest/deployment-guide/zookeeper-access-control.html 2> 2192947 INFO (closeThreadPool-3084-thread-2) [] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@13ff6207{/dql/qo,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,AVAILABLE} 2> 2192957 INFO (closeThreadPool-3084-thread-2) [] o.e.j.s.AbstractConnector Started ServerConnector@35f71550{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:45985} 2> 2192958 INFO (closeThreadPool-3084-thread-2) [] o.e.j.s.Server Started Server@738daffd{STARTING}[10.0.13,sto=0] @2193305ms 2> 2192960 ERROR (closeThreadPool-3084-thread-2) [] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. 2> 2192965 INFO (closeThreadPool-3084-thread-2) [] o.a.s.s.CoreContainerProvider Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory 2> 2192965 INFO (closeThreadPool-3084-thread-2) [] o.a.s.s.CoreContainerProvider ___ _ Welcome to Apache Solr™ version 9.2.1 2> 2192966 INFO (closeThreadPool-3084-thread-2) [] o.a.s.s.CoreContainerProvider / __| ___| |_ _ Starting in cloud mode on port null 2> 2192966 INFO (closeThreadPool-3084-thread-2) [] o.a.s.s.CoreContainerProvider \__ \/ _ \ | '_| Install dir: /build/solr/src/solr-9.2.1/solr 2> 2192966 INFO (closeThreadPool-3084-thread-1) [n:127.0.0.1:37307_dql%2Fqo] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3) 2> 2192966 INFO (closeThreadPool-3084-thread-2) [] o.a.s.s.CoreContainerProvider |___/\___/_|_| Start time: 2023-10-03T08:25:10.970940Z 2> 2192967 INFO (closeThreadPool-3084-thread-2) [] o.a.s.s.CoreContainerProvider Solr Home: /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-4-001 (source: servlet config: solr.solr.home) 2> 2192969 WARN (closeThreadPool-3084-thread-2) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2193010 INFO (closeThreadPool-3084-thread-2) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 2193030 INFO (closeThreadPool-3084-thread-1) [n:127.0.0.1:37307_dql%2Fqo] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=false. Solr will be using Overseer based cluster state updates. 2> 2193042 INFO (zkConnectionManagerCallback-3138-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 2193042 INFO (closeThreadPool-3084-thread-2) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 2193043 WARN (closeThreadPool-3084-thread-2) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2193066 INFO (closeThreadPool-3084-thread-1) [n:127.0.0.1:37307_dql%2Fqo] o.a.s.c.ZkController Publish node=127.0.0.1:37307_dql%2Fqo as DOWN 2> 2193087 INFO (closeThreadPool-3084-thread-1) [n:127.0.0.1:37307_dql%2Fqo] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37307_dql%2Fqo 2> 2193105 INFO (zkCallback-3099-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4) 2> 2193117 INFO (zkCallback-3133-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4) 2> 2193120 INFO (zkCallback-3120-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4) 2> 2193120 INFO (zkCallback-3082-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4) 2> 2193121 INFO (zkCallback-3059-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4) 2> 2193159 WARN (closeThreadPool-3084-thread-1) [n:127.0.0.1:37307_dql%2Fqo] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info 2> 2193193 INFO (closeThreadPool-3084-thread-2) [] o.a.s.c.NodeConfig Loading solr.xml from SolrHome (not found in ZooKeeper) 2> 2193194 INFO (closeThreadPool-3084-thread-2) [] o.a.s.c.SolrXmlConfig Loading solr.xml from /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-4-001/solr.xml 2> 2194590 INFO (closeThreadPool-3084-thread-1) [n:127.0.0.1:37307_dql%2Fqo] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-3-001/cores 2> 2195221 INFO (closeThreadPool-3084-thread-1) [n:127.0.0.1:37307_dql%2Fqo] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 2195544 INFO (closeThreadPool-3084-thread-1) [n:127.0.0.1:37307_dql%2Fqo] o.a.s.e.JettySolrRunner Jetty properties: {hostContext=/dql/qo, solrconfig=solrconfig.xml, solr.data.dir=/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/tempDir-001/jetty3, hostPort=37307, coreRootDirectory=/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-3-001/cores, replicaType=NRT} 2> 2195552 INFO (closeThreadPool-3084-thread-1) [] o.a.s.c.AbstractFullDistribZkTestBase waitForLiveNode: 127.0.0.1:37307_dql%2Fqo 2> 2197415 INFO (closeThreadPool-3084-thread-2) [] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38599/solr 2> 2197417 WARN (closeThreadPool-3084-thread-2) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2197428 INFO (closeThreadPool-3084-thread-2) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 2197461 INFO (zkConnectionManagerCallback-3150-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 2197462 INFO (closeThreadPool-3084-thread-2) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 2197462 WARN (closeThreadPool-3084-thread-2) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 2197583 INFO (closeThreadPool-3084-thread-2) [] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=false. Solr will be using Overseer based cluster state updates. 2> 2197626 INFO (closeThreadPool-3084-thread-2) [n:127.0.0.1:42517_dql%2Fqo] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 2197668 INFO (zkConnectionManagerCallback-3152-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 2197672 INFO (closeThreadPool-3084-thread-2) [n:127.0.0.1:42517_dql%2Fqo] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 2197865 WARN (closeThreadPool-3084-thread-2) [n:127.0.0.1:42517_dql%2Fqo] o.a.s.c.ZkController Contents of zookeeper /security.json are world-readable; consider setting up ACLs as described in https://solr.apache.org/guide/solr/latest/deployment-guide/zookeeper-access-control.html 2> 2197905 INFO (closeThreadPool-3084-thread-2) [n:127.0.0.1:42517_dql%2Fqo] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4) 2> 2197996 INFO (closeThreadPool-3084-thread-2) [n:127.0.0.1:42517_dql%2Fqo] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=false. Solr will be using Overseer based cluster state updates. 2> 2198074 INFO (closeThreadPool-3084-thread-2) [n:127.0.0.1:42517_dql%2Fqo] o.a.s.c.ZkController Publish node=127.0.0.1:42517_dql%2Fqo as DOWN 2> 2198121 INFO (closeThreadPool-3084-thread-2) [n:127.0.0.1:42517_dql%2Fqo] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42517_dql%2Fqo 2> 2198156 INFO (zkCallback-3120-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5) 2> 2198157 INFO (zkCallback-3133-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5) 2> 2198166 INFO (zkCallback-3082-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5) 2> 2198166 INFO (zkCallback-3099-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5) 2> 2198168 INFO (zkCallback-3059-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5) 2> 2198179 INFO (zkCallback-3151-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5) 2> 2198221 WARN (closeThreadPool-3084-thread-2) [n:127.0.0.1:42517_dql%2Fqo] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info 2> 2199342 INFO (closeThreadPool-3084-thread-2) [n:127.0.0.1:42517_dql%2Fqo] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-4-001/cores 2> 2199757 INFO (closeThreadPool-3084-thread-2) [n:127.0.0.1:42517_dql%2Fqo] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 2199871 INFO (closeThreadPool-3084-thread-2) [n:127.0.0.1:42517_dql%2Fqo] o.a.s.e.JettySolrRunner Jetty properties: {hostContext=/dql/qo, solrconfig=solrconfig.xml, solr.data.dir=/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/tempDir-001/jetty4, hostPort=42517, coreRootDirectory=/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-4-001/cores, replicaType=NRT} 2> 2199880 INFO (closeThreadPool-3084-thread-2) [] o.a.s.c.AbstractFullDistribZkTestBase waitForLiveNode: 127.0.0.1:42517_dql%2Fqo 2> 2199925 INFO (SocketProxy-Acceptor-37307) [] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=55862,localport=37307], receiveBufferSize: 65536 2> 2199964 INFO (SocketProxy-Acceptor-37307) [] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=36939,localport=53760], receiveBufferSize=65536 2> 2199984 INFO (SocketProxy-Acceptor-37307) [] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=55866,localport=37307], receiveBufferSize: 65536 2> 2199993 INFO (SocketProxy-Acceptor-37307) [] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=36939,localport=53768], receiveBufferSize=65536 2> 2199994 INFO (SocketProxy-Acceptor-37307) [] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=55868,localport=37307], receiveBufferSize: 65536 2> 2200012 INFO (SocketProxy-Acceptor-37307) [] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=36939,localport=53770], receiveBufferSize=65536 2> 2200013 INFO (SocketProxy-Acceptor-37307) [] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=55870,localport=37307], receiveBufferSize: 65536 2> 2200032 INFO (SocketProxy-Acceptor-37307) [] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=36939,localport=53772], receiveBufferSize=65536 2> 2200208 INFO (OverseerThreadFactory-3065-thread-3) [ c:collection1 s:shard1] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:45565_dql%2Fqo for creating new replica of shard shard1 for collection collection1 2> 2200276 INFO (OverseerThreadFactory-3065-thread-3) [ c:collection1 s:shard1] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command. 2> 2200376 INFO (OverseerStateUpdate-72080878792015876-127.0.0.1:42333_dql%2Fqo-n_0000000000) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"collection1_shard1_replica_n1", 2> "node_name":"127.0.0.1:45565_dql%2Fqo", 2> "base_url":"http://127.0.0.1:45565/dql/qo", 2> "collection":"collection1", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"addreplica"} 2> 2200424 INFO (zkCallback-3059-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2200425 INFO (zkCallback-3059-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2200452 INFO (SocketProxy-Acceptor-45565) [] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=44444,localport=45565], receiveBufferSize: 65536 2> 2200455 INFO (SocketProxy-Acceptor-45565) [] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=44821,localport=34138], receiveBufferSize=65536 2> 2200535 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo x:collection1_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&name=collection1_shard1_replica_n1&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 2200864 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 2200955 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema Schema name=test 2> 2201109 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id 2> 2201472 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n1' using configuration from configset conf1, trusted=true 2> 2201475 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-2-001/cores/collection1_shard1_replica_n1], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-2-001/cores/collection1_shard1_replica_n1/data/] 2> 2201499 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=34, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022208E12, noCFSRatio=0.0] 2> 2201512 WARN (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A, b=B}}} 2> 2202055 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 2202069 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 2202070 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 2202089 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 2202090 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 2202108 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.tests.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=919818910, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022208E12, noCFSRatio=0.1] 2> 2202125 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 2202129 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 2202135 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 2202137 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778722034302320640 2> 2202201 INFO (searcherExecutor-3160-thread-1-processing-127.0.0.1:45565_dql%2Fqo collection1_shard1_replica_n1 collection1 shard1 core_node2) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 2202248 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node2=0}, version=0} for registerTerm 2> 2202248 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1 2> 2202339 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 2202341 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 2202343 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:45565/dql/qo/collection1_shard1_replica_n1/ 2> 2202350 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 2202356 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:45565/dql/qo/collection1_shard1_replica_n1/ has no replicas 2> 2202357 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/72080878792015885-core_node2-n_0000000000 2> 2202395 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:45565/dql/qo/collection1_shard1_replica_n1/ shard1 2> 2202515 INFO (zkCallback-3120-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2202515 INFO (zkCallback-3120-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2202525 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 2202566 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&name=collection1_shard1_replica_n1&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2032 2> 2202660 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo c:collection1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:45565_dql%252Fqo&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=2616 2> 2202667 INFO (zkCallback-3120-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2202668 INFO (zkCallback-3120-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2202670 INFO (zkCallback-3120-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2204260 INFO (OverseerCollectionConfigSetProcessor-72080878792015876-127.0.0.1:42333_dql%2Fqo-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 2204292 INFO (OverseerThreadFactory-3065-thread-4) [ c:collection1 s:shard1] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:37307_dql%2Fqo for creating new replica of shard shard1 for collection collection1 2> 2204404 INFO (OverseerThreadFactory-3065-thread-4) [ c:collection1 s:shard1] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command. 2> 2204465 INFO (OverseerStateUpdate-72080878792015876-127.0.0.1:42333_dql%2Fqo-n_0000000000) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"collection1_shard1_replica_n3", 2> "node_name":"127.0.0.1:37307_dql%2Fqo", 2> "base_url":"http://127.0.0.1:37307/dql/qo", 2> "collection":"collection1", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"addreplica"} 2> 2204622 INFO (zkCallback-3120-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2204622 INFO (zkCallback-3120-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2204622 INFO (zkCallback-3120-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2204623 INFO (zkCallback-3059-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2204645 INFO (SocketProxy-Acceptor-37307) [] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=55892,localport=37307], receiveBufferSize: 65536 2> 2204665 INFO (SocketProxy-Acceptor-37307) [] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=36939,localport=53790], receiveBufferSize=65536 2> 2204711 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo x:collection1_shard1_replica_n3] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf1&name=collection1_shard1_replica_n3&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 2204982 INFO (zkCallback-3120-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2204983 INFO (zkCallback-3120-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2204982 INFO (zkCallback-3120-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2205165 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 2205268 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.s.IndexSchema Schema name=test 2> 2205459 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id 2> 2205860 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n3' using configuration from configset conf1, trusted=true 2> 2205863 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-3-001/cores/collection1_shard1_replica_n3], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-3-001/cores/collection1_shard1_replica_n3/data/] 2> 2205899 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=34, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022208E12, noCFSRatio=0.0] 2> 2205918 WARN (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A, b=B}}} 2> 2206574 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 2206610 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 2206611 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 2206642 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 2206642 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 2206669 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.tests.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=919818910, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022208E12, noCFSRatio=0.1] 2> 2206712 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 2206724 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 2206730 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 2206731 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778722039120527360 2> 2206865 INFO (searcherExecutor-3167-thread-1-processing-127.0.0.1:37307_dql%2Fqo collection1_shard1_replica_n3 collection1 shard1 core_node4) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 2206933 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node2=0, core_node4=0}, version=1} for registerTerm 2> 2206935 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1 2> 2207063 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.ZkController Core needs to recover:collection1_shard1_replica_n3 2> 2207072 INFO (updateExecutor-3128-thread-1-processing-127.0.0.1:37307_dql%2Fqo collection1_shard1_replica_n3 collection1 shard1 core_node4) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.DefaultSolrCoreState Running recovery 2> 2207078 INFO (recoveryExecutor-3130-thread-1-processing-127.0.0.1:37307_dql%2Fqo collection1_shard1_replica_n3 collection1 shard1 core_node4) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true 2> 2207079 INFO (recoveryExecutor-3130-thread-1-processing-127.0.0.1:37307_dql%2Fqo collection1_shard1_replica_n3 collection1 shard1 core_node4) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy startupVersions is empty 2> 2207129 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf1&name=collection1_shard1_replica_n3&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2419 2> 2207131 INFO (SocketProxy-Acceptor-45565) [] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=44466,localport=45565], receiveBufferSize: 65536 2> 2207177 INFO (SocketProxy-Acceptor-45565) [] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=44821,localport=34160], receiveBufferSize=65536 2> 2207213 INFO (qtp1570696396-4789) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.S.Request webapp=/dql/qo path=/admin/ping params={wt=javabin&version=2} hits=0 status=0 QTime=7 2> 2207213 INFO (qtp1570696396-4789) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.S.Request webapp=/dql/qo path=/admin/ping params={wt=javabin&version=2} status=0 QTime=7 2> 2207225 INFO (recoveryExecutor-3130-thread-1-processing-127.0.0.1:37307_dql%2Fqo collection1_shard1_replica_n3 collection1 shard1 core_node4) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1_shard1_replica_n3] 2> 2207232 INFO (recoveryExecutor-3130-thread-1-processing-127.0.0.1:37307_dql%2Fqo collection1_shard1_replica_n3 collection1 shard1 core_node4) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null} 2> 2207233 INFO (recoveryExecutor-3130-thread-1-processing-127.0.0.1:37307_dql%2Fqo collection1_shard1_replica_n3 collection1 shard1 core_node4) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy Publishing state of core [collection1_shard1_replica_n3] as recovering, leader is [http://127.0.0.1:45565/dql/qo/collection1_shard1_replica_n1/] and I am [http://127.0.0.1:37307/dql/qo/collection1_shard1_replica_n3/] 2> 2207258 INFO (recoveryExecutor-3130-thread-1-processing-127.0.0.1:37307_dql%2Fqo collection1_shard1_replica_n3 collection1 shard1 core_node4) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:45565/dql/qo]; [WaitForState: action=PREPRECOVERY&core=collection1_shard1_replica_n1&nodeName=127.0.0.1:37307_dql%252Fqo&coreNodeName=core_node4&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true] 2> 2207273 INFO (qtp1570696396-4788) [n:127.0.0.1:45565_dql%2Fqo x:collection1_shard1_replica_n1] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true 2> 2207284 INFO (qtp1570696396-4788) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:37307_dql%2Fqo, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{ 2> "core":"collection1_shard1_replica_n3", 2> "node_name":"127.0.0.1:37307_dql%2Fqo", 2> "base_url":"http://127.0.0.1:37307/dql/qo", 2> "state":"down", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 2207285 INFO (qtp1570696396-4788) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:37307_dql%2Fqo, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{ 2> "core":"collection1_shard1_replica_n3", 2> "node_name":"127.0.0.1:37307_dql%2Fqo", 2> "base_url":"http://127.0.0.1:37307/dql/qo", 2> "state":"down", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 2207287 INFO (qtp1570696396-4788) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:37307_dql%2Fqo, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{ 2> "core":"collection1_shard1_replica_n3", 2> "node_name":"127.0.0.1:37307_dql%2Fqo", 2> "base_url":"http://127.0.0.1:37307/dql/qo", 2> "state":"down", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 2207312 INFO (qtp568058158-4811) [n:127.0.0.1:37307_dql%2Fqo c:collection1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:37307_dql%252Fqo&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=7229 2> 2207368 INFO (zkCallback-3120-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2207368 INFO (zkCallback-3120-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2207369 INFO (zkCallback-3133-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2207369 INFO (zkCallback-3133-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2207368 INFO (zkCallback-3120-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2207381 INFO (zkCallback-3133-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2207388 INFO (watches-3122-thread-1) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:37307_dql%2Fqo, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{ 2> "core":"collection1_shard1_replica_n3", 2> "node_name":"127.0.0.1:37307_dql%2Fqo", 2> "base_url":"http://127.0.0.1:37307/dql/qo", 2> "state":"recovering", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 2207405 INFO (qtp1570696396-4788) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:37307_dql%252Fqo&onlyIfLeaderActive=true&core=collection1_shard1_replica_n1&coreNodeName=core_node4&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=133 2> 2207909 INFO (recoveryExecutor-3130-thread-1-processing-127.0.0.1:37307_dql%2Fqo collection1_shard1_replica_n3 collection1 shard1 core_node4) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:45565/dql/qo/collection1_shard1_replica_n1/] - recoveringAfterStartup=[true] 2> 2207911 WARN (recoveryExecutor-3130-thread-1-processing-127.0.0.1:37307_dql%2Fqo collection1_shard1_replica_n3 collection1 shard1 core_node4) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.PeerSyncWithLeader no frame of reference to tell if we've missed updates 2> 2207912 INFO (recoveryExecutor-3130-thread-1-processing-127.0.0.1:37307_dql%2Fqo collection1_shard1_replica_n3 collection1 shard1 core_node4) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy PeerSync Recovery was not successful - trying replication. 2> 2207912 INFO (recoveryExecutor-3130-thread-1-processing-127.0.0.1:37307_dql%2Fqo collection1_shard1_replica_n3 collection1 shard1 core_node4) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy Starting Replication Recovery. 2> 2207913 INFO (recoveryExecutor-3130-thread-1-processing-127.0.0.1:37307_dql%2Fqo collection1_shard1_replica_n3 collection1 shard1 core_node4) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy Attempting to replicate from [http://127.0.0.1:45565/dql/qo/collection1_shard1_replica_n1/]. 2> 2207970 INFO (SocketProxy-Acceptor-37307) [] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=55904,localport=37307], receiveBufferSize: 65536 2> 2207976 INFO (SocketProxy-Acceptor-37307) [] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=36939,localport=53802], receiveBufferSize=65536 2> 2208040 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.p.DistributedUpdateProcessor Ignoring commit while not ACTIVE - state: BUFFERING replay: false 2> 2208040 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory webapp=/dql/qo path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:45565/dql/qo/collection1_shard1_replica_n1/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 19 2> 2208049 INFO (qtp1570696396-4785) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory webapp=/dql/qo path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 131 2> 2208060 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.S.Request webapp=/dql/qo path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 2208064 INFO (recoveryExecutor-3130-thread-1-processing-127.0.0.1:37307_dql%2Fqo collection1_shard1_replica_n3 collection1 shard1 core_node4) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.h.IndexFetcher Leader's generation: 1 2> 2208064 INFO (recoveryExecutor-3130-thread-1-processing-127.0.0.1:37307_dql%2Fqo collection1_shard1_replica_n3 collection1 shard1 core_node4) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.h.IndexFetcher Leader's version: 0 2> 2208065 INFO (recoveryExecutor-3130-thread-1-processing-127.0.0.1:37307_dql%2Fqo collection1_shard1_replica_n3 collection1 shard1 core_node4) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.h.IndexFetcher Follower's generation: 1 2> 2208065 INFO (recoveryExecutor-3130-thread-1-processing-127.0.0.1:37307_dql%2Fqo collection1_shard1_replica_n3 collection1 shard1 core_node4) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.h.IndexFetcher Follower's version: 0 2> 2208065 INFO (recoveryExecutor-3130-thread-1-processing-127.0.0.1:37307_dql%2Fqo collection1_shard1_replica_n3 collection1 shard1 core_node4) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy No replay needed. 2> 2208074 INFO (recoveryExecutor-3130-thread-1-processing-127.0.0.1:37307_dql%2Fqo collection1_shard1_replica_n3 collection1 shard1 core_node4) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy Replication Recovery was successful. 2> 2208074 INFO (recoveryExecutor-3130-thread-1-processing-127.0.0.1:37307_dql%2Fqo collection1_shard1_replica_n3 collection1 shard1 core_node4) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy Registering as Active after recovery. 2> 2208080 INFO (recoveryExecutor-3130-thread-1-processing-127.0.0.1:37307_dql%2Fqo collection1_shard1_replica_n3 collection1 shard1 core_node4) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy Updating version bucket highest from index after successful recovery. 2> 2208081 INFO (recoveryExecutor-3130-thread-1-processing-127.0.0.1:37307_dql%2Fqo collection1_shard1_replica_n3 collection1 shard1 core_node4) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778722040536104960 2> 2208095 INFO (recoveryExecutor-3130-thread-1-processing-127.0.0.1:37307_dql%2Fqo collection1_shard1_replica_n3 collection1 shard1 core_node4) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy Finished recovery process, successful=[true] msTimeTaken=1016.0 2> 2208095 INFO (recoveryExecutor-3130-thread-1-processing-127.0.0.1:37307_dql%2Fqo collection1_shard1_replica_n3 collection1 shard1 core_node4) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.RecoveryStrategy Finished recovery process. recoveringAfterStartup=true msTimeTaken=1016.0 2> 2208209 INFO (zkCallback-3133-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2208209 INFO (zkCallback-3133-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2208209 INFO (zkCallback-3120-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2208209 INFO (zkCallback-3120-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2208209 INFO (zkCallback-3133-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2208210 INFO (zkCallback-3120-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2208336 INFO (OverseerCollectionConfigSetProcessor-72080878792015876-127.0.0.1:42333_dql%2Fqo-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 2208368 INFO (OverseerThreadFactory-3065-thread-5) [ c:collection1 s:shard1] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:39977_dql%2Fqo for creating new replica of shard shard1 for collection collection1 2> 2208423 INFO (OverseerThreadFactory-3065-thread-5) [ c:collection1 s:shard1] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command. 2> 2208455 INFO (OverseerStateUpdate-72080878792015876-127.0.0.1:42333_dql%2Fqo-n_0000000000) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"collection1_shard1_replica_n5", 2> "node_name":"127.0.0.1:39977_dql%2Fqo", 2> "base_url":"http://127.0.0.1:39977/dql/qo", 2> "collection":"collection1", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"addreplica"} 2> 2208599 INFO (zkCallback-3120-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2208599 INFO (zkCallback-3120-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2208599 INFO (zkCallback-3120-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2208599 INFO (zkCallback-3133-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2208599 INFO (zkCallback-3133-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2208600 INFO (zkCallback-3133-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2208602 INFO (zkCallback-3059-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2208632 INFO (SocketProxy-Acceptor-39977) [] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=37306,localport=39977], receiveBufferSize: 65536 2> 2208659 INFO (SocketProxy-Acceptor-39977) [] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=43375,localport=32944], receiveBufferSize=65536 2> 2208735 INFO (qtp725172967-4762) [n:127.0.0.1:39977_dql%2Fqo x:collection1_shard1_replica_n5] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf1&name=collection1_shard1_replica_n5&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 2208980 INFO (zkCallback-3133-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2208984 INFO (zkCallback-3133-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2208985 INFO (zkCallback-3133-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2209000 INFO (zkCallback-3120-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2209000 INFO (zkCallback-3120-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2209000 INFO (zkCallback-3120-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2209129 INFO (qtp725172967-4762) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 2209628 INFO (qtp725172967-4762) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.s.IndexSchema Schema name=test 2> 2209741 INFO (qtp725172967-4762) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id 2> 2210171 INFO (qtp725172967-4762) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n5' using configuration from configset conf1, trusted=true 2> 2210174 INFO (qtp725172967-4762) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-1-001/cores/collection1_shard1_replica_n5], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-1-001/cores/collection1_shard1_replica_n5/data/] 2> 2210222 INFO (qtp725172967-4762) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=34, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022208E12, noCFSRatio=0.0] 2> 2210237 WARN (qtp725172967-4762) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A, b=B}}} 2> 2211038 INFO (qtp725172967-4762) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 2211070 INFO (qtp725172967-4762) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 2211071 INFO (qtp725172967-4762) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 2211091 INFO (qtp725172967-4762) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 2211092 INFO (qtp725172967-4762) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 2211112 INFO (qtp725172967-4762) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.tests.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=919818910, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022208E12, noCFSRatio=0.1] 2> 2211139 INFO (qtp725172967-4762) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 2211144 INFO (qtp725172967-4762) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 2211152 INFO (qtp725172967-4762) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 2211154 INFO (qtp725172967-4762) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778722043758379008 2> 2211247 INFO (searcherExecutor-3175-thread-1-processing-127.0.0.1:39977_dql%2Fqo collection1_shard1_replica_n5 collection1 shard1 core_node6) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 2211251 INFO (qtp725172967-4762) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node2=0, core_node6=0, core_node4=0}, version=2} for registerTerm 2> 2211252 INFO (qtp725172967-4762) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1 2> 2211333 INFO (qtp725172967-4762) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.c.ZkController Core needs to recover:collection1_shard1_replica_n5 2> 2211340 INFO (updateExecutor-3092-thread-1-processing-127.0.0.1:39977_dql%2Fqo collection1_shard1_replica_n5 collection1 shard1 core_node6) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.u.DefaultSolrCoreState Running recovery 2> 2211355 INFO (recoveryExecutor-3094-thread-1-processing-127.0.0.1:39977_dql%2Fqo collection1_shard1_replica_n5 collection1 shard1 core_node6) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true 2> 2211356 INFO (recoveryExecutor-3094-thread-1-processing-127.0.0.1:39977_dql%2Fqo collection1_shard1_replica_n5 collection1 shard1 core_node6) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.c.RecoveryStrategy startupVersions is empty 2> 2211363 INFO (SocketProxy-Acceptor-45565) [] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=44478,localport=45565], receiveBufferSize: 65536 2> 2211380 INFO (SocketProxy-Acceptor-45565) [] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=44821,localport=34172], receiveBufferSize=65536 2> 2211390 INFO (qtp725172967-4762) [n:127.0.0.1:39977_dql%2Fqo] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf1&name=collection1_shard1_replica_n5&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2657 2> 2211434 INFO (qtp568058158-4806) [n:127.0.0.1:37307_dql%2Fqo c:collection1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:39977_dql%252Fqo&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=11344 2> 2211441 INFO (qtp1570696396-4789) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.S.Request webapp=/dql/qo path=/admin/ping params={wt=javabin&version=2} hits=0 status=0 QTime=11 2> 2211441 INFO (qtp1570696396-4789) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.S.Request webapp=/dql/qo path=/admin/ping params={wt=javabin&version=2} status=0 QTime=12 2> 2211443 INFO (recoveryExecutor-3094-thread-1-processing-127.0.0.1:39977_dql%2Fqo collection1_shard1_replica_n5 collection1 shard1 core_node6) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1_shard1_replica_n5] 2> 2211447 INFO (recoveryExecutor-3094-thread-1-processing-127.0.0.1:39977_dql%2Fqo collection1_shard1_replica_n5 collection1 shard1 core_node6) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null} 2> 2211447 INFO (recoveryExecutor-3094-thread-1-processing-127.0.0.1:39977_dql%2Fqo collection1_shard1_replica_n5 collection1 shard1 core_node6) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.c.RecoveryStrategy Publishing state of core [collection1_shard1_replica_n5] as recovering, leader is [http://127.0.0.1:45565/dql/qo/collection1_shard1_replica_n1/] and I am [http://127.0.0.1:39977/dql/qo/collection1_shard1_replica_n5/] 2> 2211460 INFO (recoveryExecutor-3094-thread-1-processing-127.0.0.1:39977_dql%2Fqo collection1_shard1_replica_n5 collection1 shard1 core_node6) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:45565/dql/qo]; [WaitForState: action=PREPRECOVERY&core=collection1_shard1_replica_n1&nodeName=127.0.0.1:39977_dql%252Fqo&coreNodeName=core_node6&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true] 2> 2211480 INFO (zkCallback-3099-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2211481 INFO (zkCallback-3120-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2211481 INFO (zkCallback-3120-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2211481 INFO (zkCallback-3120-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2211482 INFO (zkCallback-3133-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2211482 INFO (qtp1570696396-4788) [n:127.0.0.1:45565_dql%2Fqo x:collection1_shard1_replica_n1] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node6, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true 2> 2211483 INFO (zkCallback-3133-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2211488 INFO (qtp1570696396-4788) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:39977_dql%2Fqo, coreNodeName=core_node6, onlyIfActiveCheckResult=false, nodeProps: core_node6:{ 2> "core":"collection1_shard1_replica_n5", 2> "node_name":"127.0.0.1:39977_dql%2Fqo", 2> "base_url":"http://127.0.0.1:39977/dql/qo", 2> "state":"down", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 2211480 INFO (zkCallback-3099-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2211488 INFO (zkCallback-3133-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2211494 INFO (qtp1570696396-4788) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:39977_dql%2Fqo, coreNodeName=core_node6, onlyIfActiveCheckResult=false, nodeProps: core_node6:{ 2> "core":"collection1_shard1_replica_n5", 2> "node_name":"127.0.0.1:39977_dql%2Fqo", 2> "base_url":"http://127.0.0.1:39977/dql/qo", 2> "state":"down", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 2211495 INFO (qtp1570696396-4788) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:39977_dql%2Fqo, coreNodeName=core_node6, onlyIfActiveCheckResult=false, nodeProps: core_node6:{ 2> "core":"collection1_shard1_replica_n5", 2> "node_name":"127.0.0.1:39977_dql%2Fqo", 2> "base_url":"http://127.0.0.1:39977/dql/qo", 2> "state":"down", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 2211500 INFO (watches-3122-thread-2) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:39977_dql%2Fqo, coreNodeName=core_node6, onlyIfActiveCheckResult=false, nodeProps: core_node6:{ 2> "core":"collection1_shard1_replica_n5", 2> "node_name":"127.0.0.1:39977_dql%2Fqo", 2> "base_url":"http://127.0.0.1:39977/dql/qo", 2> "state":"recovering", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 2211501 INFO (qtp1570696396-4788) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:39977_dql%252Fqo&onlyIfLeaderActive=true&core=collection1_shard1_replica_n1&coreNodeName=core_node6&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=23 2> 2211512 INFO (zkCallback-3099-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2212009 INFO (recoveryExecutor-3094-thread-1-processing-127.0.0.1:39977_dql%2Fqo collection1_shard1_replica_n5 collection1 shard1 core_node6) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:45565/dql/qo/collection1_shard1_replica_n1/] - recoveringAfterStartup=[true] 2> 2212013 WARN (recoveryExecutor-3094-thread-1-processing-127.0.0.1:39977_dql%2Fqo collection1_shard1_replica_n5 collection1 shard1 core_node6) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.u.PeerSyncWithLeader no frame of reference to tell if we've missed updates 2> 2212013 INFO (recoveryExecutor-3094-thread-1-processing-127.0.0.1:39977_dql%2Fqo collection1_shard1_replica_n5 collection1 shard1 core_node6) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.c.RecoveryStrategy PeerSync Recovery was not successful - trying replication. 2> 2212013 INFO (recoveryExecutor-3094-thread-1-processing-127.0.0.1:39977_dql%2Fqo collection1_shard1_replica_n5 collection1 shard1 core_node6) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.c.RecoveryStrategy Starting Replication Recovery. 2> 2212014 INFO (recoveryExecutor-3094-thread-1-processing-127.0.0.1:39977_dql%2Fqo collection1_shard1_replica_n5 collection1 shard1 core_node6) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.c.RecoveryStrategy Attempting to replicate from [http://127.0.0.1:45565/dql/qo/collection1_shard1_replica_n1/]. 2> 2212063 INFO (SocketProxy-Acceptor-39977) [] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=37314,localport=39977], receiveBufferSize: 65536 2> 2212064 INFO (qtp568058158-4811) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory webapp=/dql/qo path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:45565/dql/qo/collection1_shard1_replica_n1/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 15 2> 2212068 INFO (SocketProxy-Acceptor-39977) [] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=43375,localport=32952], receiveBufferSize=65536 2> 2212150 INFO (qtp725172967-4766) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.u.p.DistributedUpdateProcessor Ignoring commit while not ACTIVE - state: BUFFERING replay: false 2> 2212150 INFO (qtp725172967-4766) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.u.p.LogUpdateProcessorFactory webapp=/dql/qo path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:45565/dql/qo/collection1_shard1_replica_n1/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 30 2> 2212155 INFO (qtp1570696396-4784) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory webapp=/dql/qo path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 137 2> 2212167 INFO (qtp1570696396-4785) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.S.Request webapp=/dql/qo path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 2212170 INFO (recoveryExecutor-3094-thread-1-processing-127.0.0.1:39977_dql%2Fqo collection1_shard1_replica_n5 collection1 shard1 core_node6) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.h.IndexFetcher Leader's generation: 1 2> 2212172 INFO (recoveryExecutor-3094-thread-1-processing-127.0.0.1:39977_dql%2Fqo collection1_shard1_replica_n5 collection1 shard1 core_node6) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.h.IndexFetcher Leader's version: 0 2> 2212173 INFO (recoveryExecutor-3094-thread-1-processing-127.0.0.1:39977_dql%2Fqo collection1_shard1_replica_n5 collection1 shard1 core_node6) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.h.IndexFetcher Follower's generation: 1 2> 2212174 INFO (recoveryExecutor-3094-thread-1-processing-127.0.0.1:39977_dql%2Fqo collection1_shard1_replica_n5 collection1 shard1 core_node6) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.h.IndexFetcher Follower's version: 0 2> 2212176 INFO (recoveryExecutor-3094-thread-1-processing-127.0.0.1:39977_dql%2Fqo collection1_shard1_replica_n5 collection1 shard1 core_node6) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.c.RecoveryStrategy No replay needed. 2> 2212190 INFO (recoveryExecutor-3094-thread-1-processing-127.0.0.1:39977_dql%2Fqo collection1_shard1_replica_n5 collection1 shard1 core_node6) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.c.RecoveryStrategy Replication Recovery was successful. 2> 2212191 INFO (recoveryExecutor-3094-thread-1-processing-127.0.0.1:39977_dql%2Fqo collection1_shard1_replica_n5 collection1 shard1 core_node6) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.c.RecoveryStrategy Registering as Active after recovery. 2> 2212201 INFO (recoveryExecutor-3094-thread-1-processing-127.0.0.1:39977_dql%2Fqo collection1_shard1_replica_n5 collection1 shard1 core_node6) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.c.RecoveryStrategy Updating version bucket highest from index after successful recovery. 2> 2212203 INFO (recoveryExecutor-3094-thread-1-processing-127.0.0.1:39977_dql%2Fqo collection1_shard1_replica_n5 collection1 shard1 core_node6) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778722044858335232 2> 2212237 INFO (recoveryExecutor-3094-thread-1-processing-127.0.0.1:39977_dql%2Fqo collection1_shard1_replica_n5 collection1 shard1 core_node6) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.c.RecoveryStrategy Finished recovery process, successful=[true] msTimeTaken=880.0 2> 2212238 INFO (recoveryExecutor-3094-thread-1-processing-127.0.0.1:39977_dql%2Fqo collection1_shard1_replica_n5 collection1 shard1 core_node6) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.c.RecoveryStrategy Finished recovery process. recoveringAfterStartup=true msTimeTaken=882.0 2> 2212388 INFO (zkCallback-3120-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2212388 INFO (zkCallback-3120-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2212389 INFO (zkCallback-3120-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2212390 INFO (zkCallback-3133-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2212390 INFO (zkCallback-3133-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2212391 INFO (zkCallback-3133-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2212396 INFO (zkCallback-3099-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2212396 INFO (zkCallback-3099-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2212400 INFO (zkCallback-3099-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2212431 INFO (OverseerCollectionConfigSetProcessor-72080878792015876-127.0.0.1:42333_dql%2Fqo-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000007 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 2212488 INFO (OverseerThreadFactory-3065-thread-5) [ c:collection1 s:shard1] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:42517_dql%2Fqo for creating new replica of shard shard1 for collection collection1 2> 2212544 INFO (OverseerThreadFactory-3065-thread-5) [ c:collection1 s:shard1] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command. 2> 2212618 INFO (OverseerStateUpdate-72080878792015876-127.0.0.1:42333_dql%2Fqo-n_0000000000) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"collection1_shard1_replica_n7", 2> "node_name":"127.0.0.1:42517_dql%2Fqo", 2> "base_url":"http://127.0.0.1:42517/dql/qo", 2> "collection":"collection1", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"addreplica"} 2> 2212805 INFO (zkCallback-3099-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2212805 INFO (zkCallback-3120-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2212805 INFO (zkCallback-3099-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2212805 INFO (zkCallback-3133-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2212806 INFO (zkCallback-3133-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2212806 INFO (zkCallback-3099-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2212806 INFO (zkCallback-3133-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2212806 INFO (zkCallback-3120-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2212807 INFO (zkCallback-3120-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2212810 INFO (zkCallback-3059-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2212826 INFO (SocketProxy-Acceptor-42517) [] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=46008,localport=42517], receiveBufferSize: 65536 2> 2212838 INFO (SocketProxy-Acceptor-42517) [] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=45985,localport=50316], receiveBufferSize=65536 2> 2212927 INFO (qtp2069714454-4849) [n:127.0.0.1:42517_dql%2Fqo x:collection1_shard1_replica_n7] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf1&name=collection1_shard1_replica_n7&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 2213194 INFO (zkCallback-3099-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2213194 INFO (zkCallback-3099-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2213194 INFO (zkCallback-3133-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2213194 INFO (zkCallback-3133-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2213194 INFO (zkCallback-3120-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2213195 INFO (zkCallback-3133-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2213195 INFO (zkCallback-3120-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2213195 INFO (zkCallback-3120-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2213194 INFO (zkCallback-3099-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2213297 INFO (qtp2069714454-4849) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 2213481 INFO (qtp2069714454-4849) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.s.IndexSchema Schema name=test 2> 2213647 INFO (qtp2069714454-4849) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id 2> 2214118 INFO (qtp2069714454-4849) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n7' using configuration from configset conf1, trusted=true 2> 2214150 INFO (qtp2069714454-4849) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-4-001/cores/collection1_shard1_replica_n7], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-4-001/cores/collection1_shard1_replica_n7/data/] 2> 2214209 INFO (qtp2069714454-4849) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=34, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022208E12, noCFSRatio=0.0] 2> 2214225 WARN (qtp2069714454-4849) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A, b=B}}} 2> 2214893 INFO (qtp2069714454-4849) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 2214916 INFO (qtp2069714454-4849) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 2214917 INFO (qtp2069714454-4849) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 2214979 INFO (qtp2069714454-4849) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 2214980 INFO (qtp2069714454-4849) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 2215025 INFO (qtp2069714454-4849) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.tests.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=919818910, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022208E12, noCFSRatio=0.1] 2> 2215084 INFO (qtp2069714454-4849) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 2215094 INFO (qtp2069714454-4849) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 2215106 INFO (qtp2069714454-4849) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 2215107 INFO (qtp2069714454-4849) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778722047903399936 2> 2215204 INFO (searcherExecutor-3183-thread-1-processing-127.0.0.1:42517_dql%2Fqo collection1_shard1_replica_n7 collection1 shard1 core_node8) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 2215317 INFO (qtp2069714454-4849) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node6=0, core_node8=0, core_node2=0, core_node4=0}, version=3} for registerTerm 2> 2215319 INFO (qtp2069714454-4849) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1 2> 2215471 INFO (qtp2069714454-4849) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.c.ZkController Core needs to recover:collection1_shard1_replica_n7 2> 2215477 INFO (updateExecutor-3146-thread-1-processing-127.0.0.1:42517_dql%2Fqo collection1_shard1_replica_n7 collection1 shard1 core_node8) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.u.DefaultSolrCoreState Running recovery 2> 2215481 INFO (recoveryExecutor-3148-thread-1-processing-127.0.0.1:42517_dql%2Fqo collection1_shard1_replica_n7 collection1 shard1 core_node8) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true 2> 2215482 INFO (recoveryExecutor-3148-thread-1-processing-127.0.0.1:42517_dql%2Fqo collection1_shard1_replica_n7 collection1 shard1 core_node8) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.c.RecoveryStrategy startupVersions is empty 2> 2215522 INFO (SocketProxy-Acceptor-45565) [] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=44496,localport=45565], receiveBufferSize: 65536 2> 2215540 INFO (SocketProxy-Acceptor-45565) [] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=44821,localport=34190], receiveBufferSize=65536 2> 2215572 INFO (qtp2069714454-4849) [n:127.0.0.1:42517_dql%2Fqo] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf1&name=collection1_shard1_replica_n7&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2647 2> 2215617 INFO (qtp1570696396-4787) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.S.Request webapp=/dql/qo path=/admin/ping params={wt=javabin&version=2} hits=0 status=0 QTime=7 2> 2215617 INFO (qtp1570696396-4787) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.S.Request webapp=/dql/qo path=/admin/ping params={wt=javabin&version=2} status=0 QTime=8 2> 2215633 INFO (recoveryExecutor-3148-thread-1-processing-127.0.0.1:42517_dql%2Fqo collection1_shard1_replica_n7 collection1 shard1 core_node8) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1_shard1_replica_n7] 2> 2215647 INFO (recoveryExecutor-3148-thread-1-processing-127.0.0.1:42517_dql%2Fqo collection1_shard1_replica_n7 collection1 shard1 core_node8) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null} 2> 2215647 INFO (recoveryExecutor-3148-thread-1-processing-127.0.0.1:42517_dql%2Fqo collection1_shard1_replica_n7 collection1 shard1 core_node8) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.c.RecoveryStrategy Publishing state of core [collection1_shard1_replica_n7] as recovering, leader is [http://127.0.0.1:45565/dql/qo/collection1_shard1_replica_n1/] and I am [http://127.0.0.1:42517/dql/qo/collection1_shard1_replica_n7/] 2> 2215647 INFO (qtp568058158-4807) [n:127.0.0.1:37307_dql%2Fqo c:collection1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:42517_dql%252Fqo&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=15551 2> 2215663 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.AbstractFullDistribZkTestBase Waiting to see 4 active replicas in collection: collection1 2> 2215686 INFO (recoveryExecutor-3148-thread-1-processing-127.0.0.1:42517_dql%2Fqo collection1_shard1_replica_n7 collection1 shard1 core_node8) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:45565/dql/qo]; [WaitForState: action=PREPRECOVERY&core=collection1_shard1_replica_n1&nodeName=127.0.0.1:42517_dql%252Fqo&coreNodeName=core_node8&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true] 2> 2215713 INFO (qtp1570696396-4789) [n:127.0.0.1:45565_dql%2Fqo x:collection1_shard1_replica_n1] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node8, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true 2> 2215720 INFO (qtp1570696396-4789) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:42517_dql%2Fqo, coreNodeName=core_node8, onlyIfActiveCheckResult=false, nodeProps: core_node8:{ 2> "core":"collection1_shard1_replica_n7", 2> "node_name":"127.0.0.1:42517_dql%2Fqo", 2> "base_url":"http://127.0.0.1:42517/dql/qo", 2> "state":"down", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 2215721 INFO (qtp1570696396-4789) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:42517_dql%2Fqo, coreNodeName=core_node8, onlyIfActiveCheckResult=false, nodeProps: core_node8:{ 2> "core":"collection1_shard1_replica_n7", 2> "node_name":"127.0.0.1:42517_dql%2Fqo", 2> "base_url":"http://127.0.0.1:42517/dql/qo", 2> "state":"down", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 2215722 INFO (qtp1570696396-4789) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:42517_dql%2Fqo, coreNodeName=core_node8, onlyIfActiveCheckResult=false, nodeProps: core_node8:{ 2> "core":"collection1_shard1_replica_n7", 2> "node_name":"127.0.0.1:42517_dql%2Fqo", 2> "base_url":"http://127.0.0.1:42517/dql/qo", 2> "state":"down", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 2215750 INFO (zkCallback-3133-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2215751 INFO (zkCallback-3099-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2215751 INFO (zkCallback-3099-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2215751 INFO (zkCallback-3133-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2215751 INFO (zkCallback-3120-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2215751 INFO (zkCallback-3133-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2215751 INFO (zkCallback-3099-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2215752 INFO (zkCallback-3151-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2215753 INFO (zkCallback-3151-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2215753 INFO (zkCallback-3120-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2215754 INFO (zkCallback-3120-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2215754 INFO (zkCallback-3082-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2215774 INFO (zkCallback-3151-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2215777 INFO (watches-3122-thread-3) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:42517_dql%2Fqo, coreNodeName=core_node8, onlyIfActiveCheckResult=false, nodeProps: core_node8:{ 2> "core":"collection1_shard1_replica_n7", 2> "node_name":"127.0.0.1:42517_dql%2Fqo", 2> "base_url":"http://127.0.0.1:42517/dql/qo", 2> "state":"recovering", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 2215780 INFO (qtp1570696396-4789) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:42517_dql%252Fqo&onlyIfLeaderActive=true&core=collection1_shard1_replica_n1&coreNodeName=core_node8&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=68 2> 2216285 INFO (recoveryExecutor-3148-thread-1-processing-127.0.0.1:42517_dql%2Fqo collection1_shard1_replica_n7 collection1 shard1 core_node8) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:45565/dql/qo/collection1_shard1_replica_n1/] - recoveringAfterStartup=[true] 2> 2216288 WARN (recoveryExecutor-3148-thread-1-processing-127.0.0.1:42517_dql%2Fqo collection1_shard1_replica_n7 collection1 shard1 core_node8) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.u.PeerSyncWithLeader no frame of reference to tell if we've missed updates 2> 2216288 INFO (recoveryExecutor-3148-thread-1-processing-127.0.0.1:42517_dql%2Fqo collection1_shard1_replica_n7 collection1 shard1 core_node8) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.c.RecoveryStrategy PeerSync Recovery was not successful - trying replication. 2> 2216289 INFO (recoveryExecutor-3148-thread-1-processing-127.0.0.1:42517_dql%2Fqo collection1_shard1_replica_n7 collection1 shard1 core_node8) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.c.RecoveryStrategy Starting Replication Recovery. 2> 2216289 INFO (recoveryExecutor-3148-thread-1-processing-127.0.0.1:42517_dql%2Fqo collection1_shard1_replica_n7 collection1 shard1 core_node8) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.c.RecoveryStrategy Attempting to replicate from [http://127.0.0.1:45565/dql/qo/collection1_shard1_replica_n1/]. 2> 2216368 INFO (SocketProxy-Acceptor-42517) [] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=46020,localport=42517], receiveBufferSize: 65536 2> 2216376 INFO (SocketProxy-Acceptor-42517) [] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=45985,localport=50328], receiveBufferSize=65536 2> 2216385 INFO (qtp568058158-4808) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory webapp=/dql/qo path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:45565/dql/qo/collection1_shard1_replica_n1/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 40 2> 2216393 INFO (qtp725172967-4761) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.u.p.LogUpdateProcessorFactory webapp=/dql/qo path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:45565/dql/qo/collection1_shard1_replica_n1/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 52 2> 2216444 INFO (qtp2069714454-4853) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.u.p.DistributedUpdateProcessor Ignoring commit while not ACTIVE - state: BUFFERING replay: false 2> 2216445 INFO (qtp2069714454-4853) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.u.p.LogUpdateProcessorFactory webapp=/dql/qo path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:45565/dql/qo/collection1_shard1_replica_n1/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 17 2> 2216453 INFO (qtp1570696396-4788) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory webapp=/dql/qo path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 160 2> 2216470 INFO (qtp1570696396-4784) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.S.Request webapp=/dql/qo path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 2216473 INFO (recoveryExecutor-3148-thread-1-processing-127.0.0.1:42517_dql%2Fqo collection1_shard1_replica_n7 collection1 shard1 core_node8) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.h.IndexFetcher Leader's generation: 1 2> 2216474 INFO (recoveryExecutor-3148-thread-1-processing-127.0.0.1:42517_dql%2Fqo collection1_shard1_replica_n7 collection1 shard1 core_node8) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.h.IndexFetcher Leader's version: 0 2> 2216476 INFO (recoveryExecutor-3148-thread-1-processing-127.0.0.1:42517_dql%2Fqo collection1_shard1_replica_n7 collection1 shard1 core_node8) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.h.IndexFetcher Follower's generation: 1 2> 2216477 INFO (recoveryExecutor-3148-thread-1-processing-127.0.0.1:42517_dql%2Fqo collection1_shard1_replica_n7 collection1 shard1 core_node8) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.h.IndexFetcher Follower's version: 0 2> 2216478 INFO (recoveryExecutor-3148-thread-1-processing-127.0.0.1:42517_dql%2Fqo collection1_shard1_replica_n7 collection1 shard1 core_node8) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.c.RecoveryStrategy No replay needed. 2> 2216484 INFO (recoveryExecutor-3148-thread-1-processing-127.0.0.1:42517_dql%2Fqo collection1_shard1_replica_n7 collection1 shard1 core_node8) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.c.RecoveryStrategy Replication Recovery was successful. 2> 2216485 INFO (recoveryExecutor-3148-thread-1-processing-127.0.0.1:42517_dql%2Fqo collection1_shard1_replica_n7 collection1 shard1 core_node8) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.c.RecoveryStrategy Registering as Active after recovery. 2> 2216495 INFO (recoveryExecutor-3148-thread-1-processing-127.0.0.1:42517_dql%2Fqo collection1_shard1_replica_n7 collection1 shard1 core_node8) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.c.RecoveryStrategy Updating version bucket highest from index after successful recovery. 2> 2216496 INFO (recoveryExecutor-3148-thread-1-processing-127.0.0.1:42517_dql%2Fqo collection1_shard1_replica_n7 collection1 shard1 core_node8) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778722049359872000 2> 2216505 INFO (OverseerCollectionConfigSetProcessor-72080878792015876-127.0.0.1:42333_dql%2Fqo-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000008 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 2216515 INFO (recoveryExecutor-3148-thread-1-processing-127.0.0.1:42517_dql%2Fqo collection1_shard1_replica_n7 collection1 shard1 core_node8) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.c.RecoveryStrategy Finished recovery process, successful=[true] msTimeTaken=1033.0 2> 2216517 INFO (recoveryExecutor-3148-thread-1-processing-127.0.0.1:42517_dql%2Fqo collection1_shard1_replica_n7 collection1 shard1 core_node8) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.c.RecoveryStrategy Finished recovery process. recoveringAfterStartup=true msTimeTaken=1035.0 2> 2216629 INFO (zkCallback-3133-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2216629 INFO (zkCallback-3133-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2216629 INFO (zkCallback-3120-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2216630 INFO (zkCallback-3120-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2216630 INFO (zkCallback-3133-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2216630 INFO (zkCallback-3120-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2216636 INFO (zkCallback-3082-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2216636 INFO (zkCallback-3151-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2216636 INFO (zkCallback-3099-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2216636 INFO (zkCallback-3151-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2216636 INFO (zkCallback-3099-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2216636 INFO (zkCallback-3099-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2216636 INFO (zkCallback-3151-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5]) 2> 2216751 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Starting test 2> 2216752 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.HttpPartitionOnCommitTest Running oneShardTest 2> 2216776 INFO (SocketProxy-Acceptor-45565) [] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=44504,localport=45565], receiveBufferSize: 65536 2> 2216806 INFO (SocketProxy-Acceptor-45565) [] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=44821,localport=34198], receiveBufferSize=65536 2> 2216932 INFO (OverseerThreadFactory-3065-thread-5) [] o.a.s.c.a.c.CreateCollectionCmd Create collection c8n_1x3_commits 2> 2217429 INFO (OverseerStateUpdate-72080878792015876-127.0.0.1:42333_dql%2Fqo-n_0000000000) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"c8n_1x3_commits_shard1_replica_n1", 2> "node_name":"127.0.0.1:37307_dql%2Fqo", 2> "base_url":"http://127.0.0.1:37307/dql/qo", 2> "collection":"c8n_1x3_commits", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 2217509 INFO (OverseerStateUpdate-72080878792015876-127.0.0.1:42333_dql%2Fqo-n_0000000000) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"c8n_1x3_commits_shard1_replica_n2", 2> "node_name":"127.0.0.1:39977_dql%2Fqo", 2> "base_url":"http://127.0.0.1:39977/dql/qo", 2> "collection":"c8n_1x3_commits", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 2217577 INFO (OverseerStateUpdate-72080878792015876-127.0.0.1:42333_dql%2Fqo-n_0000000000) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"c8n_1x3_commits_shard1_replica_n4", 2> "node_name":"127.0.0.1:42333_dql%2Fqo", 2> "base_url":"http://127.0.0.1:42333/dql/qo", 2> "collection":"c8n_1x3_commits", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 2217718 INFO (zkCallback-3059-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/c8n_1x3_commits/state.json] for collection [c8n_1x3_commits] has occurred - updating... (live nodes size: [5]) 2> 2217732 INFO (SocketProxy-Acceptor-37307) [] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=55942,localport=37307], receiveBufferSize: 65536 2> 2217739 INFO (SocketProxy-Acceptor-37307) [] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=36939,localport=53840], receiveBufferSize=65536 2> 2217740 INFO (SocketProxy-Acceptor-39977) [] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=37344,localport=39977], receiveBufferSize: 65536 2> 2217740 INFO (SocketProxy-Acceptor-42333) [] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=53822,localport=42333], receiveBufferSize: 65536 2> 2217753 INFO (SocketProxy-Acceptor-39977) [] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=43375,localport=32984], receiveBufferSize=65536 2> 2217801 INFO (SocketProxy-Acceptor-42333) [] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=39373,localport=40174], receiveBufferSize=65536 2> 2217818 INFO (qtp568058158-4811) [n:127.0.0.1:37307_dql%2Fqo x:c8n_1x3_commits_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=c8n_1x3_commits_shard1_replica_n1&action=CREATE&numShards=1&collection=c8n_1x3_commits&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 2217824 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo x:c8n_1x3_commits_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf1&newCollection=true&name=c8n_1x3_commits_shard1_replica_n2&action=CREATE&numShards=1&collection=c8n_1x3_commits&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 2217836 INFO (qtp1257576055-4707) [n:127.0.0.1:42333_dql%2Fqo x:c8n_1x3_commits_shard1_replica_n4] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf1&newCollection=true&name=c8n_1x3_commits_shard1_replica_n4&action=CREATE&numShards=1&collection=c8n_1x3_commits&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 2218057 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 2218083 INFO (qtp1257576055-4707) [n:127.0.0.1:42333_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node6 x:c8n_1x3_commits_shard1_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 2218098 INFO (qtp568058158-4811) [n:127.0.0.1:37307_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node3 x:c8n_1x3_commits_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 2218100 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.s.IndexSchema Schema name=test 2> 2218139 INFO (qtp1257576055-4707) [n:127.0.0.1:42333_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node6 x:c8n_1x3_commits_shard1_replica_n4] o.a.s.s.IndexSchema Schema name=test 2> 2218152 INFO (qtp568058158-4811) [n:127.0.0.1:37307_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node3 x:c8n_1x3_commits_shard1_replica_n1] o.a.s.s.IndexSchema Schema name=test 2> 2218295 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id 2> 2218411 INFO (qtp1257576055-4707) [n:127.0.0.1:42333_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node6 x:c8n_1x3_commits_shard1_replica_n4] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id 2> 2218438 INFO (qtp568058158-4811) [n:127.0.0.1:37307_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node3 x:c8n_1x3_commits_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id 2> 2218816 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'c8n_1x3_commits_shard1_replica_n2' using configuration from configset conf1, trusted=true 2> 2218824 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-1-001/cores/c8n_1x3_commits_shard1_replica_n2], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-1-001/cores/c8n_1x3_commits_shard1_replica_n2/data/] 2> 2218899 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=34, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022208E12, noCFSRatio=0.0] 2> 2218925 WARN (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A, b=B}}} 2> 2219018 INFO (qtp1257576055-4707) [n:127.0.0.1:42333_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node6 x:c8n_1x3_commits_shard1_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'c8n_1x3_commits_shard1_replica_n4' using configuration from configset conf1, trusted=true 2> 2219023 INFO (qtp1257576055-4707) [n:127.0.0.1:42333_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node6 x:c8n_1x3_commits_shard1_replica_n4] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/control-001/cores/c8n_1x3_commits_shard1_replica_n4], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/control-001/cores/c8n_1x3_commits_shard1_replica_n4/data/] 2> 2219113 INFO (qtp568058158-4811) [n:127.0.0.1:37307_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node3 x:c8n_1x3_commits_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'c8n_1x3_commits_shard1_replica_n1' using configuration from configset conf1, trusted=true 2> 2219118 INFO (qtp568058158-4811) [n:127.0.0.1:37307_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node3 x:c8n_1x3_commits_shard1_replica_n1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-3-001/cores/c8n_1x3_commits_shard1_replica_n1], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001/shard-3-001/cores/c8n_1x3_commits_shard1_replica_n1/data/] 2> 2219138 INFO (qtp1257576055-4707) [n:127.0.0.1:42333_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node6 x:c8n_1x3_commits_shard1_replica_n4] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=34, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022208E12, noCFSRatio=0.0] 2> 2219186 INFO (qtp568058158-4811) [n:127.0.0.1:37307_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node3 x:c8n_1x3_commits_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=34, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022208E12, noCFSRatio=0.0] 2> 2219193 WARN (qtp1257576055-4707) [n:127.0.0.1:42333_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node6 x:c8n_1x3_commits_shard1_replica_n4] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A, b=B}}} 2> 2219239 WARN (qtp568058158-4811) [n:127.0.0.1:37307_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node3 x:c8n_1x3_commits_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A, b=B}}} 2> 2221018 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 2221018 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 2221053 INFO (qtp568058158-4811) [n:127.0.0.1:37307_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node3 x:c8n_1x3_commits_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 2221054 INFO (qtp568058158-4811) [n:127.0.0.1:37307_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node3 x:c8n_1x3_commits_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 2221060 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 2221061 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 2221077 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.tests.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=919818910, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022208E12, noCFSRatio=0.1] 2> 2221103 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 2221112 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 2221122 INFO (qtp568058158-4811) [n:127.0.0.1:37307_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node3 x:c8n_1x3_commits_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 2221123 INFO (qtp568058158-4811) [n:127.0.0.1:37307_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node3 x:c8n_1x3_commits_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 2221146 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 2221153 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778722054243090432 2> 2221201 INFO (qtp568058158-4811) [n:127.0.0.1:37307_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node3 x:c8n_1x3_commits_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.tests.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=919818910, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022208E12, noCFSRatio=0.1] 2> 2221261 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/c8n_1x3_commits/terms/shard1 to Terms{values={core_node5=0}, version=0} for registerTerm 2> 2221283 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/c8n_1x3_commits/leaders/shard1 2> 2221288 INFO (qtp568058158-4811) [n:127.0.0.1:37307_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node3 x:c8n_1x3_commits_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 2221296 INFO (qtp568058158-4811) [n:127.0.0.1:37307_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node3 x:c8n_1x3_commits_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 2221313 INFO (qtp568058158-4811) [n:127.0.0.1:37307_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node3 x:c8n_1x3_commits_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 2221314 INFO (qtp568058158-4811) [n:127.0.0.1:37307_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node3 x:c8n_1x3_commits_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778722054411911168 2> 2221345 INFO (qtp1257576055-4707) [n:127.0.0.1:42333_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node6 x:c8n_1x3_commits_shard1_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 2221345 INFO (qtp1257576055-4707) [n:127.0.0.1:42333_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node6 x:c8n_1x3_commits_shard1_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 2221352 INFO (searcherExecutor-3191-thread-1-processing-127.0.0.1:39977_dql%2Fqo c8n_1x3_commits_shard1_replica_n2 c8n_1x3_commits shard1 core_node5) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 2221397 INFO (qtp1257576055-4707) [n:127.0.0.1:42333_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node6 x:c8n_1x3_commits_shard1_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 2221404 INFO (qtp1257576055-4707) [n:127.0.0.1:42333_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node6 x:c8n_1x3_commits_shard1_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 2221503 INFO (searcherExecutor-3195-thread-1-processing-127.0.0.1:37307_dql%2Fqo c8n_1x3_commits_shard1_replica_n1 c8n_1x3_commits shard1 core_node3) [n:127.0.0.1:37307_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node3 x:c8n_1x3_commits_shard1_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 2221524 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=3 found=1 timeoute in=14980ms 2> 2221555 INFO (qtp1257576055-4707) [n:127.0.0.1:42333_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node6 x:c8n_1x3_commits_shard1_replica_n4] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.tests.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=919818910, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022208E12, noCFSRatio=0.1] 2> 2221594 INFO (qtp1257576055-4707) [n:127.0.0.1:42333_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node6 x:c8n_1x3_commits_shard1_replica_n4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 2221612 INFO (qtp1257576055-4707) [n:127.0.0.1:42333_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node6 x:c8n_1x3_commits_shard1_replica_n4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 2221618 INFO (qtp1257576055-4707) [n:127.0.0.1:42333_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node6 x:c8n_1x3_commits_shard1_replica_n4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 2221619 INFO (qtp1257576055-4707) [n:127.0.0.1:42333_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node6 x:c8n_1x3_commits_shard1_replica_n4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778722054731726848 2> 2221628 INFO (qtp568058158-4811) [n:127.0.0.1:37307_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node3 x:c8n_1x3_commits_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/c8n_1x3_commits/terms/shard1 to Terms{values={core_node3=0, core_node5=0}, version=1} for registerTerm 2> 2221628 INFO (qtp568058158-4811) [n:127.0.0.1:37307_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node3 x:c8n_1x3_commits_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/c8n_1x3_commits/leaders/shard1 2> 2221718 INFO (searcherExecutor-3193-thread-1-processing-127.0.0.1:42333_dql%2Fqo c8n_1x3_commits_shard1_replica_n4 c8n_1x3_commits shard1 core_node6) [n:127.0.0.1:42333_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node6 x:c8n_1x3_commits_shard1_replica_n4] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 2221850 INFO (qtp1257576055-4707) [n:127.0.0.1:42333_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node6 x:c8n_1x3_commits_shard1_replica_n4] o.a.s.c.ZkShardTerms Successful update of terms at /collections/c8n_1x3_commits/terms/shard1 to Terms{values={core_node6=0, core_node3=0, core_node5=0}, version=2} for registerTerm 2> 2221850 INFO (qtp1257576055-4707) [n:127.0.0.1:42333_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node6 x:c8n_1x3_commits_shard1_replica_n4] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/c8n_1x3_commits/leaders/shard1 2> 2222068 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 2222068 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 2222069 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39977/dql/qo/c8n_1x3_commits_shard1_replica_n2/ 2> 2222081 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=c8n_1x3_commits_shard1_replica_n2 url=http://127.0.0.1:39977/dql/qo START replicas=[http://127.0.0.1:37307/dql/qo/c8n_1x3_commits_shard1_replica_n1/, http://127.0.0.1:42333/dql/qo/c8n_1x3_commits_shard1_replica_n4/] nUpdates=100 2> 2222104 INFO (SocketProxy-Acceptor-37307) [] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=55954,localport=37307], receiveBufferSize: 65536 2> 2222112 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=c8n_1x3_commits_shard1_replica_n2 url=http://127.0.0.1:39977/dql/qo DONE. We have no versions. sync failed. 2> 2222121 INFO (SocketProxy-Acceptor-42333) [] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=53832,localport=42333], receiveBufferSize: 65536 2> 2222130 INFO (SocketProxy-Acceptor-37307) [] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=36939,localport=53852], receiveBufferSize=65536 2> 2222177 INFO (SocketProxy-Acceptor-42333) [] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=39373,localport=40182], receiveBufferSize=65536 2> 2222194 INFO (qtp1257576055-4706) [n:127.0.0.1:42333_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node6 x:c8n_1x3_commits_shard1_replica_n4] o.a.s.c.S.Request webapp=/dql/qo path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=2 2> 2222197 INFO (qtp568058158-4810) [n:127.0.0.1:37307_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node3 x:c8n_1x3_commits_shard1_replica_n1] o.a.s.c.S.Request webapp=/dql/qo path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=2 2> 2222216 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate 2> 2222217 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway 2> 2222217 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/c8n_1x3_commits/leaders/shard1/leader after winning as /collections/c8n_1x3_commits/leader_elect/shard1/election/72080878792015882-core_node5-n_0000000000 2> 2222275 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39977/dql/qo/c8n_1x3_commits_shard1_replica_n2/ shard1 2> 2222455 INFO (zkCallback-3099-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/c8n_1x3_commits/state.json] for collection [c8n_1x3_commits] has occurred - updating... (live nodes size: [5]) 2> 2222484 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node5 x:c8n_1x3_commits_shard1_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 2222572 INFO (qtp725172967-4765) [n:127.0.0.1:39977_dql%2Fqo] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf1&newCollection=true&name=c8n_1x3_commits_shard1_replica_n2&action=CREATE&numShards=1&collection=c8n_1x3_commits&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=4750 2> 2222647 INFO (zkCallback-3099-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/c8n_1x3_commits/state.json] for collection [c8n_1x3_commits] has occurred - updating... (live nodes size: [5]) 2> 2222656 INFO (zkCallback-3099-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/c8n_1x3_commits/state.json] for collection [c8n_1x3_commits] has occurred - updating... (live nodes size: [5]) 2> 2222733 ERROR (OverseerThreadFactory-3065-thread-5) [] o.a.s.c.a.c.CollectionHandlingUtils Error from shard: http://127.0.0.1:37307/dql/qo 2> => org.apache.solr.client.solrj.SolrServerException: Total timeout 5000 ms elapsed 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) 2> org.apache.solr.client.solrj.SolrServerException: Total timeout 5000 ms elapsed 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) ~[solr-solrj-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:192) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:184) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpReceiver.abort(HttpReceiver.java:544) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpChannel.abortResponse(HttpChannel.java:149) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpChannel.abort(HttpChannel.java:142) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpExchange.abort(HttpExchange.java:273) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpConversation.abort(HttpConversation.java:159) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpRequest.abort(HttpRequest.java:865) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpConnection$RequestTimeouts.onExpired(HttpConnection.java:331) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpConnection$RequestTimeouts.onExpired(HttpConnection.java:311) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.CyclicTimeouts.onTimeoutExpired(CyclicTimeouts.java:106) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.CyclicTimeouts$Timeouts.onTimeoutExpired(CyclicTimeouts.java:193) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.CyclicTimeout$Wakeup.run(CyclicTimeout.java:294) ~[jetty-io-10.0.13.jar:10.0.13] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[?:?] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> Caused by: java.util.concurrent.TimeoutException: Total timeout 5000 ms elapsed 2> ... 11 more 2> 2222798 ERROR (OverseerThreadFactory-3065-thread-5) [] o.a.s.c.a.c.CollectionHandlingUtils Error from shard: http://127.0.0.1:42333/dql/qo 2> => org.apache.solr.client.solrj.SolrServerException: Total timeout 5000 ms elapsed 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) 2> org.apache.solr.client.solrj.SolrServerException: Total timeout 5000 ms elapsed 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) ~[solr-solrj-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:192) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:184) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpReceiver.abort(HttpReceiver.java:544) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpChannel.abortResponse(HttpChannel.java:149) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpChannel.abort(HttpChannel.java:142) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpExchange.abort(HttpExchange.java:273) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpConversation.abort(HttpConversation.java:159) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpRequest.abort(HttpRequest.java:865) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpConnection$RequestTimeouts.onExpired(HttpConnection.java:331) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpConnection$RequestTimeouts.onExpired(HttpConnection.java:311) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.CyclicTimeouts.onTimeoutExpired(CyclicTimeouts.java:106) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.CyclicTimeouts$Timeouts.onTimeoutExpired(CyclicTimeouts.java:193) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.CyclicTimeout$Wakeup.run(CyclicTimeout.java:294) ~[jetty-io-10.0.13.jar:10.0.13] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[?:?] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> Caused by: java.util.concurrent.TimeoutException: Total timeout 5000 ms elapsed 2> ... 11 more 2> 2222824 ERROR (OverseerThreadFactory-3065-thread-5) [] o.a.s.c.a.c.CollectionHandlingUtils Cleaning up collection [c8n_1x3_commits]. 2> 2222923 INFO (OverseerThreadFactory-3065-thread-5) [] o.a.s.c.a.c.CollectionHandlingUtils Executing Collection Cmd=action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true, asyncId=null 2> 2222954 INFO (qtp568058158-4807) [n:127.0.0.1:37307_dql%2Fqo x:c8n_1x3_commits_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.c8n_1x3_commits.shard1.replica_n1 tag=null 2> 2222959 INFO (qtp725172967-4763) [n:127.0.0.1:39977_dql%2Fqo x:c8n_1x3_commits_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.c8n_1x3_commits.shard1.replica_n2 tag=null 2> 2222970 INFO (qtp568058158-4811) [n:127.0.0.1:37307_dql%2Fqo] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=c8n_1x3_commits_shard1_replica_n1&action=CREATE&numShards=1&collection=c8n_1x3_commits&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=5155 2> 2223000 INFO (qtp1257576055-4702) [n:127.0.0.1:42333_dql%2Fqo x:c8n_1x3_commits_shard1_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.c8n_1x3_commits.shard1.replica_n4 tag=null 2> 2223062 INFO (zkCallback-3099-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/c8n_1x3_commits/state.json] for collection [c8n_1x3_commits] has occurred - updating... (live nodes size: [5]) 2> 2223062 INFO (zkCallback-3099-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/c8n_1x3_commits/state.json] for collection [c8n_1x3_commits] has occurred - updating... (live nodes size: [5]) 2> 2223141 INFO (qtp1257576055-4707) [n:127.0.0.1:42333_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node6 x:c8n_1x3_commits_shard1_replica_n4] o.a.s.c.ZkShardTerms Successful update of terms at /collections/c8n_1x3_commits/terms/shard1 to Terms{values={core_node3=0, core_node5=0}, version=3} for removeTerm 2> 2223165 INFO (qtp1257576055-4707) [n:127.0.0.1:42333_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node6 x:c8n_1x3_commits_shard1_replica_n4] o.a.s.c.ShardLeaderElectionContextBase No version found for ephemeral leader parent node, won't remove previous leader registration. 2> 2223166 ERROR (qtp1257576055-4707) [n:127.0.0.1:42333_dql%2Fqo c:c8n_1x3_commits s:shard1 r:core_node6 x:c8n_1x3_commits_shard1_replica_n4] o.a.s.c.ZkContainer 2> => org.apache.solr.common.SolrException: SolrCore is no longer available to register 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1384) 2> org.apache.solr.common.SolrException: SolrCore is no longer available to register 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1384) ~[main/:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1279) ~[main/:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.core.ZkContainer.lambda$registerInZk$1(ZkContainer.java:217) [main/:?] 2> at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:246) [main/:?] 2> at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:1453) [main/:?] 2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1676) [main/:?] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1532) [main/:?] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:111) [main/:?] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:398) [main/:?] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:354) [main/:?] 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:219) [main/:?] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:224) [main/:?] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:929) [main/:?] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:877) [main/:?] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:548) [main/:?] 2> at org.apache.solr.servlet.SolrDispatchFilter.dispatch(SolrDispatchFilter.java:252) [main/:?] 2> at org.apache.solr.servlet.SolrDispatchFilter.lambda$doFilter$0(SolrDispatchFilter.java:220) [main/:?] 2> at org.apache.solr.servlet.ServletUtils.traceHttpRequestExecution2(ServletUtils.java:257) [main/:?] 2> at org.apache.solr.servlet.ServletUtils.rateLimitRequest(ServletUtils.java:227) [main/:?] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:215) [main/:?] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197) [main/:?] 2> at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.apache.solr.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:187) [solr-test-framework-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:527) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1570) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1383) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1543) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1305) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:301) [jetty-rewrite-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:790) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.Server.handle(Server.java:563) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.HttpChannel.lambda$handle$0(HttpChannel.java:505) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:762) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:497) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:457) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:934) [jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1078) [jetty-util-10.0.13.jar:10.0.13] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> 2223186 INFO (qtp1257576055-4707) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf1&newCollection=true&name=c8n_1x3_commits_shard1_replica_n4&action=CREATE&numShards=1&collection=c8n_1x3_commits&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=5352 2> 2225716 INFO (qtp568058158-4807) [n:127.0.0.1:37307_dql%2Fqo x:c8n_1x3_commits_shard1_replica_n1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@73f15459 c8n_1x3_commits_shard1_replica_n1 2> 2225717 INFO (qtp568058158-4807) [n:127.0.0.1:37307_dql%2Fqo x:c8n_1x3_commits_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.c8n_1x3_commits.shard1.replica_n1 tag=SolrCore@73f15459 2> 2225724 INFO (qtp568058158-4807) [n:127.0.0.1:37307_dql%2Fqo x:c8n_1x3_commits_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.c8n_1x3_commits.shard1.leader tag=SolrCore@73f15459 2> 2225742 INFO (qtp568058158-4807) [n:127.0.0.1:37307_dql%2Fqo x:c8n_1x3_commits_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 2225877 INFO (qtp1257576055-4702) [n:127.0.0.1:42333_dql%2Fqo x:c8n_1x3_commits_shard1_replica_n4] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@886301 c8n_1x3_commits_shard1_replica_n4 2> 2225878 INFO (qtp1257576055-4702) [n:127.0.0.1:42333_dql%2Fqo x:c8n_1x3_commits_shard1_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.c8n_1x3_commits.shard1.replica_n4 tag=SolrCore@886301 2> 2225878 INFO (qtp1257576055-4702) [n:127.0.0.1:42333_dql%2Fqo x:c8n_1x3_commits_shard1_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.c8n_1x3_commits.shard1.leader tag=SolrCore@886301 2> 2225879 INFO (qtp1257576055-4702) [n:127.0.0.1:42333_dql%2Fqo x:c8n_1x3_commits_shard1_replica_n4] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 2225928 INFO (qtp568058158-4807) [n:127.0.0.1:37307_dql%2Fqo] o.a.s.c.ZkShardTerms Successful update of terms at /collections/c8n_1x3_commits/terms/shard1 to Terms{values={core_node5=0}, version=4} for removeTerm 2> 2225984 INFO (qtp568058158-4807) [n:127.0.0.1:37307_dql%2Fqo] o.a.s.c.ShardLeaderElectionContextBase No version found for ephemeral leader parent node, won't remove previous leader registration. 2> 2226005 INFO (qtp568058158-4807) [n:127.0.0.1:37307_dql%2Fqo] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=c8n_1x3_commits_shard1_replica_n1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=3065 2> 2226008 WARN (zkCallback-3059-thread-1) [] o.a.s.c.LeaderElector Our node is no longer in line to be leader 2> 2226090 INFO (qtp1257576055-4702) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=c8n_1x3_commits_shard1_replica_n4&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=3097 2> 2226108 INFO (qtp725172967-4763) [n:127.0.0.1:39977_dql%2Fqo x:c8n_1x3_commits_shard1_replica_n2] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@1cc6c801 c8n_1x3_commits_shard1_replica_n2 2> 2226108 INFO (qtp725172967-4763) [n:127.0.0.1:39977_dql%2Fqo x:c8n_1x3_commits_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.c8n_1x3_commits.shard1.replica_n2 tag=SolrCore@1cc6c801 2> 2226109 INFO (qtp725172967-4763) [n:127.0.0.1:39977_dql%2Fqo x:c8n_1x3_commits_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.c8n_1x3_commits.shard1.leader tag=SolrCore@1cc6c801 2> 2226110 INFO (qtp725172967-4763) [n:127.0.0.1:39977_dql%2Fqo x:c8n_1x3_commits_shard1_replica_n2] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 2226212 INFO (qtp725172967-4763) [n:127.0.0.1:39977_dql%2Fqo] o.a.s.c.ZkShardTerms Successful update of terms at /collections/c8n_1x3_commits/terms/shard1 to Terms{values={}, version=5} for removeTerm 2> 2226244 WARN (zkCallback-3133-thread-1) [] o.a.s.c.LeaderElector Our node is no longer in line to be leader 2> 2226300 INFO (qtp725172967-4763) [n:127.0.0.1:39977_dql%2Fqo] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=c8n_1x3_commits_shard1_replica_n2&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=3360 2> 2226533 INFO (zkCallback-3059-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/c8n_1x3_commits/state.json] for collection [c8n_1x3_commits] has occurred - updating... (live nodes size: [5]) 2> 2226534 INFO (zkCallback-3059-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/c8n_1x3_commits/state.json] for collection [c8n_1x3_commits] has occurred - updating... (live nodes size: [5]) 2> 2226534 INFO (zkCallback-3059-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/c8n_1x3_commits/state.json] for collection [c8n_1x3_commits] has occurred - updating... (live nodes size: [5]) 2> 2226548 INFO (zkCallback-3059-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/c8n_1x3_commits/state.json] for collection [c8n_1x3_commits] has occurred - updating... (live nodes size: [5]) 2> 2226556 INFO (zkCallback-3059-thread-5) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/c8n_1x3_commits/state.json] for collection [c8n_1x3_commits] has occurred - updating... (live nodes size: [5]) 2> 2226569 INFO (zkCallback-3059-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/c8n_1x3_commits/state.json] for collection [c8n_1x3_commits] has occurred - updating... (live nodes size: [5]) 2> 2226903 INFO (OverseerThreadFactory-3065-thread-5) [] o.a.s.c.a.c.CreateCollectionCmd Cleaned up artifacts for failed create collection for [c8n_1x3_commits] 2> 2226904 ERROR (OverseerThreadFactory-3065-thread-5) [] o.a.s.c.a.c.OverseerCollectionMessageHandler Collection: c8n_1x3_commits operation: create failed 2> => org.apache.solr.common.SolrException: Underlying core creation failed while creating collection: c8n_1x3_commits 2> at org.apache.solr.cloud.api.collections.CreateCollectionCmd.call(CreateCollectionCmd.java:449) 2> org.apache.solr.common.SolrException: Underlying core creation failed while creating collection: c8n_1x3_commits 2> at org.apache.solr.cloud.api.collections.CreateCollectionCmd.call(CreateCollectionCmd.java:449) ~[main/:?] 2> at org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:126) [main/:?] 2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:566) [main/:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289) [solr-solrj-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> 2226956 INFO (qtp1570696396-4786) [n:127.0.0.1:45565_dql%2Fqo] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=3&collection.configName=conf1&name=c8n_1x3_commits&action=CREATE&numShards=1&wt=javabin&version=2} status=400 QTime=10118 2> 2226963 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Ending test 2> 2227082 INFO (OverseerCollectionConfigSetProcessor-72080878792015876-127.0.0.1:42333_dql%2Fqo-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000012 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 2227159 INFO (closeThreadPool-3209-thread-2) [] o.e.j.s.Server Stopped Server@b8bdf0d{STOPPING}[10.0.13,sto=0] 2> 2227173 INFO (closeThreadPool-3209-thread-3) [] o.e.j.s.Server Stopped Server@3d77ebda{STOPPING}[10.0.13,sto=0] 2> 2227173 INFO (closeThreadPool-3209-thread-2) [] o.e.j.s.AbstractConnector Stopped ServerConnector@370d36ea{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0} 2> 2227174 INFO (closeThreadPool-3209-thread-4) [] o.e.j.s.Server Stopped Server@7f9bcce2{STOPPING}[10.0.13,sto=0] 2> 2227177 INFO (closeThreadPool-3209-thread-2) [] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@65e9aeb1{/dql/qo,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,STOPPED} 2> 2227177 INFO (closeThreadPool-3209-thread-5) [] o.e.j.s.Server Stopped Server@422145a3{STOPPING}[10.0.13,sto=0] 2> 2227179 INFO (closeThreadPool-3209-thread-6) [] o.e.j.s.Server Stopped Server@738daffd{STOPPING}[10.0.13,sto=0] 2> 2227188 INFO (closeThreadPool-3209-thread-4) [] o.e.j.s.AbstractConnector Stopped ServerConnector@59a173a7{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0} 2> 2227190 INFO (closeThreadPool-3209-thread-4) [] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6ef7c7e2{/dql/qo,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,STOPPED} 2> 2227196 INFO (closeThreadPool-3209-thread-3) [] o.e.j.s.AbstractConnector Stopped ServerConnector@2794b6f5{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0} 2> 2227192 INFO (closeThreadPool-3209-thread-5) [] o.e.j.s.AbstractConnector Stopped ServerConnector@2f06c358{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0} 2> 2227203 INFO (closeThreadPool-3209-thread-5) [] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@60c0e855{/dql/qo,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,STOPPED} 2> 2227203 INFO (closeThreadPool-3209-thread-3) [] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@78e95412{/dql/qo,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,STOPPED} 2> 2227206 INFO (closeThreadPool-3209-thread-6) [] o.e.j.s.AbstractConnector Stopped ServerConnector@35f71550{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0} 2> 2227208 INFO (closeThreadPool-3209-thread-6) [] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@13ff6207{/dql/qo,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,STOPPED} 2> 2227220 INFO (closeThreadPool-3209-thread-6) [] o.a.s.c.CoreContainer Shutting down CoreContainer instance=452947260 2> 2227275 INFO (closeThreadPool-3209-thread-6) [] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:42517_dql%2Fqo 2> 2227281 INFO (closeThreadPool-3209-thread-5) [] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1133181149 2> 2227251 INFO (closeThreadPool-3209-thread-4) [] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1299558643 2> 2227282 INFO (closeThreadPool-3209-thread-5) [] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:39977_dql%2Fqo 2> 2227282 INFO (closeThreadPool-3209-thread-4) [] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:45565_dql%2Fqo 2> 2227288 INFO (closeThreadPool-3209-thread-6) [] o.a.s.c.ZkController Publish this node as DOWN... 2> 2227288 INFO (closeThreadPool-3209-thread-6) [] o.a.s.c.ZkController Publish node=127.0.0.1:42517_dql%2Fqo as DOWN 2> 2227317 INFO (closeThreadPool-3209-thread-3) [] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2057796091 2> 2227318 INFO (closeThreadPool-3209-thread-3) [] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:37307_dql%2Fqo 2> 2227337 INFO (closeThreadPool-3209-thread-4) [] o.a.s.c.ZkController Publish this node as DOWN... 2> 2227337 INFO (closeThreadPool-3209-thread-4) [] o.a.s.c.ZkController Publish node=127.0.0.1:45565_dql%2Fqo as DOWN 2> 2227339 INFO (zkCallback-3099-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (2) 2> 2227340 INFO (closeThreadPool-3209-thread-5) [] o.a.s.c.ZkController Publish this node as DOWN... 2> 2227337 INFO (zkCallback-3133-thread-4) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (2) 2> 2227340 INFO (closeThreadPool-3209-thread-5) [] o.a.s.c.ZkController Publish node=127.0.0.1:39977_dql%2Fqo as DOWN 2> 2227341 INFO (zkCallback-3059-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (2) 2> 2227344 INFO (zkCallback-3120-thread-3) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (2) 2> 2227366 INFO (closeThreadPool-3209-thread-3) [] o.a.s.c.ZkController Publish this node as DOWN... 2> 2227366 INFO (closeThreadPool-3209-thread-3) [] o.a.s.c.ZkController Publish node=127.0.0.1:37307_dql%2Fqo as DOWN 2> 2227368 INFO (zkCallback-3151-thread-3) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (1) 2> 2227381 INFO (zkCallback-3120-thread-3) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1) 2> 2227385 INFO (zkCallback-3059-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1) 2> 2227397 INFO (zkCallback-3099-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1) 2> 2227398 INFO (zkCallback-3133-thread-4) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1) 2> 2227501 INFO (coreCloseExecutor-3218-thread-1) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@a3a995e collection1_shard1_replica_n7 2> 2227501 INFO (coreCloseExecutor-3218-thread-1) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard1.replica_n7 tag=SolrCore@a3a995e 2> 2227503 INFO (coreCloseExecutor-3220-thread-1) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@4b8b82c8 collection1_shard1_replica_n3 2> 2227505 INFO (coreCloseExecutor-3220-thread-1) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard1.replica_n3 tag=SolrCore@4b8b82c8 2> 2227513 INFO (coreCloseExecutor-3219-thread-1) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@36e11c15 collection1_shard1_replica_n5 2> 2227513 INFO (coreCloseExecutor-3219-thread-1) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard1.replica_n5 tag=SolrCore@36e11c15 2> 2227534 INFO (coreCloseExecutor-3221-thread-1) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@33dc57cb collection1_shard1_replica_n1 2> 2227543 INFO (coreCloseExecutor-3221-thread-1) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard1.replica_n1 tag=SolrCore@33dc57cb 2> 2227681 INFO (zkCallback-3120-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) 2> 2227681 INFO (zkCallback-3133-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) 2> 2227681 INFO (zkCallback-3120-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) 2> 2227681 INFO (zkCallback-3120-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) 2> 2227682 INFO (zkCallback-3133-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) 2> 2227682 INFO (zkCallback-3133-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) 2> 2227683 INFO (zkCallback-3099-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) 2> 2227683 INFO (zkCallback-3099-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) 2> 2227684 INFO (zkCallback-3151-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) 2> 2227684 INFO (zkCallback-3151-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) 2> 2227684 INFO (zkCallback-3099-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) 2> 2227689 INFO (zkCallback-3151-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1]) 2> 2227976 INFO (closeThreadPool-3209-thread-2) [] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1268906099 2> 2227977 INFO (closeThreadPool-3209-thread-2) [] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:42333_dql%2Fqo 2> 2227990 INFO (closeThreadPool-3209-thread-2) [] o.a.s.c.ZkController Publish this node as DOWN... 2> 2227990 INFO (closeThreadPool-3209-thread-2) [] o.a.s.c.ZkController Publish node=127.0.0.1:42333_dql%2Fqo as DOWN 2> 2227991 INFO (zkCallback-3133-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) 2> 2227992 INFO (zkCallback-3120-thread-2) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) 2> 2227996 INFO (zkCallback-3099-thread-4) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) 2> 2228000 INFO (zkCallback-3151-thread-3) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) 2> 2228004 INFO (zkCallback-3059-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) 2> 2228059 INFO (coreCloseExecutor-3224-thread-1) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@3469fdda control_collection_shard1_replica_n1 2> 2228061 INFO (coreCloseExecutor-3224-thread-1) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.control_collection.shard1.replica_n1 tag=SolrCore@3469fdda 2> 2228191 INFO (zkCallback-3059-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [0]) 2> 2228191 INFO (zkCallback-3059-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [0]) 2> 2228192 INFO (zkCallback-3059-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [0]) 2> 2231641 INFO (coreCloseExecutor-3219-thread-1) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader tag=SolrCore@36e11c15 2> 2231701 INFO (coreCloseExecutor-3219-thread-1) [n:127.0.0.1:39977_dql%2Fqo c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_n5] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 2231702 INFO (coreCloseExecutor-3218-thread-1) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader tag=SolrCore@a3a995e 2> 2231743 INFO (coreCloseExecutor-3218-thread-1) [n:127.0.0.1:42517_dql%2Fqo c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 2231746 INFO (coreCloseExecutor-3221-thread-1) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader tag=SolrCore@33dc57cb 2> 2231788 INFO (coreCloseExecutor-3221-thread-1) [n:127.0.0.1:45565_dql%2Fqo c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 2231800 INFO (closeThreadPool-3209-thread-5) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null 2> 2231845 INFO (closeThreadPool-3209-thread-6) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null 2> 2231854 INFO (coreCloseExecutor-3220-thread-1) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader tag=SolrCore@4b8b82c8 2> 2231880 INFO (coreCloseExecutor-3220-thread-1) [n:127.0.0.1:37307_dql%2Fqo c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 2231894 INFO (coreCloseExecutor-3224-thread-1) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.control_collection.shard1.leader tag=SolrCore@3469fdda 2> 2231904 INFO (closeThreadPool-3209-thread-4) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null 2> 2231940 INFO (coreCloseExecutor-3224-thread-1) [n:127.0.0.1:42333_dql%2Fqo c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 2231962 INFO (closeThreadPool-3209-thread-3) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null 2> 2231981 INFO (closeThreadPool-3209-thread-2) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null 2> 2232809 INFO (closeThreadPool-3209-thread-2) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null 2> 2232828 INFO (closeThreadPool-3209-thread-5) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null 2> 2232927 INFO (closeThreadPool-3209-thread-3) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null 2> 2232951 INFO (closeThreadPool-3209-thread-4) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null 2> 2233345 INFO (closeThreadPool-3209-thread-6) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null 2> 2233567 INFO (closeThreadPool-3209-thread-5) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null 2> 2233568 INFO (closeThreadPool-3209-thread-5) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null 2> 2233641 INFO (closeThreadPool-3209-thread-2) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null 2> 2233643 INFO (closeThreadPool-3209-thread-2) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null 2> 2233662 INFO (closeThreadPool-3209-thread-3) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null 2> 2233662 INFO (closeThreadPool-3209-thread-3) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null 2> 2233672 INFO (closeThreadPool-3209-thread-4) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null 2> 2233679 INFO (closeThreadPool-3209-thread-4) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null 2> 2233685 INFO (closeThreadPool-3226-thread-2) [] o.a.s.c.Overseer Overseer (id=72080878792015876-127.0.0.1:42333_dql%2Fqo-n_0000000000) closing 2> 2233686 INFO (OverseerStateUpdate-72080878792015876-127.0.0.1:42333_dql%2Fqo-n_0000000000) [n:127.0.0.1:42333_dql%2Fqo] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42333_dql%2Fqo 2> 2233864 WARN (closeThreadPool-3209-thread-5) [] o.a.s.c.s.c.SocketProxy Closing 3 connections to: http://127.0.0.1:39977/dql/qo, target: http://127.0.0.1:43375/dql/qo 2> 2233866 INFO (closeThreadPool-3226-thread-1) [] o.a.s.c.Overseer Overseer (id=72080878792015876-127.0.0.1:42333_dql%2Fqo-n_0000000000) closing 2> 2233868 INFO (closeThreadPool-3209-thread-2) [] o.a.s.c.Overseer Overseer (id=72080878792015876-127.0.0.1:42333_dql%2Fqo-n_0000000000) closing 2> 2233957 INFO (closeThreadPool-3209-thread-6) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null 2> 2233958 INFO (closeThreadPool-3209-thread-6) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null 2> 2233979 WARN (closeThreadPool-3209-thread-2) [] o.a.s.c.s.c.SocketProxy Closing 5 connections to: http://127.0.0.1:42333/dql/qo, target: http://127.0.0.1:39373/dql/qo 2> 2233981 WARN (closeThreadPool-3209-thread-4) [] o.a.s.c.s.c.SocketProxy Closing 5 connections to: http://127.0.0.1:45565/dql/qo, target: http://127.0.0.1:44821/dql/qo 2> 2234032 WARN (closeThreadPool-3209-thread-3) [] o.a.s.c.s.c.SocketProxy Closing 8 connections to: http://127.0.0.1:37307/dql/qo, target: http://127.0.0.1:36939/dql/qo 2> 2234136 WARN (closeThreadPool-3209-thread-6) [] o.a.s.c.s.c.SocketProxy Closing 2 connections to: http://127.0.0.1:42517/dql/qo, target: http://127.0.0.1:45985/dql/qo 2> 2234140 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.u.ErrorLogMuter Closing ErrorLogMuter-regex-102 after mutting 0 log messages 2> 2234140 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.u.ErrorLogMuter Creating ErrorLogMuter-regex-103 for ERROR logs matching regex: ignore_exception 2> 2234143 INFO (TEST-HttpPartitionOnCommitTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkTestServer Shutting down ZkTestServer. 2> 2234380 WARN (ZkTestServer Run Thread) [] o.a.s.c.ZkTestServer Watch limit violations: 2> Maximum concurrent create/delete watches above limit: 2> 2> 15 /solr/collections/c8n_1x3_commits/terms/shard1 2> 14 /solr/collections/collection1/terms/shard1 2> 7 /solr/clusterprops.json 2> 7 /solr/aliases.json 2> 6 /solr/collections/c8n_1x3_commits/state.json 2> 5 /solr/packages.json 2> 5 /solr/security.json 2> 5 /solr/configs/conf1 2> 4 /solr/collections/collection1/collectionprops.json 2> 2 /solr/collections/control_collection/terms/shard1 2> 2> Maximum concurrent data watches above limit: 2> 2> 146 /solr/collections/collection1/state.json 2> 22 /solr/collections/c8n_1x3_commits/state.json 2> 16 /solr/collections/control_collection/state.json 2> 2 /solr/collections/collection1/leader_elect/shard1/election/72080878792015887-core_node4-n_0000000001 2> 2 /solr/overseer_elect/election/72080878792015876-127.0.0.1:42333_dql%2Fqo-n_0000000000 2> 2> Maximum concurrent children watches above limit: 2> 2> 79 /solr/overseer/queue 2> 40 /solr/live_nodes 2> 32 /solr/collections/collection1/state.json 2> 19 /solr/collections 2> 19 /solr/overseer/collection-queue-work 2> 16 /solr/collections/c8n_1x3_commits/state.json 2> 7 /solr/collections/control_collection/state.json 2> > org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:45565/dql/qo: Underlying core creation failed while creating collection: c8n_1x3_commits > at __randomizedtesting.SeedInfo.seed([9CAA27B60427D55F:14FE186CAADBB8A7]:0) > at app//org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:745) > at app//org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259) > at app//org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:240) > at app//org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1193) > at app//org.apache.solr.cloud.AbstractBasicDistributedZkTestBase.createCollection(AbstractBasicDistributedZkTestBase.java:1172) > at app//org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:2013) > at app//org.apache.solr.cloud.HttpPartitionOnCommitTest.oneShardTest(HttpPartitionOnCommitTest.java:131) > at app//org.apache.solr.cloud.HttpPartitionOnCommitTest.test(HttpPartitionOnCommitTest.java:66) > at java.base@11.0.20/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at java.base@11.0.20/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at java.base@11.0.20/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base@11.0.20/java.lang.reflect.Method.invoke(Method.java:566) > at app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) > at app//com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) > at app//com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) > at app//com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) > at app//org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1165) > at app//org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1136) > at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) > at app//org.junit.rules.RunRules.evaluate(RunRules.java:20) > at app//org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:44) > at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at app//org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) > at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) > at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) > at app//org.junit.rules.RunRules.evaluate(RunRules.java:20) > at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) > at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) > at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) > at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) > at app//com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) > at app//com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) > at app//com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) > at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) > at app//org.junit.rules.RunRules.evaluate(RunRules.java:20) > at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at app//org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) > at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) > at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) > at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at app//org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) > at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) > at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) > at app//org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) > at app//org.junit.rules.RunRules.evaluate(RunRules.java:20) > at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) > at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) > at java.base@11.0.20/java.lang.Thread.run(Thread.java:829) 2> NOTE: reproduce with: gradlew test --tests HttpPartitionOnCommitTest.test -Dtests.seed=9CAA27B60427D55F -Dtests.badapples=true -Dtests.locale=en-VC -Dtests.timezone=America/Shiprock -Dtests.asserts=true -Dtests.file.encoding=UTF-8 2> 2234456 INFO (SUITE-HttpPartitionOnCommitTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.u.ErrorLogMuter Closing ErrorLogMuter-regex-103 after mutting 0 log messages 2> 2234457 INFO (SUITE-HttpPartitionOnCommitTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.u.ErrorLogMuter Creating ErrorLogMuter-regex-104 for ERROR logs matching regex: ignore_exception 2> NOTE: leaving temporary files on disk at: /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.HttpPartitionOnCommitTest_9CAA27B60427D55F-001 2> NOTE: test params are: codec=Asserting(Lucene94): {}, docValues:{}, maxPointsInLeafNode=93, maxMBSortInHeap=6.690403445933855, sim=Asserting(RandomSimilarity(queryNorm=false): {}), locale=en-VC, timezone=America/Shiprock 2> NOTE: Linux 5.10.113-g387b6863253c-dirty riscv64/Oracle Corporation 11.0.20 (64-bit)/cpus=1,threads=2,free=81686944,total=393740288 2> NOTE: All tests run in this JVM: [BasicFunctionalityTest, CursorPagingTest, DistributedIntervalFacetingTest, HelloWorldSolrCloudTestCase, OutputWriterTest, SolrInfoBeanTest, SolrTestCaseJ4Test, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, TestDocumentBuilder, TestHighlightDedupGrouping, TestRandomDVFaceting, TestSimpleTrackingShardHandler, TestTolerantSearch, CommonGramsPhraseQueryTest, ProtectedTermFilterFactoryTest, TestDeprecatedFilters, TestReversedWildcardFilterFactory, TokenizerChainTest, TestEmbeddedSolrServerAdminHandler, TestEmbeddedSolrServerSchemaAPI, ActionThrottleTest, AliasIntegrationTest, BasicDistributedZk2Test, ChaosMonkeyNothingIsSafeTest, ChaosMonkeySafeLeaderTest, CloudExitableDirectoryReaderTest, ClusterStateTest, CollectionPropsTest, CollectionsAPISolrJTest, ConfigSetApiLockingTest, CreateCollectionCleanupTest, DeleteInactiveReplicaTest, DeleteNodeTest, DeleteShardTest, DistribCursorPagingTest, DistribJoinFromCollectionTest, DistributedQueueTest, DocValuesNotIndexedTest, ForceLeaderWithTlogReplicasTest, HttpPartitionOnCommitTest] org.apache.solr.cloud.ReplaceNodeTest > testGoodSpreadDuringAssignWithNoTarget FAILED java.lang.AssertionError: There should be no more replicas on the sourceNode after a replaceNode request. expected:<[]> but was:<[core_node11:{ "core":"replacenodetest_coll_shard1_replica_n5", "leader":"true", "node_name":"127.0.0.1:46857_solr", "base_url":"http://127.0.0.1:46857/solr", "state":"active", "type":"NRT", "force_set_state":"false"}, core_node12:{ "core":"replacenodetest_coll_shard2_replica_n6", "leader":"true", "node_name":"127.0.0.1:46857_solr", "base_url":"http://127.0.0.1:46857/solr", "state":"active", "type":"NRT", "force_set_state":"false"}, core_node8:{ "core":"replacenodetest_coll_shard3_replica_n2", "leader":"true", "node_name":"127.0.0.1:46857_solr", "base_url":"http://127.0.0.1:46857/solr", "state":"active", "type":"NRT", "force_set_state":"false"}]> at __randomizedtesting.SeedInfo.seed([9CAA27B60427D55F:B2C1F38C3144042B]:0) at org.junit.Assert.fail(Assert.java:89) at org.junit.Assert.failNotEquals(Assert.java:835) at org.junit.Assert.assertEquals(Assert.java:120) at org.apache.solr.cloud.ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget(ReplaceNodeTest.java:274) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:44) at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) at java.base/java.lang.Thread.run(Thread.java:829) org.apache.solr.cloud.ReplaceNodeTest > test suite's output saved to /build/solr/src/solr-9.2.1/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.ReplaceNodeTest.txt, copied below: 2> 3494314 INFO (SUITE-ReplaceNodeTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/build/solr/src/solr-9.2.1/solr/server/solr/configsets/_default/conf' 2> 3494316 INFO (SUITE-ReplaceNodeTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom 2> 3494329 INFO (SUITE-ReplaceNodeTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.u.ErrorLogMuter Closing ErrorLogMuter-regex-144 after mutting 0 log messages 2> 3494330 INFO (SUITE-ReplaceNodeTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.u.ErrorLogMuter Creating ErrorLogMuter-regex-145 for ERROR logs matching regex: ignore_exception 2> 3494337 INFO (SUITE-ReplaceNodeTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.SolrTestCaseJ4 Created dataDir: /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/data-dir-52-001 2> 3494346 WARN (SUITE-ReplaceNodeTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=154 numCloses=154 2> 3494348 INFO (SUITE-ReplaceNodeTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false 2> 3494370 INFO (SUITE-ReplaceNodeTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, clientAuth=0.0/0.0) 2> 3494612 INFO (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Starting testFailOnSingleNode 2> 3494624 INFO (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-001 2> 3494627 INFO (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER 2> 3494639 INFO (ZkTestServer Run Thread) [] o.a.s.c.ZkTestServer client port: 0.0.0.0/0.0.0.0:0 2> 3494640 INFO (ZkTestServer Run Thread) [] o.a.s.c.ZkTestServer Starting server 2> 3494674 WARN (ZkTestServer Run Thread) [] o.a.z.s.ServerCnxnFactory maxCnxns is not configured, using default value 0. 2> 3494740 INFO (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkTestServer start zk server on port: 42703 2> 3495073 WARN (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3495120 INFO (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3495180 INFO (zkConnectionManagerCallback-6543-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3495185 INFO (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3495187 WARN (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3495237 WARN (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3495282 INFO (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3495311 INFO (zkConnectionManagerCallback-6545-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3495313 INFO (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3495315 WARN (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3495324 WARN (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3495388 INFO (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3495416 INFO (zkConnectionManagerCallback-6547-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3495423 INFO (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3495424 WARN (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3495678 INFO (jetty-launcher-6548-thread-1) [] o.a.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0) 2> 3495678 INFO (jetty-launcher-6548-thread-1) [] o.a.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ... 2> 3495679 INFO (jetty-launcher-6548-thread-1) [] o.e.j.s.Server jetty-10.0.13; built: 2022-12-07T20:13:20.134Z; git: 1c2636ea05c0ca8de1ffd6ca7f3a98ac084c766d; jvm 11.0.20+8 2> 3495808 INFO (jetty-launcher-6548-thread-1) [] o.e.j.s.s.DefaultSessionIdManager Session workerName=node0 2> 3495824 INFO (jetty-launcher-6548-thread-1) [] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3befe745{/solr,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,AVAILABLE} 2> 3495843 INFO (jetty-launcher-6548-thread-1) [] o.e.j.s.AbstractConnector Started ServerConnector@1f1a2621{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:42945} 2> 3495844 INFO (jetty-launcher-6548-thread-1) [] o.e.j.s.Server Started Server@6fb3b20b{STARTING}[10.0.13,sto=0] @3496191ms 2> 3495845 ERROR (jetty-launcher-6548-thread-1) [] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. 2> 3495846 INFO (jetty-launcher-6548-thread-1) [] o.a.s.s.CoreContainerProvider Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory 2> 3495846 INFO (jetty-launcher-6548-thread-1) [] o.a.s.s.CoreContainerProvider ___ _ Welcome to Apache Solr™ version 9.2.1 2> 3495846 INFO (jetty-launcher-6548-thread-1) [] o.a.s.s.CoreContainerProvider / __| ___| |_ _ Starting in cloud mode on port null 2> 3495847 INFO (jetty-launcher-6548-thread-1) [] o.a.s.s.CoreContainerProvider \__ \/ _ \ | '_| Install dir: /build/solr/src/solr-9.2.1/solr 2> 3495847 INFO (jetty-launcher-6548-thread-1) [] o.a.s.s.CoreContainerProvider |___/\___/_|_| Start time: 2023-10-03T08:46:53.851384Z 2> 3495852 INFO (jetty-launcher-6548-thread-1) [] o.a.s.s.CoreContainerProvider Solr Home: /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-001/node1 (source: servlet config: solr.solr.home) 2> 3495853 WARN (jetty-launcher-6548-thread-1) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3495908 INFO (jetty-launcher-6548-thread-1) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3495962 INFO (zkConnectionManagerCallback-6550-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3495965 INFO (jetty-launcher-6548-thread-1) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3495965 WARN (jetty-launcher-6548-thread-1) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3495983 INFO (jetty-launcher-6548-thread-1) [] o.a.s.c.NodeConfig solr.xml found in ZooKeeper. Loading... 2> 3496443 INFO (jetty-launcher-6548-thread-1) [] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42703/solr 2> 3496443 INFO (jetty-launcher-6548-thread-1) [] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=true. Solr will be using distributed cluster state updates. 2> 3496488 INFO (jetty-launcher-6548-thread-1) [n:127.0.0.1:42945_solr] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3496533 INFO (zkConnectionManagerCallback-6560-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3496535 INFO (jetty-launcher-6548-thread-1) [n:127.0.0.1:42945_solr] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3497136 WARN (jetty-launcher-6548-thread-1) [n:127.0.0.1:42945_solr] o.a.s.c.ZkController Contents of zookeeper /security.json are world-readable; consider setting up ACLs as described in https://solr.apache.org/guide/solr/latest/deployment-guide/zookeeper-access-control.html 2> 3497281 INFO (jetty-launcher-6548-thread-1) [n:127.0.0.1:42945_solr] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=true. Solr will be using distributed cluster state updates. 2> 3497469 INFO (jetty-launcher-6548-thread-1) [n:127.0.0.1:42945_solr] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42945_solr 2> 3497512 INFO (jetty-launcher-6548-thread-1) [n:127.0.0.1:42945_solr] o.a.s.c.Overseer Overseer (id=72080967302512644-127.0.0.1:42945_solr-n_0000000000) starting 2> 3497827 INFO (jetty-launcher-6548-thread-1) [n:127.0.0.1:42945_solr] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42945_solr 2> 3497827 INFO (OverseerStateUpdate-72080967302512644-127.0.0.1:42945_solr-n_0000000000) [n:127.0.0.1:42945_solr] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:42945_solr 2> 3497946 INFO (OverseerStateUpdate-72080967302512644-127.0.0.1:42945_solr-n_0000000000) [n:127.0.0.1:42945_solr] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) 2> 3497994 WARN (jetty-launcher-6548-thread-1) [n:127.0.0.1:42945_solr] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info 2> 3501489 INFO (jetty-launcher-6548-thread-1) [n:127.0.0.1:42945_solr] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-001/node1 2> 3502742 INFO (jetty-launcher-6548-thread-1) [n:127.0.0.1:42945_solr] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 3503083 INFO (jetty-launcher-6548-thread-1) [n:127.0.0.1:42945_solr] o.a.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42945, zkHost=127.0.0.1:42703/solr} 2> 3503119 INFO (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForAllNodes: numServers=1 2> 3503121 INFO (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForNode: 127.0.0.1:42945_solr 2> 3503124 WARN (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3503176 INFO (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 15000ms for client to connect to ZooKeeper 2> 3503204 INFO (zkConnectionManagerCallback-6572-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3503216 INFO (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3503216 WARN (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3503252 INFO (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) 2> 3503312 INFO (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42703/solr ready 2> 3503313 WARN (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3503355 INFO (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 45000ms for client to connect to ZooKeeper 2> 3503392 INFO (zkConnectionManagerCallback-6574-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3503400 INFO (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3503400 WARN (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3503589 INFO (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.ReplaceNodeTest total_jettys: 1 2> 3503720 INFO (OverseerThreadFactory-6565-thread-1) [] o.a.s.c.a.c.CreateCollectionCmd Create collection replacesinglenodetest_coll 2> 3504231 INFO (OverseerThreadFactory-6565-thread-1) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replacesinglenodetest_coll_shard4_replica_n1", 2> "node_name":"127.0.0.1:42945_solr", 2> "base_url":"http://127.0.0.1:42945/solr", 2> "collection":"replacesinglenodetest_coll", 2> "shard":"shard4", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 3504281 INFO (OverseerThreadFactory-6565-thread-1) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replacesinglenodetest_coll_shard1_replica_n2", 2> "node_name":"127.0.0.1:42945_solr", 2> "base_url":"http://127.0.0.1:42945/solr", 2> "collection":"replacesinglenodetest_coll", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 3504394 INFO (OverseerThreadFactory-6565-thread-1) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replacesinglenodetest_coll_shard2_replica_n3", 2> "node_name":"127.0.0.1:42945_solr", 2> "base_url":"http://127.0.0.1:42945/solr", 2> "collection":"replacesinglenodetest_coll", 2> "shard":"shard2", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 3504427 INFO (OverseerThreadFactory-6565-thread-1) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replacesinglenodetest_coll_shard3_replica_n4", 2> "node_name":"127.0.0.1:42945_solr", 2> "base_url":"http://127.0.0.1:42945/solr", 2> "collection":"replacesinglenodetest_coll", 2> "shard":"shard3", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 3504458 INFO (OverseerThreadFactory-6565-thread-1) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replacesinglenodetest_coll_shard5_replica_n5", 2> "node_name":"127.0.0.1:42945_solr", 2> "base_url":"http://127.0.0.1:42945/solr", 2> "collection":"replacesinglenodetest_coll", 2> "shard":"shard5", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 3504602 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf1&newCollection=true&name=replacesinglenodetest_coll_shard4_replica_n1&action=CREATE&numShards=5&collection=replacesinglenodetest_coll&shard=shard4&wt=javabin&version=2&replicaType=NRT 2> 3504628 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node9&collection.configName=conf1&newCollection=true&name=replacesinglenodetest_coll_shard3_replica_n4&action=CREATE&numShards=5&collection=replacesinglenodetest_coll&shard=shard3&wt=javabin&version=2&replicaType=NRT 2> 3504631 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=replacesinglenodetest_coll_shard1_replica_n2&action=CREATE&numShards=5&collection=replacesinglenodetest_coll&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 3504632 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf1&newCollection=true&name=replacesinglenodetest_coll_shard2_replica_n3&action=CREATE&numShards=5&collection=replacesinglenodetest_coll&shard=shard2&wt=javabin&version=2&replicaType=NRT 2> 3504635 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node10&collection.configName=conf1&newCollection=true&name=replacesinglenodetest_coll_shard5_replica_n5&action=CREATE&numShards=5&collection=replacesinglenodetest_coll&shard=shard5&wt=javabin&version=2&replicaType=NRT 2> 3504750 INFO (zkCallback-6559-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacesinglenodetest_coll/state.json] for collection [replacesinglenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3504860 INFO (zkCallback-6559-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacesinglenodetest_coll/state.json] for collection [replacesinglenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3505305 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 3505309 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 3505372 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 3505406 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 3505423 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 3505503 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.s.IndexSchema Schema name=cloud-dynamic 2> 3505503 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.s.IndexSchema Schema name=cloud-dynamic 2> 3505503 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.s.IndexSchema Schema name=cloud-dynamic 2> 3505505 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.s.IndexSchema Schema name=cloud-dynamic 2> 3505532 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id 2> 3505533 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.c.CoreContainer Creating SolrCore 'replacesinglenodetest_coll_shard5_replica_n5' using configuration from configset conf1, trusted=true 2> 3505574 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-001/node1/replacesinglenodetest_coll_shard5_replica_n5], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-001/node1/replacesinglenodetest_coll_shard5_replica_n5/data/] 2> 3505578 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.s.IndexSchema Schema name=cloud-dynamic 2> 3505732 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id 2> 3505733 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'replacesinglenodetest_coll_shard1_replica_n2' using configuration from configset conf1, trusted=true 2> 3505741 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id 2> 3505742 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'replacesinglenodetest_coll_shard3_replica_n4' using configuration from configset conf1, trusted=true 2> 3505722 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id 2> 3505747 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'replacesinglenodetest_coll_shard2_replica_n3' using configuration from configset conf1, trusted=true 2> 3505755 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-001/node1/replacesinglenodetest_coll_shard2_replica_n3], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-001/node1/replacesinglenodetest_coll_shard2_replica_n3/data/] 2> 3505759 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-001/node1/replacesinglenodetest_coll_shard1_replica_n2], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-001/node1/replacesinglenodetest_coll_shard1_replica_n2/data/] 2> 3505775 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-001/node1/replacesinglenodetest_coll_shard3_replica_n4], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-001/node1/replacesinglenodetest_coll_shard3_replica_n4/data/] 2> 3505807 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id 2> 3505821 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'replacesinglenodetest_coll_shard4_replica_n1' using configuration from configset conf1, trusted=true 2> 3505866 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-001/node1/replacesinglenodetest_coll_shard4_replica_n1], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-001/node1/replacesinglenodetest_coll_shard4_replica_n1/data/] 2> 3510313 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 3510405 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 3510405 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 3510437 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 3510437 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 3510452 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 3510452 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 3510458 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 3510458 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 3510482 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 3510482 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 3510501 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 3510502 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 3510525 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 3510525 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 3510529 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 3510542 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 3510594 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 3510595 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723406321025024 2> 3510603 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 3510604 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 3510613 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 3510614 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 3510684 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 3510685 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 3510823 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 3510837 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 3510857 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replacesinglenodetest_coll/terms/shard5 to Terms{values={core_node10=0}, version=0} for registerTerm 2> 3510858 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replacesinglenodetest_coll/leaders/shard5 2> 3510883 INFO (searcherExecutor-6576-thread-1-processing-127.0.0.1:42945_solr replacesinglenodetest_coll_shard5_replica_n5 replacesinglenodetest_coll shard5 core_node10) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 3510883 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 3510888 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723406628257792 2> 3510934 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 3510952 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 3510956 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 3510972 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 3510974 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 3510984 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 3510987 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 3511018 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723406764572672 2> 3511021 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 3511024 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723406770864128 2> 3511042 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 3511043 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723406790787072 2> 3511151 INFO (searcherExecutor-6578-thread-1-processing-127.0.0.1:42945_solr replacesinglenodetest_coll_shard1_replica_n2 replacesinglenodetest_coll shard1 core_node7) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 3511180 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 3511183 INFO (searcherExecutor-6584-thread-1-processing-127.0.0.1:42945_solr replacesinglenodetest_coll_shard4_replica_n1 replacesinglenodetest_coll shard4 core_node6) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 3511229 INFO (searcherExecutor-6580-thread-1-processing-127.0.0.1:42945_solr replacesinglenodetest_coll_shard3_replica_n4 replacesinglenodetest_coll shard3 core_node9) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 3511289 INFO (searcherExecutor-6582-thread-1-processing-127.0.0.1:42945_solr replacesinglenodetest_coll_shard2_replica_n3 replacesinglenodetest_coll shard2 core_node8) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 3511373 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replacesinglenodetest_coll/terms/shard1 to Terms{values={core_node7=0}, version=0} for registerTerm 2> 3511384 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replacesinglenodetest_coll/leaders/shard1 2> 3511540 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replacesinglenodetest_coll/terms/shard4 to Terms{values={core_node6=0}, version=0} for registerTerm 2> 3511540 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replacesinglenodetest_coll/leaders/shard4 2> 3511541 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 3511541 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42945/solr/replacesinglenodetest_coll_shard5_replica_n5/ 2> 3511570 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 3511626 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 3511628 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.c.SyncStrategy http://127.0.0.1:42945/solr/replacesinglenodetest_coll_shard5_replica_n5/ has no replicas 2> 3511628 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/replacesinglenodetest_coll/leaders/shard5/leader after winning as /collections/replacesinglenodetest_coll/leader_elect/shard5/election/72080967302512644-core_node10-n_0000000000 2> 3511660 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replacesinglenodetest_coll/terms/shard2 to Terms{values={core_node8=0}, version=0} for registerTerm 2> 3511660 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replacesinglenodetest_coll/leaders/shard2 2> 3511796 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42945/solr/replacesinglenodetest_coll_shard5_replica_n5/ shard5 2> 3511905 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replacesinglenodetest_coll/terms/shard3 to Terms{values={core_node9=0}, version=0} for registerTerm 2> 3511905 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 3511906 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replacesinglenodetest_coll/leaders/shard3 2> 3511906 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 3511906 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 3511906 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42945/solr/replacesinglenodetest_coll_shard4_replica_n1/ 2> 3511906 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42945/solr/replacesinglenodetest_coll_shard1_replica_n2/ 2> 3511940 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 3511978 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 3511992 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 3511994 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 3511994 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42945/solr/replacesinglenodetest_coll_shard2_replica_n3/ 2> 3512096 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 3512117 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:42945/solr/replacesinglenodetest_coll_shard4_replica_n1/ has no replicas 2> 3512117 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/replacesinglenodetest_coll/leaders/shard4/leader after winning as /collections/replacesinglenodetest_coll/leader_elect/shard4/election/72080967302512644-core_node6-n_0000000000 2> 3512153 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 3512158 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.c.SyncStrategy http://127.0.0.1:42945/solr/replacesinglenodetest_coll_shard2_replica_n3/ has no replicas 2> 3512159 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/replacesinglenodetest_coll/leaders/shard2/leader after winning as /collections/replacesinglenodetest_coll/leader_elect/shard2/election/72080967302512644-core_node8-n_0000000000 2> 3512173 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:42945/solr/replacesinglenodetest_coll_shard1_replica_n2/ has no replicas 2> 3512173 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/replacesinglenodetest_coll/leaders/shard1/leader after winning as /collections/replacesinglenodetest_coll/leader_elect/shard1/election/72080967302512644-core_node7-n_0000000000 2> 3512213 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 3512213 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 3512213 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42945/solr/replacesinglenodetest_coll_shard3_replica_n4/ 2> 3512267 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42945/solr/replacesinglenodetest_coll_shard4_replica_n1/ shard4 2> 3512278 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 3512279 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.c.SyncStrategy http://127.0.0.1:42945/solr/replacesinglenodetest_coll_shard3_replica_n4/ has no replicas 2> 3512279 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/replacesinglenodetest_coll/leaders/shard3/leader after winning as /collections/replacesinglenodetest_coll/leader_elect/shard3/election/72080967302512644-core_node9-n_0000000000 2> 3512339 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node10&collection.configName=conf1&newCollection=true&name=replacesinglenodetest_coll_shard5_replica_n5&action=CREATE&numShards=5&collection=replacesinglenodetest_coll&shard=shard5&wt=javabin&version=2&replicaType=NRT} status=0 QTime=7717 2> 3512368 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42945/solr/replacesinglenodetest_coll_shard2_replica_n3/ shard2 2> 3512371 INFO (zkCallback-6559-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacesinglenodetest_coll/state.json] for collection [replacesinglenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3512407 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42945/solr/replacesinglenodetest_coll_shard1_replica_n2/ shard1 2> 3512426 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 3512428 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 3512438 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 3512463 INFO (zkCallback-6559-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacesinglenodetest_coll/state.json] for collection [replacesinglenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3512493 INFO (qtp299752042-10099) [n:127.0.0.1:42945_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf1&newCollection=true&name=replacesinglenodetest_coll_shard4_replica_n1&action=CREATE&numShards=5&collection=replacesinglenodetest_coll&shard=shard4&wt=javabin&version=2&replicaType=NRT} status=0 QTime=7894 2> 3512542 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42945/solr/replacesinglenodetest_coll_shard3_replica_n4/ shard3 2> 3512548 INFO (zkCallback-6559-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacesinglenodetest_coll/state.json] for collection [replacesinglenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3512579 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 3512585 INFO (zkCallback-6559-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacesinglenodetest_coll/state.json] for collection [replacesinglenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3512624 INFO (qtp299752042-10098) [n:127.0.0.1:42945_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=replacesinglenodetest_coll_shard1_replica_n2&action=CREATE&numShards=5&collection=replacesinglenodetest_coll&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=8019 2> 3512644 INFO (qtp299752042-10094) [n:127.0.0.1:42945_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf1&newCollection=true&name=replacesinglenodetest_coll_shard2_replica_n3&action=CREATE&numShards=5&collection=replacesinglenodetest_coll&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=8039 2> 3512704 INFO (zkCallback-6559-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacesinglenodetest_coll/state.json] for collection [replacesinglenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3512729 INFO (qtp299752042-10096) [n:127.0.0.1:42945_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node9&collection.configName=conf1&newCollection=true&name=replacesinglenodetest_coll_shard3_replica_n4&action=CREATE&numShards=5&collection=replacesinglenodetest_coll&shard=shard3&wt=javabin&version=2&replicaType=NRT} status=0 QTime=8107 2> 3512802 INFO (qtp299752042-10095) [n:127.0.0.1:42945_solr] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas 2> 3512804 INFO (qtp299752042-10095) [n:127.0.0.1:42945_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&collection.configName=conf1&name=replacesinglenodetest_coll&nrtReplicas=1&action=CREATE&numShards=5&tlogReplicas=0&wt=javabin&version=2} status=0 QTime=9180 2> 3512806 INFO (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForActiveCollection: replacesinglenodetest_coll 2> 3513041 ERROR (OverseerThreadFactory-6565-thread-2) [] o.a.s.c.a.c.OverseerCollectionMessageHandler Operation replacenode failed 2> => org.apache.solr.common.SolrException: No nodes other than the source node: 127.0.0.1:42945_solr are live, therefore replicas cannot be moved 2> at org.apache.solr.cloud.api.collections.ReplaceNodeCmd.call(ReplaceNodeCmd.java:86) 2> org.apache.solr.common.SolrException: No nodes other than the source node: 127.0.0.1:42945_solr are live, therefore replicas cannot be moved 2> at org.apache.solr.cloud.api.collections.ReplaceNodeCmd.call(ReplaceNodeCmd.java:86) ~[main/:?] 2> at org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:126) [main/:?] 2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:566) [main/:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289) [solr-solrj-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> 3513095 INFO (OverseerCollectionConfigSetProcessor-72080967302512644-127.0.0.1:42945_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 3513141 ERROR (qtp299752042-10097) [n:127.0.0.1:42945_solr] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: No nodes other than the source node: 127.0.0.1:42945_solr are live, therefore replicas cannot be moved 2> => org.apache.solr.common.SolrException: No nodes other than the source node: 127.0.0.1:42945_solr are live, therefore replicas cannot be moved 2> at org.apache.solr.client.solrj.SolrResponse.getException(SolrResponse.java:56) 2> org.apache.solr.common.SolrException: No nodes other than the source node: 127.0.0.1:42945_solr are live, therefore replicas cannot be moved 2> at org.apache.solr.client.solrj.SolrResponse.getException(SolrResponse.java:56) ~[solr-solrj-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.admin.api.ReplaceNodeAPI.replaceNode(ReplaceNodeAPI.java:89) ~[main/:?] 2> at org.apache.solr.handler.admin.CollectionsHandler$CollectionOperation.lambda$static$39(CollectionsHandler.java:1791) ~[main/:?] 2> at org.apache.solr.handler.admin.CollectionsHandler$CollectionOperation.execute(CollectionsHandler.java:1878) ~[main/:?] 2> at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:355) ~[main/:?] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:333) ~[main/:?] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:224) [main/:?] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:929) [main/:?] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:877) [main/:?] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:548) [main/:?] 2> at org.apache.solr.servlet.SolrDispatchFilter.dispatch(SolrDispatchFilter.java:252) [main/:?] 2> at org.apache.solr.servlet.SolrDispatchFilter.lambda$doFilter$0(SolrDispatchFilter.java:220) [main/:?] 2> at org.apache.solr.servlet.ServletUtils.traceHttpRequestExecution2(ServletUtils.java:257) [main/:?] 2> at org.apache.solr.servlet.ServletUtils.rateLimitRequest(ServletUtils.java:227) [main/:?] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:215) [main/:?] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197) [main/:?] 2> at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.apache.solr.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:187) [solr-test-framework-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:527) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1570) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1383) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1543) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1305) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:301) [jetty-rewrite-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:822) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.Server.handle(Server.java:563) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.HttpChannel.lambda$handle$0(HttpChannel.java:505) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:762) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:497) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:282) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314) [jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100) [jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) [jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:934) [jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1078) [jetty-util-10.0.13.jar:10.0.13] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> 3513227 INFO (qtp299752042-10097) [n:127.0.0.1:42945_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={sourceNode=127.0.0.1:42945_solr&action=REPLACENODE&wt=javabin&version=2} status=400 QTime=315 2> 3513241 INFO (TEST-ReplaceNodeTest.testFailOnSingleNode-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Ending testFailOnSingleNode 2> 3513810 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Starting test 2> 3513939 INFO (jetty-closer-6605-thread-1) [] o.e.j.s.Server Stopped Server@6fb3b20b{STOPPING}[10.0.13,sto=0] 2> 3513949 INFO (jetty-closer-6605-thread-1) [] o.e.j.s.AbstractConnector Stopped ServerConnector@1f1a2621{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0} 2> 3513950 INFO (jetty-closer-6605-thread-1) [] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3befe745{/solr,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,STOPPED} 2> 3514100 INFO (jetty-closer-6605-thread-1) [] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2035939958 2> 3514101 INFO (jetty-closer-6605-thread-1) [] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:42945_solr 2> 3514112 INFO (jetty-closer-6605-thread-1) [] o.a.s.c.ZkController Publish this node as DOWN... 2> 3514113 INFO (jetty-closer-6605-thread-1) [] o.a.s.c.ZkController Publish node=127.0.0.1:42945_solr as DOWN 2> 3514160 INFO (zkCallback-6559-thread-2) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) 2> 3514346 INFO (zkCallback-6559-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacesinglenodetest_coll/state.json] for collection [replacesinglenodetest_coll] has occurred - updating... (live nodes size: [0]) 2> 3514428 INFO (coreCloseExecutor-6608-thread-1) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@672895fc replacesinglenodetest_coll_shard5_replica_n5 2> 3514429 INFO (coreCloseExecutor-6608-thread-1) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replacesinglenodetest_coll.shard5.replica_n5 tag=SolrCore@672895fc 2> 3514437 INFO (coreCloseExecutor-6608-thread-2) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@3cafc6e8 replacesinglenodetest_coll_shard1_replica_n2 2> 3514449 INFO (coreCloseExecutor-6608-thread-3) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@47272f21 replacesinglenodetest_coll_shard4_replica_n1 2> 3514455 INFO (coreCloseExecutor-6608-thread-4) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@7b94e03a replacesinglenodetest_coll_shard3_replica_n4 2> 3514464 INFO (coreCloseExecutor-6608-thread-5) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@1bf9a219 replacesinglenodetest_coll_shard2_replica_n3 2> 3515432 INFO (coreCloseExecutor-6608-thread-1) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replacesinglenodetest_coll.shard5.leader tag=SolrCore@672895fc 2> 3515440 INFO (coreCloseExecutor-6608-thread-2) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replacesinglenodetest_coll.shard1.replica_n2 tag=SolrCore@3cafc6e8 2> 3515448 INFO (coreCloseExecutor-6608-thread-1) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard5 r:core_node10 x:replacesinglenodetest_coll_shard5_replica_n5] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 3517137 INFO (coreCloseExecutor-6608-thread-2) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replacesinglenodetest_coll.shard1.leader tag=SolrCore@3cafc6e8 2> 3517137 INFO (coreCloseExecutor-6608-thread-4) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replacesinglenodetest_coll.shard3.replica_n4 tag=SolrCore@7b94e03a 2> 3517203 INFO (coreCloseExecutor-6608-thread-2) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard1 r:core_node7 x:replacesinglenodetest_coll_shard1_replica_n2] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 3518259 INFO (coreCloseExecutor-6608-thread-4) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replacesinglenodetest_coll.shard3.leader tag=SolrCore@7b94e03a 2> 3518260 INFO (coreCloseExecutor-6608-thread-5) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replacesinglenodetest_coll.shard2.replica_n3 tag=SolrCore@1bf9a219 2> 3518298 INFO (coreCloseExecutor-6608-thread-4) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard3 r:core_node9 x:replacesinglenodetest_coll_shard3_replica_n4] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 3519429 INFO (coreCloseExecutor-6608-thread-5) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replacesinglenodetest_coll.shard2.leader tag=SolrCore@1bf9a219 2> 3519436 INFO (coreCloseExecutor-6608-thread-3) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replacesinglenodetest_coll.shard4.replica_n1 tag=SolrCore@47272f21 2> 3519471 INFO (coreCloseExecutor-6608-thread-5) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard2 r:core_node8 x:replacesinglenodetest_coll_shard2_replica_n3] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 3520671 INFO (coreCloseExecutor-6608-thread-3) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replacesinglenodetest_coll.shard4.leader tag=SolrCore@47272f21 2> 3520708 INFO (coreCloseExecutor-6608-thread-3) [n:127.0.0.1:42945_solr c:replacesinglenodetest_coll s:shard4 r:core_node6 x:replacesinglenodetest_coll_shard4_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 3520780 INFO (jetty-closer-6605-thread-1) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null 2> 3521398 INFO (jetty-closer-6605-thread-1) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null 2> 3521531 INFO (jetty-closer-6605-thread-1) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null 2> 3521533 INFO (jetty-closer-6605-thread-1) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null 2> 3521549 INFO (closeThreadPool-6609-thread-1) [] o.a.s.c.Overseer Overseer (id=72080967302512644-127.0.0.1:42945_solr-n_0000000000) closing 2> 3521551 INFO (OverseerStateUpdate-72080967302512644-127.0.0.1:42945_solr-n_0000000000) [n:127.0.0.1:42945_solr] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42945_solr 2> 3521580 INFO (closeThreadPool-6609-thread-2) [] o.a.s.c.Overseer Overseer (id=72080967302512644-127.0.0.1:42945_solr-n_0000000000) closing 2> 3521744 INFO (jetty-closer-6605-thread-1) [] o.a.s.c.Overseer Overseer (id=72080967302512644-127.0.0.1:42945_solr-n_0000000000) closing 2> 3521749 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkTestServer Shutting down ZkTestServer. 2> 3522019 WARN (ZkTestServer Run Thread) [] o.a.s.c.ZkTestServer Watch limit violations: 2> Maximum concurrent create/delete watches above limit: 2> 2> 2 /solr/clusterprops.json 2> 2 /solr/aliases.json 2> 2 /solr/collections/replacesinglenodetest_coll/terms/shard1 2> 2 /solr/collections/replacesinglenodetest_coll/terms/shard3 2> 2 /solr/collections/replacesinglenodetest_coll/terms/shard2 2> 2 /solr/collections/replacesinglenodetest_coll/terms/shard5 2> 2 /solr/collections/replacesinglenodetest_coll/terms/shard4 2> 2> Maximum concurrent data watches above limit: 2> 2> 24 /solr/collections/replacesinglenodetest_coll/state.json 2> 2> Maximum concurrent children watches above limit: 2> 2> 16 /solr/collections/replacesinglenodetest_coll/state.json 2> 7 /solr/overseer/collection-queue-work 2> 4 /solr/live_nodes 2> 4 /solr/collections 2> 2> 3522095 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster Starting cluster of 6 servers in /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002 2> 3522105 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER 2> 3522106 INFO (ZkTestServer Run Thread) [] o.a.s.c.ZkTestServer client port: 0.0.0.0/0.0.0.0:0 2> 3522106 INFO (ZkTestServer Run Thread) [] o.a.s.c.ZkTestServer Starting server 2> 3522139 WARN (ZkTestServer Run Thread) [] o.a.z.s.ServerCnxnFactory maxCnxns is not configured, using default value 0. 2> 3522206 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkTestServer start zk server on port: 33211 2> 3522491 WARN (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3522548 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3522604 INFO (zkConnectionManagerCallback-6611-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3522605 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3522605 WARN (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3522638 WARN (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3522676 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3522728 INFO (zkConnectionManagerCallback-6613-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3522729 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3522730 WARN (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3522730 WARN (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3522765 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3522797 INFO (zkConnectionManagerCallback-6615-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3522798 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3522798 WARN (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3522954 INFO (jetty-launcher-6616-thread-2) [] o.a.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0) 2> 3522954 INFO (jetty-launcher-6616-thread-2) [] o.a.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ... 2> 3522970 INFO (jetty-launcher-6616-thread-1) [] o.a.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0) 2> 3522971 INFO (jetty-launcher-6616-thread-1) [] o.a.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ... 2> 3523010 INFO (jetty-launcher-6616-thread-3) [] o.a.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0) 2> 3523010 INFO (jetty-launcher-6616-thread-3) [] o.a.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ... 2> 3523013 INFO (jetty-launcher-6616-thread-4) [] o.a.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0) 2> 3523013 INFO (jetty-launcher-6616-thread-4) [] o.a.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ... 2> 3523037 INFO (jetty-launcher-6616-thread-5) [] o.a.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0) 2> 3523038 INFO (jetty-launcher-6616-thread-5) [] o.a.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ... 2> 3523040 INFO (jetty-launcher-6616-thread-2) [] o.e.j.s.Server jetty-10.0.13; built: 2022-12-07T20:13:20.134Z; git: 1c2636ea05c0ca8de1ffd6ca7f3a98ac084c766d; jvm 11.0.20+8 2> 3523041 INFO (jetty-launcher-6616-thread-1) [] o.e.j.s.Server jetty-10.0.13; built: 2022-12-07T20:13:20.134Z; git: 1c2636ea05c0ca8de1ffd6ca7f3a98ac084c766d; jvm 11.0.20+8 2> 3523051 INFO (jetty-launcher-6616-thread-3) [] o.e.j.s.Server jetty-10.0.13; built: 2022-12-07T20:13:20.134Z; git: 1c2636ea05c0ca8de1ffd6ca7f3a98ac084c766d; jvm 11.0.20+8 2> 3523053 INFO (jetty-launcher-6616-thread-4) [] o.e.j.s.Server jetty-10.0.13; built: 2022-12-07T20:13:20.134Z; git: 1c2636ea05c0ca8de1ffd6ca7f3a98ac084c766d; jvm 11.0.20+8 2> 3523055 INFO (jetty-launcher-6616-thread-5) [] o.e.j.s.Server jetty-10.0.13; built: 2022-12-07T20:13:20.134Z; git: 1c2636ea05c0ca8de1ffd6ca7f3a98ac084c766d; jvm 11.0.20+8 2> 3523075 INFO (jetty-launcher-6616-thread-6) [] o.a.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0) 2> 3523075 INFO (jetty-launcher-6616-thread-6) [] o.a.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ... 2> 3523083 INFO (jetty-launcher-6616-thread-6) [] o.e.j.s.Server jetty-10.0.13; built: 2022-12-07T20:13:20.134Z; git: 1c2636ea05c0ca8de1ffd6ca7f3a98ac084c766d; jvm 11.0.20+8 2> 3523156 INFO (jetty-launcher-6616-thread-1) [] o.e.j.s.s.DefaultSessionIdManager Session workerName=node0 2> 3523166 INFO (jetty-launcher-6616-thread-1) [] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@535b1b74{/solr,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,AVAILABLE} 2> 3523174 INFO (jetty-launcher-6616-thread-1) [] o.e.j.s.AbstractConnector Started ServerConnector@61d107bc{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:39891} 2> 3523175 INFO (jetty-launcher-6616-thread-1) [] o.e.j.s.Server Started Server@11062e7b{STARTING}[10.0.13,sto=0] @3523521ms 2> 3523177 ERROR (jetty-launcher-6616-thread-1) [] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. 2> 3523177 INFO (jetty-launcher-6616-thread-1) [] o.a.s.s.CoreContainerProvider Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory 2> 3523177 INFO (jetty-launcher-6616-thread-1) [] o.a.s.s.CoreContainerProvider ___ _ Welcome to Apache Solr™ version 9.2.1 2> 3523178 INFO (jetty-launcher-6616-thread-1) [] o.a.s.s.CoreContainerProvider / __| ___| |_ _ Starting in cloud mode on port null 2> 3523178 INFO (jetty-launcher-6616-thread-1) [] o.a.s.s.CoreContainerProvider \__ \/ _ \ | '_| Install dir: /build/solr/src/solr-9.2.1/solr 2> 3523179 INFO (jetty-launcher-6616-thread-1) [] o.a.s.s.CoreContainerProvider |___/\___/_|_| Start time: 2023-10-03T08:47:21.183047Z 2> 3523181 INFO (jetty-launcher-6616-thread-1) [] o.a.s.s.CoreContainerProvider Solr Home: /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node1 (source: servlet config: solr.solr.home) 2> 3523183 WARN (jetty-launcher-6616-thread-1) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3523343 INFO (jetty-launcher-6616-thread-4) [] o.e.j.s.s.DefaultSessionIdManager Session workerName=node0 2> 3523460 INFO (jetty-launcher-6616-thread-3) [] o.e.j.s.s.DefaultSessionIdManager Session workerName=node0 2> 3523467 INFO (jetty-launcher-6616-thread-3) [] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4c402246{/solr,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,AVAILABLE} 2> 3523482 INFO (jetty-launcher-6616-thread-4) [] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3b84765d{/solr,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,AVAILABLE} 2> 3523498 INFO (jetty-launcher-6616-thread-4) [] o.e.j.s.AbstractConnector Started ServerConnector@8c24ea7{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:45375} 2> 3523498 INFO (jetty-launcher-6616-thread-4) [] o.e.j.s.Server Started Server@787c0d27{STARTING}[10.0.13,sto=0] @3523845ms 2> 3523512 ERROR (jetty-launcher-6616-thread-4) [] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. 2> 3523513 INFO (jetty-launcher-6616-thread-4) [] o.a.s.s.CoreContainerProvider Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory 2> 3523514 INFO (jetty-launcher-6616-thread-4) [] o.a.s.s.CoreContainerProvider ___ _ Welcome to Apache Solr™ version 9.2.1 2> 3523514 INFO (jetty-launcher-6616-thread-4) [] o.a.s.s.CoreContainerProvider / __| ___| |_ _ Starting in cloud mode on port null 2> 3523515 INFO (jetty-launcher-6616-thread-4) [] o.a.s.s.CoreContainerProvider \__ \/ _ \ | '_| Install dir: /build/solr/src/solr-9.2.1/solr 2> 3523515 INFO (jetty-launcher-6616-thread-4) [] o.a.s.s.CoreContainerProvider |___/\___/_|_| Start time: 2023-10-03T08:47:21.519194Z 2> 3523524 INFO (jetty-launcher-6616-thread-4) [] o.a.s.s.CoreContainerProvider Solr Home: /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node4 (source: servlet config: solr.solr.home) 2> 3523525 WARN (jetty-launcher-6616-thread-4) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3523545 INFO (jetty-launcher-6616-thread-5) [] o.e.j.s.s.DefaultSessionIdManager Session workerName=node0 2> 3523554 INFO (jetty-launcher-6616-thread-3) [] o.e.j.s.AbstractConnector Started ServerConnector@6364fe31{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:33755} 2> 3523555 INFO (jetty-launcher-6616-thread-3) [] o.e.j.s.Server Started Server@13b170aa{STARTING}[10.0.13,sto=0] @3523902ms 2> 3523557 ERROR (jetty-launcher-6616-thread-3) [] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. 2> 3523557 INFO (jetty-launcher-6616-thread-3) [] o.a.s.s.CoreContainerProvider Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory 2> 3523558 INFO (jetty-launcher-6616-thread-3) [] o.a.s.s.CoreContainerProvider ___ _ Welcome to Apache Solr™ version 9.2.1 2> 3523558 INFO (jetty-launcher-6616-thread-3) [] o.a.s.s.CoreContainerProvider / __| ___| |_ _ Starting in cloud mode on port null 2> 3523559 INFO (jetty-launcher-6616-thread-3) [] o.a.s.s.CoreContainerProvider \__ \/ _ \ | '_| Install dir: /build/solr/src/solr-9.2.1/solr 2> 3523559 INFO (jetty-launcher-6616-thread-3) [] o.a.s.s.CoreContainerProvider |___/\___/_|_| Start time: 2023-10-03T08:47:21.563367Z 2> 3523572 INFO (jetty-launcher-6616-thread-1) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3523592 INFO (jetty-launcher-6616-thread-3) [] o.a.s.s.CoreContainerProvider Solr Home: /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node3 (source: servlet config: solr.solr.home) 2> 3523594 WARN (jetty-launcher-6616-thread-3) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3523624 INFO (jetty-launcher-6616-thread-4) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3523634 INFO (jetty-launcher-6616-thread-5) [] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@71793b12{/solr,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,AVAILABLE} 2> 3523649 INFO (jetty-launcher-6616-thread-5) [] o.e.j.s.AbstractConnector Started ServerConnector@48fde0cb{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:33535} 2> 3523649 INFO (jetty-launcher-6616-thread-5) [] o.e.j.s.Server Started Server@388344da{STARTING}[10.0.13,sto=0] @3523996ms 2> 3523651 ERROR (jetty-launcher-6616-thread-5) [] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. 2> 3523651 INFO (jetty-launcher-6616-thread-5) [] o.a.s.s.CoreContainerProvider Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory 2> 3523652 INFO (jetty-launcher-6616-thread-5) [] o.a.s.s.CoreContainerProvider ___ _ Welcome to Apache Solr™ version 9.2.1 2> 3523652 INFO (jetty-launcher-6616-thread-5) [] o.a.s.s.CoreContainerProvider / __| ___| |_ _ Starting in cloud mode on port null 2> 3523653 INFO (jetty-launcher-6616-thread-5) [] o.a.s.s.CoreContainerProvider \__ \/ _ \ | '_| Install dir: /build/solr/src/solr-9.2.1/solr 2> 3523653 INFO (jetty-launcher-6616-thread-5) [] o.a.s.s.CoreContainerProvider |___/\___/_|_| Start time: 2023-10-03T08:47:21.657187Z 2> 3523654 INFO (jetty-launcher-6616-thread-5) [] o.a.s.s.CoreContainerProvider Solr Home: /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node5 (source: servlet config: solr.solr.home) 2> 3523655 WARN (jetty-launcher-6616-thread-5) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3523730 INFO (zkConnectionManagerCallback-6618-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3523736 INFO (jetty-launcher-6616-thread-1) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3523736 WARN (jetty-launcher-6616-thread-1) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3523753 INFO (jetty-launcher-6616-thread-1) [] o.a.s.c.NodeConfig solr.xml found in ZooKeeper. Loading... 2> 3523793 INFO (jetty-launcher-6616-thread-6) [] o.e.j.s.s.DefaultSessionIdManager Session workerName=node0 2> 3523806 INFO (jetty-launcher-6616-thread-5) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3523812 INFO (jetty-launcher-6616-thread-6) [] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@67b6baa8{/solr,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,AVAILABLE} 2> 3523814 INFO (jetty-launcher-6616-thread-2) [] o.e.j.s.s.DefaultSessionIdManager Session workerName=node0 2> 3523828 INFO (jetty-launcher-6616-thread-3) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3523850 INFO (zkConnectionManagerCallback-6620-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3523852 INFO (jetty-launcher-6616-thread-4) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3523853 WARN (jetty-launcher-6616-thread-4) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3523864 INFO (jetty-launcher-6616-thread-6) [] o.e.j.s.AbstractConnector Started ServerConnector@2063ff38{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:46533} 2> 3523865 INFO (jetty-launcher-6616-thread-6) [] o.e.j.s.Server Started Server@50f535ad{STARTING}[10.0.13,sto=0] @3524211ms 2> 3523868 INFO (jetty-launcher-6616-thread-2) [] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@48cb1d9b{/solr,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,AVAILABLE} 2> 3523873 INFO (jetty-launcher-6616-thread-4) [] o.a.s.c.NodeConfig solr.xml found in ZooKeeper. Loading... 2> 3523877 INFO (zkConnectionManagerCallback-6622-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3523877 INFO (jetty-launcher-6616-thread-3) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3523878 WARN (jetty-launcher-6616-thread-3) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3523885 INFO (jetty-launcher-6616-thread-2) [] o.e.j.s.AbstractConnector Started ServerConnector@20f7ea15{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:45673} 2> 3523889 INFO (jetty-launcher-6616-thread-2) [] o.e.j.s.Server Started Server@50e12401{STARTING}[10.0.13,sto=0] @3524235ms 2> 3523888 INFO (zkConnectionManagerCallback-6624-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3523893 ERROR (jetty-launcher-6616-thread-2) [] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. 2> 3523895 INFO (jetty-launcher-6616-thread-2) [] o.a.s.s.CoreContainerProvider Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory 2> 3523897 INFO (jetty-launcher-6616-thread-2) [] o.a.s.s.CoreContainerProvider ___ _ Welcome to Apache Solr™ version 9.2.1 2> 3523905 ERROR (jetty-launcher-6616-thread-6) [] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. 2> 3523907 INFO (jetty-launcher-6616-thread-3) [] o.a.s.c.NodeConfig solr.xml found in ZooKeeper. Loading... 2> 3523907 INFO (jetty-launcher-6616-thread-2) [] o.a.s.s.CoreContainerProvider / __| ___| |_ _ Starting in cloud mode on port null 2> 3523920 INFO (jetty-launcher-6616-thread-2) [] o.a.s.s.CoreContainerProvider \__ \/ _ \ | '_| Install dir: /build/solr/src/solr-9.2.1/solr 2> 3523925 INFO (jetty-launcher-6616-thread-2) [] o.a.s.s.CoreContainerProvider |___/\___/_|_| Start time: 2023-10-03T08:47:21.929392Z 2> 3523937 INFO (jetty-launcher-6616-thread-2) [] o.a.s.s.CoreContainerProvider Solr Home: /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node2 (source: servlet config: solr.solr.home) 2> 3523939 WARN (jetty-launcher-6616-thread-2) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3523932 INFO (jetty-launcher-6616-thread-6) [] o.a.s.s.CoreContainerProvider Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory 2> 3523943 INFO (jetty-launcher-6616-thread-6) [] o.a.s.s.CoreContainerProvider ___ _ Welcome to Apache Solr™ version 9.2.1 2> 3523944 INFO (jetty-launcher-6616-thread-6) [] o.a.s.s.CoreContainerProvider / __| ___| |_ _ Starting in cloud mode on port null 2> 3523944 INFO (jetty-launcher-6616-thread-6) [] o.a.s.s.CoreContainerProvider \__ \/ _ \ | '_| Install dir: /build/solr/src/solr-9.2.1/solr 2> 3523945 INFO (jetty-launcher-6616-thread-6) [] o.a.s.s.CoreContainerProvider |___/\___/_|_| Start time: 2023-10-03T08:47:21.949174Z 2> 3523946 INFO (jetty-launcher-6616-thread-6) [] o.a.s.s.CoreContainerProvider Solr Home: /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node6 (source: servlet config: solr.solr.home) 2> 3523947 WARN (jetty-launcher-6616-thread-6) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3523894 INFO (jetty-launcher-6616-thread-5) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3523952 WARN (jetty-launcher-6616-thread-5) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3523974 INFO (jetty-launcher-6616-thread-5) [] o.a.s.c.NodeConfig solr.xml found in ZooKeeper. Loading... 2> 3524060 INFO (jetty-launcher-6616-thread-6) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3524037 INFO (jetty-launcher-6616-thread-2) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3524100 INFO (zkConnectionManagerCallback-6628-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3524100 INFO (jetty-launcher-6616-thread-6) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3524101 WARN (jetty-launcher-6616-thread-6) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3524113 INFO (jetty-launcher-6616-thread-6) [] o.a.s.c.NodeConfig solr.xml found in ZooKeeper. Loading... 2> 3524128 INFO (zkConnectionManagerCallback-6626-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3524135 INFO (jetty-launcher-6616-thread-2) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3524136 WARN (jetty-launcher-6616-thread-2) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3524153 INFO (jetty-launcher-6616-thread-2) [] o.a.s.c.NodeConfig solr.xml found in ZooKeeper. Loading... 2> 3524378 INFO (jetty-launcher-6616-thread-4) [] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33211/solr 2> 3524379 INFO (jetty-launcher-6616-thread-4) [] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=false. Solr will be using Overseer based cluster state updates. 2> 3524409 INFO (jetty-launcher-6616-thread-5) [] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33211/solr 2> 3524410 INFO (jetty-launcher-6616-thread-5) [] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=false. Solr will be using Overseer based cluster state updates. 2> 3524489 INFO (jetty-launcher-6616-thread-5) [n:127.0.0.1:33535_solr] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3524541 INFO (jetty-launcher-6616-thread-1) [] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33211/solr 2> 3524542 INFO (jetty-launcher-6616-thread-1) [] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=false. Solr will be using Overseer based cluster state updates. 2> 3524592 INFO (jetty-launcher-6616-thread-4) [n:127.0.0.1:45375_solr] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3524605 INFO (zkConnectionManagerCallback-6668-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3524605 INFO (jetty-launcher-6616-thread-5) [n:127.0.0.1:33535_solr] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3524609 INFO (zkConnectionManagerCallback-6664-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3524610 INFO (jetty-launcher-6616-thread-4) [n:127.0.0.1:45375_solr] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3524673 INFO (jetty-launcher-6616-thread-1) [n:127.0.0.1:39891_solr] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3524728 INFO (zkConnectionManagerCallback-6674-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3524730 INFO (jetty-launcher-6616-thread-1) [n:127.0.0.1:39891_solr] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3524893 INFO (jetty-launcher-6616-thread-6) [] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33211/solr 2> 3524895 INFO (jetty-launcher-6616-thread-6) [] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=false. Solr will be using Overseer based cluster state updates. 2> 3524899 INFO (jetty-launcher-6616-thread-2) [] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33211/solr 2> 3524902 INFO (jetty-launcher-6616-thread-2) [] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=false. Solr will be using Overseer based cluster state updates. 2> 3524960 INFO (jetty-launcher-6616-thread-6) [n:127.0.0.1:46533_solr] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3524970 INFO (jetty-launcher-6616-thread-3) [] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33211/solr 2> 3524972 INFO (jetty-launcher-6616-thread-3) [] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=false. Solr will be using Overseer based cluster state updates. 2> 3525070 INFO (jetty-launcher-6616-thread-2) [n:127.0.0.1:45673_solr] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3525080 INFO (zkConnectionManagerCallback-6686-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3525084 INFO (jetty-launcher-6616-thread-6) [n:127.0.0.1:46533_solr] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3525149 INFO (jetty-launcher-6616-thread-3) [n:127.0.0.1:33755_solr] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3525164 INFO (zkConnectionManagerCallback-6691-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3525184 INFO (jetty-launcher-6616-thread-2) [n:127.0.0.1:45673_solr] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3525223 INFO (zkConnectionManagerCallback-6694-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3525228 INFO (jetty-launcher-6616-thread-3) [n:127.0.0.1:33755_solr] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3526046 WARN (jetty-launcher-6616-thread-4) [n:127.0.0.1:45375_solr] o.a.s.c.ZkController Contents of zookeeper /security.json are world-readable; consider setting up ACLs as described in https://solr.apache.org/guide/solr/latest/deployment-guide/zookeeper-access-control.html 2> 3526049 WARN (jetty-launcher-6616-thread-5) [n:127.0.0.1:33535_solr] o.a.s.c.ZkController Contents of zookeeper /security.json are world-readable; consider setting up ACLs as described in https://solr.apache.org/guide/solr/latest/deployment-guide/zookeeper-access-control.html 2> 3526067 WARN (jetty-launcher-6616-thread-6) [n:127.0.0.1:46533_solr] o.a.s.c.ZkController Contents of zookeeper /security.json are world-readable; consider setting up ACLs as described in https://solr.apache.org/guide/solr/latest/deployment-guide/zookeeper-access-control.html 2> 3526069 WARN (jetty-launcher-6616-thread-3) [n:127.0.0.1:33755_solr] o.a.s.c.ZkController Contents of zookeeper /security.json are world-readable; consider setting up ACLs as described in https://solr.apache.org/guide/solr/latest/deployment-guide/zookeeper-access-control.html 2> 3526069 WARN (jetty-launcher-6616-thread-2) [n:127.0.0.1:45673_solr] o.a.s.c.ZkController Contents of zookeeper /security.json are world-readable; consider setting up ACLs as described in https://solr.apache.org/guide/solr/latest/deployment-guide/zookeeper-access-control.html 2> 3526071 WARN (jetty-launcher-6616-thread-1) [n:127.0.0.1:39891_solr] o.a.s.c.ZkController Contents of zookeeper /security.json are world-readable; consider setting up ACLs as described in https://solr.apache.org/guide/solr/latest/deployment-guide/zookeeper-access-control.html 2> 3526245 INFO (jetty-launcher-6616-thread-4) [n:127.0.0.1:45375_solr] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=false. Solr will be using Overseer based cluster state updates. 2> 3526246 INFO (jetty-launcher-6616-thread-5) [n:127.0.0.1:33535_solr] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=false. Solr will be using Overseer based cluster state updates. 2> 3526273 INFO (jetty-launcher-6616-thread-1) [n:127.0.0.1:39891_solr] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=false. Solr will be using Overseer based cluster state updates. 2> 3526282 INFO (jetty-launcher-6616-thread-6) [n:127.0.0.1:46533_solr] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=false. Solr will be using Overseer based cluster state updates. 2> 3526320 INFO (jetty-launcher-6616-thread-3) [n:127.0.0.1:33755_solr] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=false. Solr will be using Overseer based cluster state updates. 2> 3526356 INFO (jetty-launcher-6616-thread-2) [n:127.0.0.1:45673_solr] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=false. Solr will be using Overseer based cluster state updates. 2> 3526481 INFO (jetty-launcher-6616-thread-6) [n:127.0.0.1:46533_solr] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46533_solr 2> 3526516 INFO (jetty-launcher-6616-thread-4) [n:127.0.0.1:45375_solr] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45375_solr 2> 3526528 INFO (jetty-launcher-6616-thread-6) [n:127.0.0.1:46533_solr] o.a.s.c.Overseer Overseer (id=72080969102262284-127.0.0.1:46533_solr-n_0000000000) starting 2> 3526532 INFO (jetty-launcher-6616-thread-2) [n:127.0.0.1:45673_solr] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45673_solr 2> 3526542 INFO (jetty-launcher-6616-thread-3) [n:127.0.0.1:33755_solr] o.a.s.c.ZkController Publish node=127.0.0.1:33755_solr as DOWN 2> 3526554 INFO (jetty-launcher-6616-thread-5) [n:127.0.0.1:33535_solr] o.a.s.c.ZkController Publish node=127.0.0.1:33535_solr as DOWN 2> 3526561 INFO (jetty-launcher-6616-thread-1) [n:127.0.0.1:39891_solr] o.a.s.c.ZkController Publish node=127.0.0.1:39891_solr as DOWN 2> 3526584 INFO (zkCallback-6693-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2) 2> 3526605 INFO (zkCallback-6673-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2) 2> 3526626 INFO (zkCallback-6667-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2) 2> 3526628 INFO (zkCallback-6685-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2) 2> 3526650 INFO (zkCallback-6663-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2) 2> 3526667 INFO (jetty-launcher-6616-thread-3) [n:127.0.0.1:33755_solr] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33755_solr 2> 3526705 INFO (zkCallback-6690-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2) 2> 3526714 INFO (jetty-launcher-6616-thread-5) [n:127.0.0.1:33535_solr] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33535_solr 2> 3526729 INFO (jetty-launcher-6616-thread-1) [n:127.0.0.1:39891_solr] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39891_solr 2> 3526744 INFO (zkCallback-6693-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) 2> 3526753 INFO (zkCallback-6667-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) 2> 3526754 INFO (zkCallback-6685-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) 2> 3526760 INFO (zkCallback-6673-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) 2> 3526760 INFO (zkCallback-6663-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) 2> 3526788 INFO (zkCallback-6690-thread-2) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4) 2> 3526821 INFO (zkCallback-6693-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (5) 2> 3526824 INFO (zkCallback-6685-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (5) 2> 3526837 INFO (zkCallback-6663-thread-2) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (5) 2> 3526848 WARN (jetty-launcher-6616-thread-4) [n:127.0.0.1:45375_solr] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info 2> 3526852 INFO (zkCallback-6673-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (5) 2> 3526852 INFO (zkCallback-6690-thread-2) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5) 2> 3526852 INFO (zkCallback-6667-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (5) 2> 3526878 WARN (jetty-launcher-6616-thread-2) [n:127.0.0.1:45673_solr] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info 2> 3526915 WARN (jetty-launcher-6616-thread-3) [n:127.0.0.1:33755_solr] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info 2> 3526964 INFO (OverseerStateUpdate-72080969102262284-127.0.0.1:46533_solr-n_0000000000) [n:127.0.0.1:46533_solr] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:46533_solr 2> 3526972 WARN (jetty-launcher-6616-thread-1) [n:127.0.0.1:39891_solr] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info 2> 3526978 WARN (jetty-launcher-6616-thread-5) [n:127.0.0.1:33535_solr] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info 2> 3527005 INFO (jetty-launcher-6616-thread-6) [n:127.0.0.1:46533_solr] o.a.s.c.ZkController Publish node=127.0.0.1:46533_solr as DOWN 2> 3527160 INFO (jetty-launcher-6616-thread-6) [n:127.0.0.1:46533_solr] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46533_solr 2> 3527263 INFO (zkCallback-6693-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6) 2> 3527274 INFO (zkCallback-6663-thread-2) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6) 2> 3527276 INFO (zkCallback-6673-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6) 2> 3527282 INFO (zkCallback-6667-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6) 2> 3527282 INFO (zkCallback-6685-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6) 2> 3527286 INFO (zkCallback-6690-thread-2) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6) 2> 3527512 WARN (jetty-launcher-6616-thread-6) [n:127.0.0.1:46533_solr] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info 2> 3533950 INFO (jetty-launcher-6616-thread-4) [n:127.0.0.1:45375_solr] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node4 2> 3535657 INFO (jetty-launcher-6616-thread-4) [n:127.0.0.1:45375_solr] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 3536244 INFO (jetty-launcher-6616-thread-4) [n:127.0.0.1:45375_solr] o.a.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45375, zkHost=127.0.0.1:33211/solr} 2> 3541046 INFO (jetty-launcher-6616-thread-2) [n:127.0.0.1:45673_solr] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node2 2> 3541592 INFO (jetty-launcher-6616-thread-3) [n:127.0.0.1:33755_solr] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node3 2> 3542593 INFO (jetty-launcher-6616-thread-5) [n:127.0.0.1:33535_solr] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node5 2> 3542617 INFO (jetty-launcher-6616-thread-6) [n:127.0.0.1:46533_solr] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node6 2> 3542651 INFO (jetty-launcher-6616-thread-1) [n:127.0.0.1:39891_solr] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node1 2> 3542659 INFO (jetty-launcher-6616-thread-2) [n:127.0.0.1:45673_solr] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 3542993 INFO (jetty-launcher-6616-thread-2) [n:127.0.0.1:45673_solr] o.a.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45673, zkHost=127.0.0.1:33211/solr} 2> 3543533 INFO (jetty-launcher-6616-thread-3) [n:127.0.0.1:33755_solr] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 3543831 INFO (jetty-launcher-6616-thread-5) [n:127.0.0.1:33535_solr] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 3543911 INFO (jetty-launcher-6616-thread-3) [n:127.0.0.1:33755_solr] o.a.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33755, zkHost=127.0.0.1:33211/solr} 2> 3544266 INFO (jetty-launcher-6616-thread-1) [n:127.0.0.1:39891_solr] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 3544434 INFO (jetty-launcher-6616-thread-5) [n:127.0.0.1:33535_solr] o.a.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33535, zkHost=127.0.0.1:33211/solr} 2> 3544680 INFO (jetty-launcher-6616-thread-1) [n:127.0.0.1:39891_solr] o.a.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39891, zkHost=127.0.0.1:33211/solr} 2> 3544878 INFO (jetty-launcher-6616-thread-6) [n:127.0.0.1:46533_solr] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 3545139 INFO (jetty-launcher-6616-thread-6) [n:127.0.0.1:46533_solr] o.a.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46533, zkHost=127.0.0.1:33211/solr} 2> 3545185 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForAllNodes: numServers=6 2> 3545185 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForNode: 127.0.0.1:45375_solr 2> 3545186 WARN (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3545243 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 15000ms for client to connect to ZooKeeper 2> 3545280 INFO (zkConnectionManagerCallback-6720-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3545283 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3545283 WARN (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3545372 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (6) 2> 3545439 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33211/solr ready 2> 3545440 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForNode: 127.0.0.1:45673_solr 2> 3545444 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForNode: 127.0.0.1:33755_solr 2> 3545445 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForNode: 127.0.0.1:33535_solr 2> 3545445 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForNode: 127.0.0.1:39891_solr 2> 3545445 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForNode: 127.0.0.1:46533_solr 2> 3545446 WARN (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3545513 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 45000ms for client to connect to ZooKeeper 2> 3545540 INFO (zkConnectionManagerCallback-6722-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3545544 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3545545 WARN (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3545724 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.ReplaceNodeTest total_jettys: 6 2> 3545897 INFO (OverseerThreadFactory-6703-thread-1) [] o.a.s.c.a.c.CreateCollectionCmd Create collection replacenodetest_coll 2> 3546363 INFO (OverseerStateUpdate-72080969102262284-127.0.0.1:46533_solr-n_0000000000) [n:127.0.0.1:46533_solr] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replacenodetest_coll_shard5_replica_n1", 2> "node_name":"127.0.0.1:45673_solr", 2> "base_url":"http://127.0.0.1:45673/solr", 2> "collection":"replacenodetest_coll", 2> "shard":"shard5", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 3546505 INFO (OverseerStateUpdate-72080969102262284-127.0.0.1:46533_solr-n_0000000000) [n:127.0.0.1:46533_solr] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replacenodetest_coll_shard2_replica_n2", 2> "node_name":"127.0.0.1:33535_solr", 2> "base_url":"http://127.0.0.1:33535/solr", 2> "collection":"replacenodetest_coll", 2> "shard":"shard2", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 3546653 INFO (OverseerStateUpdate-72080969102262284-127.0.0.1:46533_solr-n_0000000000) [n:127.0.0.1:46533_solr] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replacenodetest_coll_shard3_replica_n4", 2> "node_name":"127.0.0.1:33755_solr", 2> "base_url":"http://127.0.0.1:33755/solr", 2> "collection":"replacenodetest_coll", 2> "shard":"shard3", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 3546830 INFO (OverseerStateUpdate-72080969102262284-127.0.0.1:46533_solr-n_0000000000) [n:127.0.0.1:46533_solr] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replacenodetest_coll_shard1_replica_n6", 2> "node_name":"127.0.0.1:46533_solr", 2> "base_url":"http://127.0.0.1:46533/solr", 2> "collection":"replacenodetest_coll", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 3546969 INFO (OverseerStateUpdate-72080969102262284-127.0.0.1:46533_solr-n_0000000000) [n:127.0.0.1:46533_solr] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replacenodetest_coll_shard4_replica_n9", 2> "node_name":"127.0.0.1:45375_solr", 2> "base_url":"http://127.0.0.1:45375/solr", 2> "collection":"replacenodetest_coll", 2> "shard":"shard4", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 3547124 INFO (zkCallback-6685-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3547188 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr x:replacenodetest_coll_shard3_replica_n4] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_shard3_replica_n4&action=CREATE&numShards=5&collection=replacenodetest_coll&shard=shard3&wt=javabin&version=2&replicaType=NRT 2> 3547232 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr x:replacenodetest_coll_shard5_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_shard5_replica_n1&action=CREATE&numShards=5&collection=replacenodetest_coll&shard=shard5&wt=javabin&version=2&replicaType=NRT 2> 3547251 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr x:replacenodetest_coll_shard4_replica_n9] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node10&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_shard4_replica_n9&action=CREATE&numShards=5&collection=replacenodetest_coll&shard=shard4&wt=javabin&version=2&replicaType=NRT 2> 3547305 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr x:replacenodetest_coll_shard2_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_shard2_replica_n2&action=CREATE&numShards=5&collection=replacenodetest_coll&shard=shard2&wt=javabin&version=2&replicaType=NRT 2> 3547347 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr x:replacenodetest_coll_shard1_replica_n6] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_shard1_replica_n6&action=CREATE&numShards=5&collection=replacenodetest_coll&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 3547694 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 3547709 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 3547711 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 3547871 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.s.IndexSchema Schema name=cloud-dynamic 2> 3547872 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.s.IndexSchema Schema name=cloud-dynamic 2> 3547880 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.s.IndexSchema Schema name=cloud-dynamic 2> 3547880 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 3547910 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 3547996 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.s.IndexSchema Schema name=cloud-dynamic 2> 3548026 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.s.IndexSchema Schema name=cloud-dynamic 2> 3548107 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id 2> 3548107 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_shard4_replica_n9' using configuration from configset conf1, trusted=true 2> 3548109 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id 2> 3548109 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_shard5_replica_n1' using configuration from configset conf1, trusted=true 2> 3548111 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id 2> 3548111 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_shard1_replica_n6' using configuration from configset conf1, trusted=true 2> 3548119 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node6/replacenodetest_coll_shard1_replica_n6], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node6/replacenodetest_coll_shard1_replica_n6/data/] 2> 3548123 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id 2> 3548133 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node2/replacenodetest_coll_shard5_replica_n1], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node2/replacenodetest_coll_shard5_replica_n1/data/] 2> 3548214 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node4/replacenodetest_coll_shard4_replica_n9], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node4/replacenodetest_coll_shard4_replica_n9/data/] 2> 3548224 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_shard3_replica_n4' using configuration from configset conf1, trusted=true 2> 3548228 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node3/replacenodetest_coll_shard3_replica_n4], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node3/replacenodetest_coll_shard3_replica_n4/data/] 2> 3548246 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id 2> 3548247 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_shard2_replica_n2' using configuration from configset conf1, trusted=true 2> 3548259 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node5/replacenodetest_coll_shard2_replica_n2], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node5/replacenodetest_coll_shard2_replica_n2/data/] 2> 3552430 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 3552486 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 3552522 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 3552523 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 3552563 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 3552573 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 3552661 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 3552664 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 3552682 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 3552702 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 3552729 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 3554256 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 3554289 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 3554290 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 3554298 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 3554413 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 3554413 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 3554436 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 3554448 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 3554451 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 3554457 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 3554457 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 3554458 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 3554460 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723452316811264 2> 3554468 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 3554470 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723452327297024 2> 3554566 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 3554599 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 3554568 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 3554652 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 3554728 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 3554729 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723452598878208 2> 3554738 INFO (searcherExecutor-6734-thread-1-processing-127.0.0.1:33755_solr replacenodetest_coll_shard3_replica_n4 replacenodetest_coll shard3 core_node7) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 3554758 INFO (searcherExecutor-6730-thread-1-processing-127.0.0.1:46533_solr replacenodetest_coll_shard1_replica_n6 replacenodetest_coll shard1 core_node8) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 3554881 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replacenodetest_coll/terms/shard1 to Terms{values={core_node8=0}, version=0} for registerTerm 2> 3554881 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replacenodetest_coll/leaders/shard1 2> 3554891 INFO (searcherExecutor-6728-thread-1-processing-127.0.0.1:45673_solr replacenodetest_coll_shard5_replica_n1 replacenodetest_coll shard5 core_node3) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 3554896 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replacenodetest_coll/terms/shard3 to Terms{values={core_node7=0}, version=0} for registerTerm 2> 3554897 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replacenodetest_coll/leaders/shard3 2> 3554913 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 3554917 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 3554938 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 3554951 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 3554958 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723452839002112 2> 3554965 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 3554966 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 3555012 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 3555012 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 3555174 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replacenodetest_coll/terms/shard5 to Terms{values={core_node3=0}, version=0} for registerTerm 2> 3555174 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replacenodetest_coll/leaders/shard5 2> 3555210 INFO (searcherExecutor-6736-thread-1-processing-127.0.0.1:33535_solr replacenodetest_coll_shard2_replica_n2 replacenodetest_coll shard2 core_node5) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 3555241 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 3555241 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 3555241 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33755/solr/replacenodetest_coll_shard3_replica_n4/ 2> 3555252 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 3555254 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 3555254 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 3555255 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:46533/solr/replacenodetest_coll_shard1_replica_n6/ 2> 3555262 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 3555268 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 3555272 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 3555274 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.c.SyncStrategy http://127.0.0.1:33755/solr/replacenodetest_coll_shard3_replica_n4/ has no replicas 2> 3555274 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/replacenodetest_coll/leaders/shard3/leader after winning as /collections/replacenodetest_coll/leader_elect/shard3/election/72080969102262286-core_node7-n_0000000000 2> 3555277 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 3555278 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723453174546432 2> 3555336 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.c.SyncStrategy http://127.0.0.1:46533/solr/replacenodetest_coll_shard1_replica_n6/ has no replicas 2> 3555336 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/replacenodetest_coll/leaders/shard1/leader after winning as /collections/replacenodetest_coll/leader_elect/shard1/election/72080969102262284-core_node8-n_0000000000 2> 3555452 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replacenodetest_coll/terms/shard2 to Terms{values={core_node5=0}, version=0} for registerTerm 2> 3555452 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replacenodetest_coll/leaders/shard2 2> 3555499 INFO (searcherExecutor-6732-thread-1-processing-127.0.0.1:45375_solr replacenodetest_coll_shard4_replica_n9 replacenodetest_coll shard4 core_node10) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 3555504 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33755/solr/replacenodetest_coll_shard3_replica_n4/ shard3 2> 3555505 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 3555506 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 3555506 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:45673/solr/replacenodetest_coll_shard5_replica_n1/ 2> 3555519 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:46533/solr/replacenodetest_coll_shard1_replica_n6/ shard1 2> 3555528 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 3555549 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:45673/solr/replacenodetest_coll_shard5_replica_n1/ has no replicas 2> 3555549 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/replacenodetest_coll/leaders/shard5/leader after winning as /collections/replacenodetest_coll/leader_elect/shard5/election/72080969102262285-core_node3-n_0000000000 2> 3555637 INFO (zkCallback-6693-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3555640 INFO (zkCallback-6693-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3555644 INFO (zkCallback-6685-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3555674 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 3555721 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replacenodetest_coll/terms/shard4 to Terms{values={core_node10=0}, version=0} for registerTerm 2> 3555722 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replacenodetest_coll/leaders/shard4 2> 3555743 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 3555744 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 3555744 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33535/solr/replacenodetest_coll_shard2_replica_n2/ 2> 3555748 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:45673/solr/replacenodetest_coll_shard5_replica_n1/ shard5 2> 3555769 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 3555782 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:33535/solr/replacenodetest_coll_shard2_replica_n2/ has no replicas 2> 3555782 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/replacenodetest_coll/leaders/shard2/leader after winning as /collections/replacenodetest_coll/leader_elect/shard2/election/72080969102262281-core_node5-n_0000000000 2> 3555843 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_shard3_replica_n4&action=CREATE&numShards=5&collection=replacenodetest_coll&shard=shard3&wt=javabin&version=2&replicaType=NRT} status=0 QTime=8658 2> 3555888 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33535/solr/replacenodetest_coll_shard2_replica_n2/ shard2 2> 3555921 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 3555922 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 3555922 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:45375/solr/replacenodetest_coll_shard4_replica_n9/ 2> 3555930 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 3555946 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.c.SyncStrategy http://127.0.0.1:45375/solr/replacenodetest_coll_shard4_replica_n9/ has no replicas 2> 3555946 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/replacenodetest_coll/leaders/shard4/leader after winning as /collections/replacenodetest_coll/leader_elect/shard4/election/72080969102262282-core_node10-n_0000000000 2> 3555990 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:45375/solr/replacenodetest_coll_shard4_replica_n9/ shard4 2> 3556232 INFO (zkCallback-6693-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3556232 INFO (zkCallback-6693-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3556233 INFO (zkCallback-6667-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3556233 INFO (zkCallback-6690-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3556234 INFO (zkCallback-6663-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3556235 INFO (zkCallback-6685-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3556240 INFO (zkCallback-6685-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3556266 INFO (zkCallback-6693-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3556269 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 3556283 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 3556287 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 3556304 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 3556404 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_shard2_replica_n2&action=CREATE&numShards=5&collection=replacenodetest_coll&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=9118 2> 3556436 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node10&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_shard4_replica_n9&action=CREATE&numShards=5&collection=replacenodetest_coll&shard=shard4&wt=javabin&version=2&replicaType=NRT} status=0 QTime=9186 2> 3556459 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_shard5_replica_n1&action=CREATE&numShards=5&collection=replacenodetest_coll&shard=shard5&wt=javabin&version=2&replicaType=NRT} status=0 QTime=9229 2> 3556491 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_shard1_replica_n6&action=CREATE&numShards=5&collection=replacenodetest_coll&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=9146 2> 3556531 INFO (qtp1839402485-10227) [n:127.0.0.1:33755_solr] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas 2> 3556579 INFO (zkCallback-6690-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3556581 INFO (zkCallback-6690-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3556581 INFO (zkCallback-6667-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3556583 INFO (zkCallback-6685-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3556583 INFO (zkCallback-6693-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3556584 INFO (zkCallback-6693-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3556585 INFO (zkCallback-6685-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3556585 INFO (zkCallback-6663-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3556588 INFO (zkCallback-6663-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3556581 INFO (zkCallback-6667-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3556583 INFO (zkCallback-6693-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3556602 INFO (qtp1839402485-10227) [n:127.0.0.1:33755_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&collection.configName=conf1&name=replacenodetest_coll&nrtReplicas=1&action=CREATE&numShards=5&tlogReplicas=0&createNodeSet=127.0.0.1:46533_solr,127.0.0.1:45673_solr,127.0.0.1:33535_solr,127.0.0.1:45375_solr,127.0.0.1:33755_solr&wt=javabin&version=2} status=0 QTime=10824 2> 3556608 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForActiveCollection: replacenodetest_coll 2> 3556647 INFO (zkCallback-6685-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3556738 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.c.ReplaceNodeTest excluded_node : 127.0.0.1:39891_solr 2> 3556896 INFO (qtp1839402485-10233) [n:127.0.0.1:33755_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=000&sourceNode=127.0.0.1:46533_solr&action=REPLACENODE&targetNode=127.0.0.1:39891_solr&wt=javabin&version=2} status=0 QTime=150 2> 3556965 INFO (qtp1839402485-10226) [n:127.0.0.1:33755_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=000&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=65 2> 3556978 INFO (OverseerThreadFactory-6703-thread-2) [] o.a.s.c.a.c.ReplaceNodeCmd Going to create replica for collection=replacenodetest_coll shard=shard1 on node=127.0.0.1:39891_solr 2> 3556984 INFO (OverseerCollectionConfigSetProcessor-72080969102262284-127.0.0.1:46533_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 3557033 INFO (OverseerThreadFactory-6703-thread-2) [] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:39891_solr for creating new replica of shard shard1 for collection replacenodetest_coll 2> 3557080 INFO (OverseerThreadFactory-6703-thread-2) [] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command. 2> 3557131 INFO (OverseerStateUpdate-72080969102262284-127.0.0.1:46533_solr-n_0000000000) [n:127.0.0.1:46533_solr] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replacenodetest_coll_shard1_replica_n11", 2> "node_name":"127.0.0.1:39891_solr", 2> "base_url":"http://127.0.0.1:39891/solr", 2> "collection":"replacenodetest_coll", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"addreplica"} 2> 3557282 INFO (zkCallback-6690-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3557283 INFO (zkCallback-6667-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3557283 INFO (zkCallback-6663-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3557283 INFO (zkCallback-6690-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3557283 INFO (zkCallback-6667-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3557283 INFO (zkCallback-6663-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3557284 INFO (zkCallback-6693-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3557284 INFO (zkCallback-6693-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3557285 INFO (zkCallback-6693-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3557286 INFO (zkCallback-6685-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3557286 INFO (zkCallback-6685-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3557286 INFO (zkCallback-6685-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3557420 INFO (qtp1863775278-10206) [n:127.0.0.1:39891_solr x:replacenodetest_coll_shard1_replica_n11] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=000356710300502291&qt=/admin/cores&coreNodeName=core_node12&collection.configName=conf1&name=replacenodetest_coll_shard1_replica_n11&action=CREATE&collection=replacenodetest_coll&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=38 2> 3557422 INFO (parallelCoreAdminAPIBaseExecutor-6708-thread-1-processing-127.0.0.1:39891_solr replacenodetest_coll_shard1_replica_n11 000356710300502291 create) [n:127.0.0.1:39891_solr x:replacenodetest_coll_shard1_replica_n11] o.a.s.h.a.CoreAdminOperation core create command async=000356710300502291&qt=/admin/cores&coreNodeName=core_node12&collection.configName=conf1&name=replacenodetest_coll_shard1_replica_n11&action=CREATE&collection=replacenodetest_coll&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 3557440 INFO (qtp1863775278-10208) [n:127.0.0.1:39891_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 000356710300502291 2> 3557446 INFO (qtp1863775278-10208) [n:127.0.0.1:39891_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=000356710300502291&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=7 2> 3557721 INFO (zkCallback-6667-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3557721 INFO (zkCallback-6667-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3557721 INFO (zkCallback-6690-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3557724 INFO (zkCallback-6685-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3557727 INFO (zkCallback-6693-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3557728 INFO (zkCallback-6685-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3557729 INFO (zkCallback-6685-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3557729 INFO (zkCallback-6693-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3557730 INFO (zkCallback-6693-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3557727 INFO (zkCallback-6690-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3557735 INFO (zkCallback-6663-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3557735 INFO (zkCallback-6663-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3557859 INFO (parallelCoreAdminAPIBaseExecutor-6708-thread-1-processing-127.0.0.1:39891_solr replacenodetest_coll_shard1_replica_n11 000356710300502291 create) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 3557922 INFO (parallelCoreAdminAPIBaseExecutor-6708-thread-1-processing-127.0.0.1:39891_solr replacenodetest_coll_shard1_replica_n11 000356710300502291 create) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.s.IndexSchema Schema name=cloud-dynamic 2> 3558019 INFO (parallelCoreAdminAPIBaseExecutor-6708-thread-1-processing-127.0.0.1:39891_solr replacenodetest_coll_shard1_replica_n11 000356710300502291 create) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id 2> 3558019 INFO (parallelCoreAdminAPIBaseExecutor-6708-thread-1-processing-127.0.0.1:39891_solr replacenodetest_coll_shard1_replica_n11 000356710300502291 create) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_shard1_replica_n11' using configuration from configset conf1, trusted=true 2> 3558022 INFO (parallelCoreAdminAPIBaseExecutor-6708-thread-1-processing-127.0.0.1:39891_solr replacenodetest_coll_shard1_replica_n11 000356710300502291 create) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node1/replacenodetest_coll_shard1_replica_n11], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node1/replacenodetest_coll_shard1_replica_n11/data/] 2> 3558031 INFO (qtp1839402485-10228) [n:127.0.0.1:33755_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=000&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=58 2> 3558462 INFO (qtp1863775278-10207) [n:127.0.0.1:39891_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 000356710300502291 2> 3558463 INFO (qtp1863775278-10207) [n:127.0.0.1:39891_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=000356710300502291&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 3559120 INFO (qtp1839402485-10231) [n:127.0.0.1:33755_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=000&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=63 2> 3559482 INFO (qtp1863775278-10209) [n:127.0.0.1:39891_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 000356710300502291 2> 3559482 INFO (qtp1863775278-10209) [n:127.0.0.1:39891_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=000356710300502291&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=3 2> 3560257 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=000&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=96 2> 3560411 INFO (parallelCoreAdminAPIBaseExecutor-6708-thread-1-processing-127.0.0.1:39891_solr replacenodetest_coll_shard1_replica_n11 000356710300502291 create) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 3560466 INFO (parallelCoreAdminAPIBaseExecutor-6708-thread-1-processing-127.0.0.1:39891_solr replacenodetest_coll_shard1_replica_n11 000356710300502291 create) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 3560467 INFO (parallelCoreAdminAPIBaseExecutor-6708-thread-1-processing-127.0.0.1:39891_solr replacenodetest_coll_shard1_replica_n11 000356710300502291 create) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 3560497 INFO (qtp1863775278-10203) [n:127.0.0.1:39891_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 000356710300502291 2> 3560499 INFO (qtp1863775278-10203) [n:127.0.0.1:39891_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=000356710300502291&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=4 2> 3560527 INFO (parallelCoreAdminAPIBaseExecutor-6708-thread-1-processing-127.0.0.1:39891_solr replacenodetest_coll_shard1_replica_n11 000356710300502291 create) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 3560545 INFO (parallelCoreAdminAPIBaseExecutor-6708-thread-1-processing-127.0.0.1:39891_solr replacenodetest_coll_shard1_replica_n11 000356710300502291 create) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 3560636 INFO (parallelCoreAdminAPIBaseExecutor-6708-thread-1-processing-127.0.0.1:39891_solr replacenodetest_coll_shard1_replica_n11 000356710300502291 create) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 3560664 INFO (parallelCoreAdminAPIBaseExecutor-6708-thread-1-processing-127.0.0.1:39891_solr replacenodetest_coll_shard1_replica_n11 000356710300502291 create) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 3560673 INFO (parallelCoreAdminAPIBaseExecutor-6708-thread-1-processing-127.0.0.1:39891_solr replacenodetest_coll_shard1_replica_n11 000356710300502291 create) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 3560680 INFO (parallelCoreAdminAPIBaseExecutor-6708-thread-1-processing-127.0.0.1:39891_solr replacenodetest_coll_shard1_replica_n11 000356710300502291 create) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723458838953984 2> 3560808 INFO (searcherExecutor-6759-thread-1-processing-replacenodetest_coll_shard1_replica_n11 000356710300502291 core_node12 create 127.0.0.1:39891_solr replacenodetest_coll shard1) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 3560932 INFO (parallelCoreAdminAPIBaseExecutor-6708-thread-1-processing-127.0.0.1:39891_solr replacenodetest_coll_shard1_replica_n11 000356710300502291 create) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replacenodetest_coll/terms/shard1 to Terms{values={core_node8=0, core_node12=0}, version=1} for registerTerm 2> 3560933 INFO (parallelCoreAdminAPIBaseExecutor-6708-thread-1-processing-127.0.0.1:39891_solr replacenodetest_coll_shard1_replica_n11 000356710300502291 create) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replacenodetest_coll/leaders/shard1 2> 3561128 INFO (parallelCoreAdminAPIBaseExecutor-6708-thread-1-processing-127.0.0.1:39891_solr replacenodetest_coll_shard1_replica_n11 000356710300502291 create) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.ZkController Core needs to recover:replacenodetest_coll_shard1_replica_n11 2> 3561146 INFO (updateExecutor-6669-thread-1-processing-replacenodetest_coll_shard1_replica_n11 000356710300502291 core_node12 create 127.0.0.1:39891_solr replacenodetest_coll shard1) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.u.DefaultSolrCoreState Running recovery 2> 3561157 INFO (recoveryExecutor-6672-thread-1-processing-replacenodetest_coll_shard1_replica_n11 000356710300502291 core_node12 create 127.0.0.1:39891_solr replacenodetest_coll shard1) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true 2> 3561159 INFO (recoveryExecutor-6672-thread-1-processing-replacenodetest_coll_shard1_replica_n11 000356710300502291 core_node12 create 127.0.0.1:39891_solr replacenodetest_coll shard1) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.RecoveryStrategy startupVersions is empty 2> 3561226 INFO (qtp782501020-10253) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.c.S.Request webapp=/solr path=/admin/ping params={wt=javabin&version=2} hits=0 status=0 QTime=3 2> 3561251 INFO (qtp782501020-10253) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.c.S.Request webapp=/solr path=/admin/ping params={wt=javabin&version=2} status=0 QTime=28 2> 3561254 INFO (recoveryExecutor-6672-thread-1-processing-replacenodetest_coll_shard1_replica_n11 000356710300502291 core_node12 create 127.0.0.1:39891_solr replacenodetest_coll shard1) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[replacenodetest_coll_shard1_replica_n11] 2> 3561267 INFO (recoveryExecutor-6672-thread-1-processing-replacenodetest_coll_shard1_replica_n11 000356710300502291 core_node12 create 127.0.0.1:39891_solr replacenodetest_coll shard1) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null} 2> 3561302 INFO (recoveryExecutor-6672-thread-1-processing-replacenodetest_coll_shard1_replica_n11 000356710300502291 core_node12 create 127.0.0.1:39891_solr replacenodetest_coll shard1) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.RecoveryStrategy Publishing state of core [replacenodetest_coll_shard1_replica_n11] as recovering, leader is [http://127.0.0.1:46533/solr/replacenodetest_coll_shard1_replica_n6/] and I am [http://127.0.0.1:39891/solr/replacenodetest_coll_shard1_replica_n11/] 2> 3561327 INFO (qtp1839402485-10227) [n:127.0.0.1:33755_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=000&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=59 2> 3561382 INFO (recoveryExecutor-6672-thread-1-processing-replacenodetest_coll_shard1_replica_n11 000356710300502291 core_node12 create 127.0.0.1:39891_solr replacenodetest_coll shard1) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:46533/solr]; [WaitForState: action=PREPRECOVERY&core=replacenodetest_coll_shard1_replica_n6&nodeName=127.0.0.1:39891_solr&coreNodeName=core_node12&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true] 2> 3561388 INFO (qtp782501020-10256) [n:127.0.0.1:46533_solr x:replacenodetest_coll_shard1_replica_n6] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node12, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true 2> 3561402 INFO (zkCallback-6693-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3561402 INFO (zkCallback-6693-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3561402 INFO (zkCallback-6693-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3561404 INFO (zkCallback-6690-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3561404 INFO (zkCallback-6673-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3561404 INFO (zkCallback-6667-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3561404 INFO (zkCallback-6690-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3561404 INFO (zkCallback-6673-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3561405 INFO (zkCallback-6667-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3561420 INFO (zkCallback-6663-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3561421 INFO (zkCallback-6663-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3561422 INFO (zkCallback-6685-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3561428 INFO (zkCallback-6685-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3561428 INFO (zkCallback-6685-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3561477 INFO (zkCallback-6673-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3561505 INFO (watches-6695-thread-3) [n:127.0.0.1:46533_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=replacenodetest_coll, shard=shard1, thisCore=replacenodetest_coll_shard1_replica_n6, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:39891_solr, coreNodeName=core_node12, onlyIfActiveCheckResult=false, nodeProps: core_node12:{ 2> "core":"replacenodetest_coll_shard1_replica_n11", 2> "node_name":"127.0.0.1:39891_solr", 2> "base_url":"http://127.0.0.1:39891/solr", 2> "state":"recovering", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 3561507 INFO (qtp782501020-10256) [n:127.0.0.1:46533_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=replacenodetest_coll, shard=shard1, thisCore=replacenodetest_coll_shard1_replica_n6, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:39891_solr, coreNodeName=core_node12, onlyIfActiveCheckResult=false, nodeProps: core_node12:{ 2> "core":"replacenodetest_coll_shard1_replica_n11", 2> "node_name":"127.0.0.1:39891_solr", 2> "base_url":"http://127.0.0.1:39891/solr", 2> "state":"down", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 3561507 INFO (watches-6695-thread-1) [n:127.0.0.1:46533_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=replacenodetest_coll, shard=shard1, thisCore=replacenodetest_coll_shard1_replica_n6, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:39891_solr, coreNodeName=core_node12, onlyIfActiveCheckResult=false, nodeProps: core_node12:{ 2> "core":"replacenodetest_coll_shard1_replica_n11", 2> "node_name":"127.0.0.1:39891_solr", 2> "base_url":"http://127.0.0.1:39891/solr", 2> "state":"recovering", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 3561507 INFO (qtp782501020-10256) [n:127.0.0.1:46533_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=replacenodetest_coll, shard=shard1, thisCore=replacenodetest_coll_shard1_replica_n6, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:39891_solr, coreNodeName=core_node12, onlyIfActiveCheckResult=false, nodeProps: core_node12:{ 2> "core":"replacenodetest_coll_shard1_replica_n11", 2> "node_name":"127.0.0.1:39891_solr", 2> "base_url":"http://127.0.0.1:39891/solr", 2> "state":"recovering", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 3561508 INFO (qtp782501020-10256) [n:127.0.0.1:46533_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=replacenodetest_coll, shard=shard1, thisCore=replacenodetest_coll_shard1_replica_n6, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:39891_solr, coreNodeName=core_node12, onlyIfActiveCheckResult=false, nodeProps: core_node12:{ 2> "core":"replacenodetest_coll_shard1_replica_n11", 2> "node_name":"127.0.0.1:39891_solr", 2> "base_url":"http://127.0.0.1:39891/solr", 2> "state":"recovering", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 3561509 INFO (qtp782501020-10256) [n:127.0.0.1:46533_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:39891_solr&onlyIfLeaderActive=true&core=replacenodetest_coll_shard1_replica_n6&coreNodeName=core_node12&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=123 2> 3561519 INFO (qtp1863775278-10204) [n:127.0.0.1:39891_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 000356710300502291 2> 3561519 INFO (qtp1863775278-10204) [n:127.0.0.1:39891_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=000356710300502291&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 3561537 INFO (watches-6695-thread-2) [n:127.0.0.1:46533_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=replacenodetest_coll, shard=shard1, thisCore=replacenodetest_coll_shard1_replica_n6, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:39891_solr, coreNodeName=core_node12, onlyIfActiveCheckResult=false, nodeProps: core_node12:{ 2> "core":"replacenodetest_coll_shard1_replica_n11", 2> "node_name":"127.0.0.1:39891_solr", 2> "base_url":"http://127.0.0.1:39891/solr", 2> "state":"recovering", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 3562025 INFO (recoveryExecutor-6672-thread-1-processing-replacenodetest_coll_shard1_replica_n11 000356710300502291 core_node12 create 127.0.0.1:39891_solr replacenodetest_coll shard1) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:46533/solr/replacenodetest_coll_shard1_replica_n6/] - recoveringAfterStartup=[true] 2> 3562029 WARN (recoveryExecutor-6672-thread-1-processing-replacenodetest_coll_shard1_replica_n11 000356710300502291 core_node12 create 127.0.0.1:39891_solr replacenodetest_coll shard1) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.u.PeerSyncWithLeader no frame of reference to tell if we've missed updates 2> 3562031 INFO (recoveryExecutor-6672-thread-1-processing-replacenodetest_coll_shard1_replica_n11 000356710300502291 core_node12 create 127.0.0.1:39891_solr replacenodetest_coll shard1) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.RecoveryStrategy PeerSync Recovery was not successful - trying replication. 2> 3562032 INFO (recoveryExecutor-6672-thread-1-processing-replacenodetest_coll_shard1_replica_n11 000356710300502291 core_node12 create 127.0.0.1:39891_solr replacenodetest_coll shard1) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.RecoveryStrategy Starting Replication Recovery. 2> 3562033 INFO (recoveryExecutor-6672-thread-1-processing-replacenodetest_coll_shard1_replica_n11 000356710300502291 core_node12 create 127.0.0.1:39891_solr replacenodetest_coll shard1) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.RecoveryStrategy Attempting to replicate from [http://127.0.0.1:46533/solr/replacenodetest_coll_shard1_replica_n6/]. 2> 3562315 INFO (qtp1178207868-10236) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46533/solr/replacenodetest_coll_shard1_replica_n6/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 95 2> 3562325 INFO (qtp1839402485-10228) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46533/solr/replacenodetest_coll_shard1_replica_n6/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 67 2> 3562403 INFO (qtp774898083-10257) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46533/solr/replacenodetest_coll_shard1_replica_n6/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 79 2> 3562409 INFO (qtp101036055-10223) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46533/solr/replacenodetest_coll_shard1_replica_n6/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 73 2> 3562412 INFO (qtp1839402485-10231) [n:127.0.0.1:33755_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=000&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=58 2> 3562503 INFO (qtp1863775278-10207) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.u.p.DistributedUpdateProcessor Ignoring commit while not ACTIVE - state: BUFFERING replay: false 2> 3562503 INFO (qtp1863775278-10207) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46533/solr/replacenodetest_coll_shard1_replica_n6/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 19 2> 3562506 INFO (qtp782501020-10242) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 469 2> 3562516 INFO (qtp782501020-10249) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node8 x:replacenodetest_coll_shard1_replica_n6] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 3562521 INFO (recoveryExecutor-6672-thread-1-processing-replacenodetest_coll_shard1_replica_n11 000356710300502291 core_node12 create 127.0.0.1:39891_solr replacenodetest_coll shard1) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.h.IndexFetcher Leader's generation: 1 2> 3562521 INFO (recoveryExecutor-6672-thread-1-processing-replacenodetest_coll_shard1_replica_n11 000356710300502291 core_node12 create 127.0.0.1:39891_solr replacenodetest_coll shard1) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.h.IndexFetcher Leader's version: 0 2> 3562522 INFO (recoveryExecutor-6672-thread-1-processing-replacenodetest_coll_shard1_replica_n11 000356710300502291 core_node12 create 127.0.0.1:39891_solr replacenodetest_coll shard1) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.h.IndexFetcher Follower's generation: 1 2> 3562522 INFO (recoveryExecutor-6672-thread-1-processing-replacenodetest_coll_shard1_replica_n11 000356710300502291 core_node12 create 127.0.0.1:39891_solr replacenodetest_coll shard1) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.h.IndexFetcher Follower's version: 0 2> 3562522 INFO (recoveryExecutor-6672-thread-1-processing-replacenodetest_coll_shard1_replica_n11 000356710300502291 core_node12 create 127.0.0.1:39891_solr replacenodetest_coll shard1) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.RecoveryStrategy No replay needed. 2> 3562537 INFO (recoveryExecutor-6672-thread-1-processing-replacenodetest_coll_shard1_replica_n11 000356710300502291 core_node12 create 127.0.0.1:39891_solr replacenodetest_coll shard1) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.RecoveryStrategy Replication Recovery was successful. 2> 3562537 INFO (recoveryExecutor-6672-thread-1-processing-replacenodetest_coll_shard1_replica_n11 000356710300502291 core_node12 create 127.0.0.1:39891_solr replacenodetest_coll shard1) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.RecoveryStrategy Registering as Active after recovery. 2> 3562545 INFO (recoveryExecutor-6672-thread-1-processing-replacenodetest_coll_shard1_replica_n11 000356710300502291 core_node12 create 127.0.0.1:39891_solr replacenodetest_coll shard1) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.RecoveryStrategy Updating version bucket highest from index after successful recovery. 2> 3562546 INFO (recoveryExecutor-6672-thread-1-processing-replacenodetest_coll_shard1_replica_n11 000356710300502291 core_node12 create 127.0.0.1:39891_solr replacenodetest_coll shard1) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723460795596800 2> 3562567 INFO (recoveryExecutor-6672-thread-1-processing-replacenodetest_coll_shard1_replica_n11 000356710300502291 core_node12 create 127.0.0.1:39891_solr replacenodetest_coll shard1) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.RecoveryStrategy Finished recovery process, successful=[true] msTimeTaken=1408.0 2> 3562578 INFO (recoveryExecutor-6672-thread-1-processing-replacenodetest_coll_shard1_replica_n11 000356710300502291 core_node12 create 127.0.0.1:39891_solr replacenodetest_coll shard1) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.RecoveryStrategy Finished recovery process. recoveringAfterStartup=true msTimeTaken=1419.0 2> 3562687 INFO (zkCallback-6667-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3562688 INFO (zkCallback-6690-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3562688 INFO (zkCallback-6690-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3562688 INFO (zkCallback-6673-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3562688 INFO (zkCallback-6667-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3562688 INFO (zkCallback-6673-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3562687 INFO (zkCallback-6673-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3562703 INFO (zkCallback-6663-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3562704 INFO (zkCallback-6685-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3562704 INFO (zkCallback-6663-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3562704 INFO (zkCallback-6693-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3562705 INFO (zkCallback-6685-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3562706 INFO (zkCallback-6685-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3562707 INFO (zkCallback-6693-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3562707 INFO (zkCallback-6693-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3562766 INFO (OverseerThreadFactory-6703-thread-2) [] o.a.s.c.a.c.DeleteNodeCmd Deleting replica type=NRT for collection=replacenodetest_coll shard=shard1 on node=127.0.0.1:46533_solr 2> 3562846 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr x:replacenodetest_coll_shard1_replica_n6] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=replacenodetest_coll_shard1_replica_n6&async=000356715774176957&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2&deleteIndex=true} status=0 QTime=33 2> 3562853 INFO (parallelCoreAdminAPIBaseExecutor-6710-thread-1-processing-127.0.0.1:46533_solr replacenodetest_coll_shard1_replica_n6 000356715774176957 unload) [n:127.0.0.1:46533_solr x:replacenodetest_coll_shard1_replica_n6] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replacenodetest_coll.shard1.replica_n6 tag=null 2> 3562875 INFO (qtp782501020-10252) [n:127.0.0.1:46533_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 000356715774176957 2> 3562894 INFO (qtp782501020-10252) [n:127.0.0.1:46533_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=000356715774176957&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=20 2> 3563480 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=000&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=43 2> 3563912 INFO (qtp782501020-10253) [n:127.0.0.1:46533_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 000356715774176957 2> 3563913 INFO (qtp782501020-10253) [n:127.0.0.1:46533_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=000356715774176957&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 3564036 INFO (parallelCoreAdminAPIBaseExecutor-6710-thread-1-processing-127.0.0.1:46533_solr replacenodetest_coll_shard1_replica_n6 000356715774176957 unload) [n:127.0.0.1:46533_solr x:replacenodetest_coll_shard1_replica_n6] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@7d8663ca replacenodetest_coll_shard1_replica_n6 2> 3564036 INFO (parallelCoreAdminAPIBaseExecutor-6710-thread-1-processing-127.0.0.1:46533_solr replacenodetest_coll_shard1_replica_n6 000356715774176957 unload) [n:127.0.0.1:46533_solr x:replacenodetest_coll_shard1_replica_n6] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replacenodetest_coll.shard1.replica_n6 tag=SolrCore@7d8663ca 2> 3564037 INFO (parallelCoreAdminAPIBaseExecutor-6710-thread-1-processing-127.0.0.1:46533_solr replacenodetest_coll_shard1_replica_n6 000356715774176957 unload) [n:127.0.0.1:46533_solr x:replacenodetest_coll_shard1_replica_n6] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replacenodetest_coll.shard1.leader tag=SolrCore@7d8663ca 2> 3564038 INFO (parallelCoreAdminAPIBaseExecutor-6710-thread-1-processing-127.0.0.1:46533_solr replacenodetest_coll_shard1_replica_n6 000356715774176957 unload) [n:127.0.0.1:46533_solr x:replacenodetest_coll_shard1_replica_n6] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 3564127 INFO (parallelCoreAdminAPIBaseExecutor-6710-thread-1-processing-127.0.0.1:46533_solr replacenodetest_coll_shard1_replica_n6 000356715774176957 unload) [n:127.0.0.1:46533_solr] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replacenodetest_coll/terms/shard1 to Terms{values={core_node12=0}, version=2} for removeTerm 2> 3564230 INFO (zkCallback-6673-thread-3) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 3564396 INFO (zkCallback-6663-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3564397 INFO (zkCallback-6663-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3564397 INFO (zkCallback-6693-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3564400 INFO (zkCallback-6693-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3564401 INFO (zkCallback-6693-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3564403 INFO (zkCallback-6685-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3564404 INFO (zkCallback-6685-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3564404 INFO (zkCallback-6685-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3564404 INFO (zkCallback-6667-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3564405 INFO (zkCallback-6667-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3564408 INFO (zkCallback-6673-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3564408 INFO (zkCallback-6673-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3564412 INFO (zkCallback-6690-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3564412 INFO (zkCallback-6690-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3564438 INFO (zkCallback-6673-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3564529 INFO (qtp1839402485-10227) [n:127.0.0.1:33755_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=000&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=36 2> 3564930 INFO (qtp782501020-10256) [n:127.0.0.1:46533_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 000356715774176957 2> 3564932 INFO (qtp782501020-10256) [n:127.0.0.1:46533_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=000356715774176957&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=6 2> 3565056 INFO (OverseerThreadFactory-6703-thread-2) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 3565112 INFO (OverseerCollectionConfigSetProcessor-72080969102262284-127.0.0.1:46533_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 3565576 INFO (qtp1839402485-10233) [n:127.0.0.1:33755_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=000&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=34 2> 3565624 INFO (qtp1839402485-10226) [n:127.0.0.1:33755_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=000&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=43 2> 3565646 INFO (qtp782501020-10249) [n:127.0.0.1:46533_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={action=STATUS&indexInfo=true&wt=javabin&version=2} status=0 QTime=1 2> 3566730 INFO (zkCallback-6673-thread-3) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39891/solr/replacenodetest_coll_shard1_replica_n11/ 2> 3566731 INFO (zkCallback-6673-thread-3) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 3566731 INFO (zkCallback-6673-thread-3) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.SyncStrategy http://127.0.0.1:39891/solr/replacenodetest_coll_shard1_replica_n11/ has no replicas 2> 3566731 INFO (zkCallback-6673-thread-3) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/replacenodetest_coll/leaders/shard1/leader after winning as /collections/replacenodetest_coll/leader_elect/shard1/election/72080969102262283-core_node12-n_0000000001 2> 3566802 INFO (zkCallback-6673-thread-3) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39891/solr/replacenodetest_coll_shard1_replica_n11/ shard1 2> 3566823 INFO (zkCallback-6693-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3566823 INFO (zkCallback-6663-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3566823 INFO (zkCallback-6693-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3566823 INFO (zkCallback-6693-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3566824 INFO (zkCallback-6663-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3566827 INFO (zkCallback-6667-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3566828 INFO (zkCallback-6667-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3566829 INFO (zkCallback-6690-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3566829 INFO (zkCallback-6673-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3566830 INFO (zkCallback-6673-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3566830 INFO (zkCallback-6673-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3566835 INFO (zkCallback-6690-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3570764 INFO (qtp1839402485-10228) [n:127.0.0.1:33755_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=001&sourceNode=127.0.0.1:39891_solr¶llel=true&action=REPLACENODE&waitForFinalState=true&targetNode=127.0.0.1:46533_solr&wt=javabin&version=2} status=0 QTime=67 2> 3570831 INFO (qtp1839402485-10231) [n:127.0.0.1:33755_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=001&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=61 2> 3570844 INFO (OverseerThreadFactory-6703-thread-3) [] o.a.s.c.a.c.ReplaceNodeCmd Going to create replica for collection=replacenodetest_coll shard=shard1 on node=127.0.0.1:46533_solr 2> 3570893 INFO (OverseerThreadFactory-6703-thread-3) [] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:46533_solr for creating new replica of shard shard1 for collection replacenodetest_coll 2> 3570937 INFO (OverseerThreadFactory-6703-thread-3) [] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command. 2> 3570972 INFO (OverseerStateUpdate-72080969102262284-127.0.0.1:46533_solr-n_0000000000) [n:127.0.0.1:46533_solr] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replacenodetest_coll_shard1_replica_n13", 2> "node_name":"127.0.0.1:46533_solr", 2> "base_url":"http://127.0.0.1:46533/solr", 2> "collection":"replacenodetest_coll", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"addreplica"} 2> 3571032 INFO (zkCallback-6693-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571033 INFO (zkCallback-6663-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571033 INFO (zkCallback-6693-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571034 INFO (zkCallback-6673-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571034 INFO (zkCallback-6693-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571034 INFO (zkCallback-6663-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571034 INFO (zkCallback-6673-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571034 INFO (zkCallback-6690-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571034 INFO (zkCallback-6673-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571034 INFO (zkCallback-6690-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571035 INFO (zkCallback-6667-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571035 INFO (zkCallback-6667-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571037 INFO (zkCallback-6685-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571092 INFO (qtp782501020-10253) [n:127.0.0.1:46533_solr x:replacenodetest_coll_shard1_replica_n13] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=001356724034703413&qt=/admin/cores&coreNodeName=core_node14&collection.configName=conf1&name=replacenodetest_coll_shard1_replica_n13&action=CREATE&collection=replacenodetest_coll&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=19 2> 3571093 INFO (parallelCoreAdminAPIBaseExecutor-6710-thread-2-processing-127.0.0.1:46533_solr replacenodetest_coll_shard1_replica_n13 001356724034703413 create) [n:127.0.0.1:46533_solr x:replacenodetest_coll_shard1_replica_n13] o.a.s.h.a.CoreAdminOperation core create command async=001356724034703413&qt=/admin/cores&coreNodeName=core_node14&collection.configName=conf1&name=replacenodetest_coll_shard1_replica_n13&action=CREATE&collection=replacenodetest_coll&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 3571120 INFO (qtp782501020-10256) [n:127.0.0.1:46533_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 001356724034703413 2> 3571121 INFO (qtp782501020-10256) [n:127.0.0.1:46533_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=001356724034703413&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=5 2> 3571404 INFO (zkCallback-6690-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571404 INFO (zkCallback-6673-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571404 INFO (zkCallback-6690-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571404 INFO (zkCallback-6667-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571405 INFO (zkCallback-6673-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571405 INFO (zkCallback-6667-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571405 INFO (zkCallback-6673-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571409 INFO (zkCallback-6693-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571409 INFO (zkCallback-6693-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571409 INFO (zkCallback-6693-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571409 INFO (zkCallback-6663-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571410 INFO (zkCallback-6663-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3571529 INFO (parallelCoreAdminAPIBaseExecutor-6710-thread-2-processing-127.0.0.1:46533_solr replacenodetest_coll_shard1_replica_n13 001356724034703413 create) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 3571544 INFO (parallelCoreAdminAPIBaseExecutor-6710-thread-2-processing-127.0.0.1:46533_solr replacenodetest_coll_shard1_replica_n13 001356724034703413 create) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.s.IndexSchema Schema name=cloud-dynamic 2> 3571628 INFO (parallelCoreAdminAPIBaseExecutor-6710-thread-2-processing-127.0.0.1:46533_solr replacenodetest_coll_shard1_replica_n13 001356724034703413 create) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id 2> 3571628 INFO (parallelCoreAdminAPIBaseExecutor-6710-thread-2-processing-127.0.0.1:46533_solr replacenodetest_coll_shard1_replica_n13 001356724034703413 create) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_shard1_replica_n13' using configuration from configset conf1, trusted=true 2> 3571643 INFO (parallelCoreAdminAPIBaseExecutor-6710-thread-2-processing-127.0.0.1:46533_solr replacenodetest_coll_shard1_replica_n13 001356724034703413 create) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node6/replacenodetest_coll_shard1_replica_n13], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-002/node6/replacenodetest_coll_shard1_replica_n13/data/] 2> 3572005 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=001&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=162 2> 3572127 INFO (qtp782501020-10242) [n:127.0.0.1:46533_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 001356724034703413 2> 3572131 INFO (qtp782501020-10242) [n:127.0.0.1:46533_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=001356724034703413&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=5 2> 3572639 INFO (parallelCoreAdminAPIBaseExecutor-6710-thread-2-processing-127.0.0.1:46533_solr replacenodetest_coll_shard1_replica_n13 001356724034703413 create) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 3572640 INFO (parallelCoreAdminAPIBaseExecutor-6710-thread-2-processing-127.0.0.1:46533_solr replacenodetest_coll_shard1_replica_n13 001356724034703413 create) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 3572688 INFO (parallelCoreAdminAPIBaseExecutor-6710-thread-2-processing-127.0.0.1:46533_solr replacenodetest_coll_shard1_replica_n13 001356724034703413 create) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 3572688 INFO (parallelCoreAdminAPIBaseExecutor-6710-thread-2-processing-127.0.0.1:46533_solr replacenodetest_coll_shard1_replica_n13 001356724034703413 create) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 3572785 INFO (parallelCoreAdminAPIBaseExecutor-6710-thread-2-processing-127.0.0.1:46533_solr replacenodetest_coll_shard1_replica_n13 001356724034703413 create) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 3572801 INFO (parallelCoreAdminAPIBaseExecutor-6710-thread-2-processing-127.0.0.1:46533_solr replacenodetest_coll_shard1_replica_n13 001356724034703413 create) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 3572807 INFO (parallelCoreAdminAPIBaseExecutor-6710-thread-2-processing-127.0.0.1:46533_solr replacenodetest_coll_shard1_replica_n13 001356724034703413 create) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 3572810 INFO (parallelCoreAdminAPIBaseExecutor-6710-thread-2-processing-127.0.0.1:46533_solr replacenodetest_coll_shard1_replica_n13 001356724034703413 create) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723471558180864 2> 3572885 INFO (searcherExecutor-6766-thread-1-processing-replacenodetest_coll_shard1_replica_n13 001356724034703413 core_node14 create 127.0.0.1:46533_solr replacenodetest_coll shard1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 3573017 INFO (parallelCoreAdminAPIBaseExecutor-6710-thread-2-processing-127.0.0.1:46533_solr replacenodetest_coll_shard1_replica_n13 001356724034703413 create) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replacenodetest_coll/terms/shard1 to Terms{values={core_node14=0, core_node12=0}, version=3} for registerTerm 2> 3573017 INFO (parallelCoreAdminAPIBaseExecutor-6710-thread-2-processing-127.0.0.1:46533_solr replacenodetest_coll_shard1_replica_n13 001356724034703413 create) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replacenodetest_coll/leaders/shard1 2> 3573068 INFO (qtp1839402485-10227) [n:127.0.0.1:33755_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=001&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=45 2> 3573163 INFO (qtp782501020-10249) [n:127.0.0.1:46533_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 001356724034703413 2> 3573164 INFO (qtp782501020-10249) [n:127.0.0.1:46533_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=001356724034703413&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 3573192 INFO (parallelCoreAdminAPIBaseExecutor-6710-thread-2-processing-127.0.0.1:46533_solr replacenodetest_coll_shard1_replica_n13 001356724034703413 create) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.ZkController Core needs to recover:replacenodetest_coll_shard1_replica_n13 2> 3573193 INFO (updateExecutor-6675-thread-3-processing-replacenodetest_coll_shard1_replica_n13 001356724034703413 core_node14 create 127.0.0.1:46533_solr replacenodetest_coll shard1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.DefaultSolrCoreState Running recovery 2> 3573200 INFO (recoveryExecutor-6684-thread-1-processing-replacenodetest_coll_shard1_replica_n13 001356724034703413 core_node14 create 127.0.0.1:46533_solr replacenodetest_coll shard1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true 2> 3573211 INFO (recoveryExecutor-6684-thread-1-processing-replacenodetest_coll_shard1_replica_n13 001356724034703413 core_node14 create 127.0.0.1:46533_solr replacenodetest_coll shard1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy startupVersions is empty 2> 3573260 INFO (qtp1863775278-10203) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.S.Request webapp=/solr path=/admin/ping params={wt=javabin&version=2} hits=0 status=0 QTime=11 2> 3573260 INFO (qtp1863775278-10203) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.S.Request webapp=/solr path=/admin/ping params={wt=javabin&version=2} status=0 QTime=11 2> 3573266 INFO (recoveryExecutor-6684-thread-1-processing-replacenodetest_coll_shard1_replica_n13 001356724034703413 core_node14 create 127.0.0.1:46533_solr replacenodetest_coll shard1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[replacenodetest_coll_shard1_replica_n13] 2> 3573275 INFO (recoveryExecutor-6684-thread-1-processing-replacenodetest_coll_shard1_replica_n13 001356724034703413 core_node14 create 127.0.0.1:46533_solr replacenodetest_coll shard1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null} 2> 3573284 INFO (recoveryExecutor-6684-thread-1-processing-replacenodetest_coll_shard1_replica_n13 001356724034703413 core_node14 create 127.0.0.1:46533_solr replacenodetest_coll shard1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy Publishing state of core [replacenodetest_coll_shard1_replica_n13] as recovering, leader is [http://127.0.0.1:39891/solr/replacenodetest_coll_shard1_replica_n11/] and I am [http://127.0.0.1:46533/solr/replacenodetest_coll_shard1_replica_n13/] 2> 3573313 INFO (recoveryExecutor-6684-thread-1-processing-replacenodetest_coll_shard1_replica_n13 001356724034703413 core_node14 create 127.0.0.1:46533_solr replacenodetest_coll shard1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:39891/solr]; [WaitForState: action=PREPRECOVERY&core=replacenodetest_coll_shard1_replica_n11&nodeName=127.0.0.1:46533_solr&coreNodeName=core_node14&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true] 2> 3573323 INFO (qtp1863775278-10204) [n:127.0.0.1:39891_solr x:replacenodetest_coll_shard1_replica_n11] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node14, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true 2> 3573327 INFO (qtp1863775278-10204) [n:127.0.0.1:39891_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=replacenodetest_coll, shard=shard1, thisCore=replacenodetest_coll_shard1_replica_n11, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:46533_solr, coreNodeName=core_node14, onlyIfActiveCheckResult=false, nodeProps: core_node14:{ 2> "core":"replacenodetest_coll_shard1_replica_n13", 2> "node_name":"127.0.0.1:46533_solr", 2> "base_url":"http://127.0.0.1:46533/solr", 2> "state":"down", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 3573341 INFO (qtp1863775278-10204) [n:127.0.0.1:39891_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=replacenodetest_coll, shard=shard1, thisCore=replacenodetest_coll_shard1_replica_n11, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:46533_solr, coreNodeName=core_node14, onlyIfActiveCheckResult=false, nodeProps: core_node14:{ 2> "core":"replacenodetest_coll_shard1_replica_n13", 2> "node_name":"127.0.0.1:46533_solr", 2> "base_url":"http://127.0.0.1:46533/solr", 2> "state":"down", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 3573342 INFO (qtp1863775278-10204) [n:127.0.0.1:39891_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=replacenodetest_coll, shard=shard1, thisCore=replacenodetest_coll_shard1_replica_n11, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:46533_solr, coreNodeName=core_node14, onlyIfActiveCheckResult=false, nodeProps: core_node14:{ 2> "core":"replacenodetest_coll_shard1_replica_n13", 2> "node_name":"127.0.0.1:46533_solr", 2> "base_url":"http://127.0.0.1:46533/solr", 2> "state":"down", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 3573481 INFO (zkCallback-6673-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3573481 INFO (zkCallback-6673-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3573481 INFO (zkCallback-6673-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3573484 INFO (zkCallback-6667-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3573484 INFO (zkCallback-6690-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3573485 INFO (zkCallback-6667-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3573485 INFO (zkCallback-6690-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3573486 INFO (zkCallback-6663-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3573486 INFO (zkCallback-6663-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3573487 INFO (zkCallback-6693-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3573487 INFO (zkCallback-6693-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3573488 INFO (zkCallback-6693-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3573525 INFO (zkCallback-6685-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3573525 INFO (zkCallback-6685-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3573525 INFO (zkCallback-6685-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3573684 INFO (watches-6682-thread-2) [n:127.0.0.1:39891_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=replacenodetest_coll, shard=shard1, thisCore=replacenodetest_coll_shard1_replica_n11, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:46533_solr, coreNodeName=core_node14, onlyIfActiveCheckResult=false, nodeProps: core_node14:{ 2> "core":"replacenodetest_coll_shard1_replica_n13", 2> "node_name":"127.0.0.1:46533_solr", 2> "base_url":"http://127.0.0.1:46533/solr", 2> "state":"recovering", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 3573694 INFO (qtp1863775278-10204) [n:127.0.0.1:39891_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:46533_solr&onlyIfLeaderActive=true&core=replacenodetest_coll_shard1_replica_n11&coreNodeName=core_node14&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=372 2> 3573695 INFO (watches-6682-thread-3) [n:127.0.0.1:39891_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=replacenodetest_coll, shard=shard1, thisCore=replacenodetest_coll_shard1_replica_n11, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:46533_solr, coreNodeName=core_node14, onlyIfActiveCheckResult=false, nodeProps: core_node14:{ 2> "core":"replacenodetest_coll_shard1_replica_n13", 2> "node_name":"127.0.0.1:46533_solr", 2> "base_url":"http://127.0.0.1:46533/solr", 2> "state":"recovering", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 3574132 INFO (qtp1839402485-10233) [n:127.0.0.1:33755_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=001&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=55 2> 3574187 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 001356724034703413 2> 3574187 INFO (qtp782501020-10250) [n:127.0.0.1:46533_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=001356724034703413&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 3574199 INFO (recoveryExecutor-6684-thread-1-processing-replacenodetest_coll_shard1_replica_n13 001356724034703413 core_node14 create 127.0.0.1:46533_solr replacenodetest_coll shard1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:39891/solr/replacenodetest_coll_shard1_replica_n11/] - recoveringAfterStartup=[true] 2> 3574203 WARN (recoveryExecutor-6684-thread-1-processing-replacenodetest_coll_shard1_replica_n13 001356724034703413 core_node14 create 127.0.0.1:46533_solr replacenodetest_coll shard1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.PeerSyncWithLeader no frame of reference to tell if we've missed updates 2> 3574203 INFO (recoveryExecutor-6684-thread-1-processing-replacenodetest_coll_shard1_replica_n13 001356724034703413 core_node14 create 127.0.0.1:46533_solr replacenodetest_coll shard1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy PeerSync Recovery was not successful - trying replication. 2> 3574219 INFO (recoveryExecutor-6684-thread-1-processing-replacenodetest_coll_shard1_replica_n13 001356724034703413 core_node14 create 127.0.0.1:46533_solr replacenodetest_coll shard1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy Starting Replication Recovery. 2> 3574220 INFO (recoveryExecutor-6684-thread-1-processing-replacenodetest_coll_shard1_replica_n13 001356724034703413 core_node14 create 127.0.0.1:46533_solr replacenodetest_coll shard1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy Attempting to replicate from [http://127.0.0.1:39891/solr/replacenodetest_coll_shard1_replica_n11/]. 2> 3574443 INFO (qtp774898083-10244) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:39891/solr/replacenodetest_coll_shard1_replica_n11/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 28 2> 3574443 INFO (qtp1178207868-10216) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:39891/solr/replacenodetest_coll_shard1_replica_n11/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 26 2> 3574479 INFO (qtp1839402485-10231) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:39891/solr/replacenodetest_coll_shard1_replica_n11/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 10 2> 3574499 INFO (qtp101036055-10221) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:39891/solr/replacenodetest_coll_shard1_replica_n11/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 8 2> 3574569 INFO (qtp782501020-10256) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.p.DistributedUpdateProcessor Ignoring commit while not ACTIVE - state: BUFFERING replay: false 2> 3574570 INFO (qtp782501020-10256) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:39891/solr/replacenodetest_coll_shard1_replica_n11/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 13 2> 3574573 INFO (qtp1863775278-10206) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 340 2> 3574586 INFO (qtp1863775278-10208) [n:127.0.0.1:39891_solr c:replacenodetest_coll s:shard1 r:core_node12 x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 3574594 INFO (recoveryExecutor-6684-thread-1-processing-replacenodetest_coll_shard1_replica_n13 001356724034703413 core_node14 create 127.0.0.1:46533_solr replacenodetest_coll shard1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.h.IndexFetcher Leader's generation: 1 2> 3574595 INFO (recoveryExecutor-6684-thread-1-processing-replacenodetest_coll_shard1_replica_n13 001356724034703413 core_node14 create 127.0.0.1:46533_solr replacenodetest_coll shard1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.h.IndexFetcher Leader's version: 0 2> 3574598 INFO (recoveryExecutor-6684-thread-1-processing-replacenodetest_coll_shard1_replica_n13 001356724034703413 core_node14 create 127.0.0.1:46533_solr replacenodetest_coll shard1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.h.IndexFetcher Follower's generation: 1 2> 3574599 INFO (recoveryExecutor-6684-thread-1-processing-replacenodetest_coll_shard1_replica_n13 001356724034703413 core_node14 create 127.0.0.1:46533_solr replacenodetest_coll shard1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.h.IndexFetcher Follower's version: 0 2> 3574600 INFO (recoveryExecutor-6684-thread-1-processing-replacenodetest_coll_shard1_replica_n13 001356724034703413 core_node14 create 127.0.0.1:46533_solr replacenodetest_coll shard1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy No replay needed. 2> 3574611 INFO (recoveryExecutor-6684-thread-1-processing-replacenodetest_coll_shard1_replica_n13 001356724034703413 core_node14 create 127.0.0.1:46533_solr replacenodetest_coll shard1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy Replication Recovery was successful. 2> 3574612 INFO (recoveryExecutor-6684-thread-1-processing-replacenodetest_coll_shard1_replica_n13 001356724034703413 core_node14 create 127.0.0.1:46533_solr replacenodetest_coll shard1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy Registering as Active after recovery. 2> 3574635 INFO (recoveryExecutor-6684-thread-1-processing-replacenodetest_coll_shard1_replica_n13 001356724034703413 core_node14 create 127.0.0.1:46533_solr replacenodetest_coll shard1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy Updating version bucket highest from index after successful recovery. 2> 3574635 INFO (recoveryExecutor-6684-thread-1-processing-replacenodetest_coll_shard1_replica_n13 001356724034703413 core_node14 create 127.0.0.1:46533_solr replacenodetest_coll shard1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723473471832064 2> 3574647 INFO (recoveryExecutor-6684-thread-1-processing-replacenodetest_coll_shard1_replica_n13 001356724034703413 core_node14 create 127.0.0.1:46533_solr replacenodetest_coll shard1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy Finished recovery process, successful=[true] msTimeTaken=1437.0 2> 3574653 INFO (recoveryExecutor-6684-thread-1-processing-replacenodetest_coll_shard1_replica_n13 001356724034703413 core_node14 create 127.0.0.1:46533_solr replacenodetest_coll shard1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy Finished recovery process. recoveringAfterStartup=true msTimeTaken=1443.0 2> 3574789 INFO (zkCallback-6685-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3574789 INFO (zkCallback-6685-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3574790 INFO (zkCallback-6685-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3574790 INFO (zkCallback-6663-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3574790 INFO (zkCallback-6693-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3574790 INFO (zkCallback-6693-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3574791 INFO (zkCallback-6693-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3574790 INFO (zkCallback-6663-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3574816 INFO (zkCallback-6690-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3574818 INFO (zkCallback-6673-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3574819 INFO (zkCallback-6673-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3574822 INFO (zkCallback-6690-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3574826 INFO (zkCallback-6667-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3574827 INFO (zkCallback-6667-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3574832 INFO (zkCallback-6673-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3574861 INFO (OverseerThreadFactory-6703-thread-3) [] o.a.s.c.a.c.DeleteNodeCmd Deleting replica type=NRT for collection=replacenodetest_coll shard=shard1 on node=127.0.0.1:39891_solr 2> 3574984 INFO (qtp1863775278-10207) [n:127.0.0.1:39891_solr x:replacenodetest_coll_shard1_replica_n11] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=replacenodetest_coll_shard1_replica_n11&async=001356727916433071&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2&deleteIndex=true} status=0 QTime=25 2> 3574997 INFO (qtp1863775278-10209) [n:127.0.0.1:39891_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 001356727916433071 2> 3574998 INFO (qtp1863775278-10209) [n:127.0.0.1:39891_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=001356727916433071&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 3575000 INFO (parallelCoreAdminAPIBaseExecutor-6708-thread-2-processing-127.0.0.1:39891_solr replacenodetest_coll_shard1_replica_n11 001356727916433071 unload) [n:127.0.0.1:39891_solr x:replacenodetest_coll_shard1_replica_n11] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replacenodetest_coll.shard1.replica_n11 tag=null 2> 3575185 INFO (qtp1839402485-10232) [n:127.0.0.1:33755_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=001&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=40 2> 3575879 INFO (parallelCoreAdminAPIBaseExecutor-6708-thread-2-processing-127.0.0.1:39891_solr replacenodetest_coll_shard1_replica_n11 001356727916433071 unload) [n:127.0.0.1:39891_solr x:replacenodetest_coll_shard1_replica_n11] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@20c56741 replacenodetest_coll_shard1_replica_n11 2> 3575880 INFO (parallelCoreAdminAPIBaseExecutor-6708-thread-2-processing-127.0.0.1:39891_solr replacenodetest_coll_shard1_replica_n11 001356727916433071 unload) [n:127.0.0.1:39891_solr x:replacenodetest_coll_shard1_replica_n11] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replacenodetest_coll.shard1.replica_n11 tag=SolrCore@20c56741 2> 3575880 INFO (parallelCoreAdminAPIBaseExecutor-6708-thread-2-processing-127.0.0.1:39891_solr replacenodetest_coll_shard1_replica_n11 001356727916433071 unload) [n:127.0.0.1:39891_solr x:replacenodetest_coll_shard1_replica_n11] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replacenodetest_coll.shard1.leader tag=SolrCore@20c56741 2> 3575881 INFO (parallelCoreAdminAPIBaseExecutor-6708-thread-2-processing-127.0.0.1:39891_solr replacenodetest_coll_shard1_replica_n11 001356727916433071 unload) [n:127.0.0.1:39891_solr x:replacenodetest_coll_shard1_replica_n11] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 3575936 INFO (parallelCoreAdminAPIBaseExecutor-6708-thread-2-processing-127.0.0.1:39891_solr replacenodetest_coll_shard1_replica_n11 001356727916433071 unload) [n:127.0.0.1:39891_solr] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replacenodetest_coll/terms/shard1 to Terms{values={core_node14=0}, version=4} for removeTerm 2> 3576020 INFO (qtp1863775278-10203) [n:127.0.0.1:39891_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 001356727916433071 2> 3576020 INFO (qtp1863775278-10203) [n:127.0.0.1:39891_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=001356727916433071&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 3576053 INFO (zkCallback-6685-thread-2) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 3576206 INFO (zkCallback-6663-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3576207 INFO (zkCallback-6663-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3576207 INFO (zkCallback-6693-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3576207 INFO (zkCallback-6693-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3576208 INFO (zkCallback-6685-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3576208 INFO (zkCallback-6685-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3576209 INFO (zkCallback-6685-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3576212 INFO (zkCallback-6673-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3576212 INFO (zkCallback-6667-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3576213 INFO (zkCallback-6667-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3576213 INFO (zkCallback-6690-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3576207 INFO (zkCallback-6693-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3576216 INFO (zkCallback-6673-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3576216 INFO (zkCallback-6673-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3576216 INFO (zkCallback-6690-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3576252 INFO (qtp1839402485-10227) [n:127.0.0.1:33755_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=001&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=58 2> 3576341 INFO (OverseerThreadFactory-6703-thread-3) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000003 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 3576957 INFO (OverseerCollectionConfigSetProcessor-72080969102262284-127.0.0.1:46533_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000003 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 3577285 INFO (qtp1839402485-10233) [n:127.0.0.1:33755_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=001&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=20 2> 3577376 INFO (qtp1839402485-10226) [n:127.0.0.1:33755_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=001&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=78 2> 3577406 INFO (qtp1863775278-10206) [n:127.0.0.1:39891_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={action=STATUS&indexInfo=true&wt=javabin&version=2} status=0 QTime=1 2> 3577462 INFO (TEST-ReplaceNodeTest.test-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Ending test 2> 3578554 INFO (zkCallback-6685-thread-2) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:46533/solr/replacenodetest_coll_shard1_replica_n13/ 2> 3578554 INFO (zkCallback-6685-thread-2) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 3578555 INFO (zkCallback-6685-thread-2) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.SyncStrategy http://127.0.0.1:46533/solr/replacenodetest_coll_shard1_replica_n13/ has no replicas 2> 3578555 INFO (zkCallback-6685-thread-2) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/replacenodetest_coll/leaders/shard1/leader after winning as /collections/replacenodetest_coll/leader_elect/shard1/election/72080969102262284-core_node14-n_0000000002 2> 3578638 INFO (zkCallback-6685-thread-2) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:46533/solr/replacenodetest_coll_shard1_replica_n13/ shard1 2> 3578707 INFO (zkCallback-6693-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3578708 INFO (zkCallback-6663-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3578708 INFO (zkCallback-6693-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3578708 INFO (zkCallback-6693-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3578708 INFO (zkCallback-6663-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3578711 INFO (zkCallback-6685-thread-5) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3578714 INFO (zkCallback-6685-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3578717 INFO (zkCallback-6685-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3578747 INFO (zkCallback-6667-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3578748 INFO (zkCallback-6667-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3578750 INFO (zkCallback-6690-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3578751 INFO (zkCallback-6690-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [6]) 2> 3579787 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Starting testGoodSpreadDuringAssignWithNoTarget 2> 3579940 INFO (jetty-closer-6772-thread-1) [] o.e.j.s.Server Stopped Server@787c0d27{STOPPING}[10.0.13,sto=0] 2> 3579947 INFO (jetty-closer-6772-thread-2) [] o.e.j.s.Server Stopped Server@50e12401{STOPPING}[10.0.13,sto=0] 2> 3579952 INFO (jetty-closer-6772-thread-1) [] o.e.j.s.AbstractConnector Stopped ServerConnector@8c24ea7{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0} 2> 3579954 INFO (jetty-closer-6772-thread-1) [] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3b84765d{/solr,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,STOPPED} 2> 3579960 INFO (jetty-closer-6772-thread-3) [] o.e.j.s.Server Stopped Server@13b170aa{STOPPING}[10.0.13,sto=0] 2> 3579997 INFO (jetty-closer-6772-thread-3) [] o.e.j.s.AbstractConnector Stopped ServerConnector@6364fe31{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0} 2> 3580001 INFO (jetty-closer-6772-thread-2) [] o.e.j.s.AbstractConnector Stopped ServerConnector@20f7ea15{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0} 2> 3580003 INFO (jetty-closer-6772-thread-2) [] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@48cb1d9b{/solr,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,STOPPED} 2> 3580001 INFO (jetty-closer-6772-thread-3) [] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4c402246{/solr,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,STOPPED} 2> 3580049 INFO (jetty-closer-6772-thread-3) [] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2082359353 2> 3580050 INFO (jetty-closer-6772-thread-3) [] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:33755_solr 2> 3580049 INFO (jetty-closer-6772-thread-2) [] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1756059469 2> 3580050 INFO (jetty-closer-6772-thread-2) [] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:45673_solr 2> 3580069 INFO (jetty-closer-6772-thread-3) [] o.a.s.c.ZkController Publish this node as DOWN... 2> 3580069 INFO (jetty-closer-6772-thread-3) [] o.a.s.c.ZkController Publish node=127.0.0.1:33755_solr as DOWN 2> 3580074 INFO (jetty-closer-6772-thread-2) [] o.a.s.c.ZkController Publish this node as DOWN... 2> 3580075 INFO (jetty-closer-6772-thread-2) [] o.a.s.c.ZkController Publish node=127.0.0.1:45673_solr as DOWN 2> 3580079 INFO (zkCallback-6667-thread-2) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (4) 2> 3580088 INFO (zkCallback-6693-thread-3) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (4) 2> 3580088 INFO (zkCallback-6685-thread-5) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (4) 2> 3580091 INFO (zkCallback-6690-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (4) 2> 3580092 INFO (zkCallback-6673-thread-3) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (4) 2> 3580093 INFO (zkCallback-6663-thread-2) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (4) 2> 3580171 INFO (jetty-closer-6772-thread-6) [] o.e.j.s.Server Stopped Server@50f535ad{STOPPING}[10.0.13,sto=0] 2> 3580172 INFO (jetty-closer-6772-thread-5) [] o.e.j.s.Server Stopped Server@11062e7b{STOPPING}[10.0.13,sto=0] 2> 3580175 INFO (jetty-closer-6772-thread-4) [] o.e.j.s.Server Stopped Server@388344da{STOPPING}[10.0.13,sto=0] 2> 3580226 INFO (jetty-closer-6772-thread-5) [] o.e.j.s.AbstractConnector Stopped ServerConnector@61d107bc{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0} 2> 3580232 INFO (jetty-closer-6772-thread-5) [] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@535b1b74{/solr,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,STOPPED} 2> 3580247 INFO (jetty-closer-6772-thread-6) [] o.e.j.s.AbstractConnector Stopped ServerConnector@2063ff38{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0} 2> 3580249 INFO (jetty-closer-6772-thread-6) [] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@67b6baa8{/solr,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,STOPPED} 2> 3580271 INFO (coreCloseExecutor-6777-thread-1) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@4097a025 replacenodetest_coll_shard5_replica_n1 2> 3580272 INFO (coreCloseExecutor-6777-thread-1) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replacenodetest_coll.shard5.replica_n1 tag=SolrCore@4097a025 2> 3580448 INFO (jetty-closer-6772-thread-4) [] o.e.j.s.AbstractConnector Stopped ServerConnector@48fde0cb{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0} 2> 3580449 INFO (jetty-closer-6772-thread-4) [] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@71793b12{/solr,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,STOPPED} 2> 3580528 INFO (jetty-closer-6772-thread-5) [] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1207874612 2> 3580529 INFO (jetty-closer-6772-thread-5) [] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:39891_solr 2> 3580531 INFO (jetty-closer-6772-thread-6) [] o.a.s.c.CoreContainer Shutting down CoreContainer instance=708130848 2> 3580532 INFO (jetty-closer-6772-thread-6) [] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:46533_solr 2> 3580532 INFO (jetty-closer-6772-thread-4) [] o.a.s.c.CoreContainer Shutting down CoreContainer instance=993478053 2> 3580533 INFO (jetty-closer-6772-thread-4) [] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:33535_solr 2> 3580556 INFO (jetty-closer-6772-thread-5) [] o.a.s.c.ZkController Publish this node as DOWN... 2> 3580556 INFO (jetty-closer-6772-thread-5) [] o.a.s.c.ZkController Publish node=127.0.0.1:39891_solr as DOWN 2> 3580557 INFO (jetty-closer-6772-thread-4) [] o.a.s.c.ZkController Publish this node as DOWN... 2> 3580557 INFO (jetty-closer-6772-thread-4) [] o.a.s.c.ZkController Publish node=127.0.0.1:33535_solr as DOWN 2> 3580574 INFO (zkCallback-6693-thread-3) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (1) 2> 3580574 INFO (zkCallback-6667-thread-2) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (1) 2> 3580575 INFO (zkCallback-6663-thread-2) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (1) 2> 3580575 INFO (zkCallback-6690-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (1) 2> 3580576 INFO (jetty-closer-6772-thread-6) [] o.a.s.c.ZkController Publish this node as DOWN... 2> 3580576 INFO (jetty-closer-6772-thread-6) [] o.a.s.c.ZkController Publish node=127.0.0.1:46533_solr as DOWN 2> 3580588 INFO (zkCallback-6673-thread-3) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (1) 2> 3580594 INFO (zkCallback-6685-thread-5) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (1) 2> 3580613 INFO (zkCallback-6667-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3580614 INFO (zkCallback-6667-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3580617 INFO (zkCallback-6685-thread-5) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3580617 INFO (zkCallback-6685-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3580619 INFO (zkCallback-6693-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3580619 INFO (zkCallback-6685-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3580620 INFO (zkCallback-6693-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3580620 INFO (zkCallback-6693-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3580620 INFO (zkCallback-6663-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3580621 INFO (zkCallback-6663-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3580619 INFO (coreCloseExecutor-6781-thread-1) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@3da1411e replacenodetest_coll_shard3_replica_n4 2> 3580625 INFO (coreCloseExecutor-6781-thread-1) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replacenodetest_coll.shard3.replica_n4 tag=SolrCore@3da1411e 2> 3580632 INFO (zkCallback-6690-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3580632 INFO (zkCallback-6690-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3580766 INFO (coreCloseExecutor-6784-thread-1) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@73102890 replacenodetest_coll_shard2_replica_n2 2> 3580766 INFO (coreCloseExecutor-6784-thread-1) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replacenodetest_coll.shard2.replica_n2 tag=SolrCore@73102890 2> 3580832 INFO (coreCloseExecutor-6786-thread-1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@40144389 replacenodetest_coll_shard1_replica_n13 2> 3580833 INFO (coreCloseExecutor-6786-thread-1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replacenodetest_coll.shard1.replica_n13 tag=SolrCore@40144389 2> 3580852 INFO (jetty-closer-6772-thread-5) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null 2> 3580937 INFO (zkCallback-6667-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3580937 INFO (zkCallback-6667-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3580940 INFO (zkCallback-6690-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3580943 INFO (zkCallback-6693-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3580943 INFO (zkCallback-6693-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3580940 INFO (zkCallback-6690-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3580944 INFO (zkCallback-6685-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3580944 INFO (zkCallback-6685-thread-5) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3580945 INFO (zkCallback-6693-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3580945 INFO (zkCallback-6663-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3580945 INFO (zkCallback-6663-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3580945 INFO (zkCallback-6685-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3581745 INFO (jetty-closer-6772-thread-1) [] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1133854968 2> 3581745 INFO (jetty-closer-6772-thread-1) [] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:45375_solr 2> 3581795 INFO (jetty-closer-6772-thread-1) [] o.a.s.c.ZkController Publish this node as DOWN... 2> 3581796 INFO (zkCallback-6693-thread-2) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) 2> 3581796 INFO (jetty-closer-6772-thread-1) [] o.a.s.c.ZkController Publish node=127.0.0.1:45375_solr as DOWN 2> 3581802 INFO (zkCallback-6685-thread-3) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) 2> 3581804 INFO (zkCallback-6690-thread-2) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) 2> 3581799 INFO (zkCallback-6667-thread-2) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) 2> 3581813 INFO (zkCallback-6673-thread-3) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) 2> 3581821 INFO (zkCallback-6663-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) 2> 3582084 INFO (coreCloseExecutor-6789-thread-1) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@f6c252c replacenodetest_coll_shard4_replica_n9 2> 3582085 INFO (coreCloseExecutor-6789-thread-1) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replacenodetest_coll.shard4.replica_n9 tag=SolrCore@f6c252c 2> 3582191 INFO (zkCallback-6693-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [0]) 2> 3582192 INFO (zkCallback-6667-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [0]) 2> 3582192 INFO (zkCallback-6667-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [0]) 2> 3582191 INFO (zkCallback-6663-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [0]) 2> 3582196 INFO (zkCallback-6693-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [0]) 2> 3582196 INFO (zkCallback-6663-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [0]) 2> 3582197 INFO (zkCallback-6690-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [0]) 2> 3582197 INFO (zkCallback-6690-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [0]) 2> 3582191 INFO (zkCallback-6693-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [0]) 2> 3582212 INFO (zkCallback-6685-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [0]) 2> 3582212 INFO (zkCallback-6685-thread-5) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [0]) 2> 3582220 INFO (zkCallback-6685-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [0]) 2> 3583983 INFO (jetty-closer-6772-thread-5) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null 2> 3585821 INFO (coreCloseExecutor-6777-thread-1) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replacenodetest_coll.shard5.leader tag=SolrCore@4097a025 2> 3585850 INFO (coreCloseExecutor-6777-thread-1) [n:127.0.0.1:45673_solr c:replacenodetest_coll s:shard5 r:core_node3 x:replacenodetest_coll_shard5_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 3585946 INFO (jetty-closer-6772-thread-2) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null 2> 3586736 INFO (jetty-closer-6772-thread-5) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null 2> 3586737 INFO (jetty-closer-6772-thread-5) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null 2> 3587223 INFO (coreCloseExecutor-6781-thread-1) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replacenodetest_coll.shard3.leader tag=SolrCore@3da1411e 2> 3587280 INFO (coreCloseExecutor-6784-thread-1) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replacenodetest_coll.shard2.leader tag=SolrCore@73102890 2> 3587283 INFO (coreCloseExecutor-6781-thread-1) [n:127.0.0.1:33755_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n4] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 3587287 INFO (coreCloseExecutor-6786-thread-1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replacenodetest_coll.shard1.leader tag=SolrCore@40144389 2> 3587327 INFO (coreCloseExecutor-6784-thread-1) [n:127.0.0.1:33535_solr c:replacenodetest_coll s:shard2 r:core_node5 x:replacenodetest_coll_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 3587427 INFO (jetty-closer-6772-thread-3) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null 2> 3587451 INFO (coreCloseExecutor-6786-thread-1) [n:127.0.0.1:46533_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 3587528 INFO (jetty-closer-6772-thread-4) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null 2> 3587624 INFO (jetty-closer-6772-thread-6) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null 2> 3589439 INFO (coreCloseExecutor-6789-thread-1) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replacenodetest_coll.shard4.leader tag=SolrCore@f6c252c 2> 3589446 INFO (coreCloseExecutor-6789-thread-1) [n:127.0.0.1:45375_solr c:replacenodetest_coll s:shard4 r:core_node10 x:replacenodetest_coll_shard4_replica_n9] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 3589511 INFO (jetty-closer-6772-thread-1) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null 2> 3589709 INFO (jetty-closer-6772-thread-2) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null 2> 3590846 INFO (jetty-closer-6772-thread-3) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null 2> 3590940 INFO (jetty-closer-6772-thread-2) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null 2> 3590941 INFO (jetty-closer-6772-thread-2) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null 2> 3591058 INFO (jetty-closer-6772-thread-4) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null 2> 3591129 INFO (jetty-closer-6772-thread-6) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null 2> 3591211 INFO (jetty-closer-6772-thread-1) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null 2> 3591438 INFO (jetty-closer-6772-thread-3) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null 2> 3591444 INFO (jetty-closer-6772-thread-3) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null 2> 3591579 INFO (jetty-closer-6772-thread-6) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null 2> 3591581 INFO (jetty-closer-6772-thread-6) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null 2> 3591623 INFO (closeThreadPool-6793-thread-2) [] o.a.s.c.Overseer Overseer (id=72080969102262284-127.0.0.1:46533_solr-n_0000000000) closing 2> 3591625 INFO (OverseerStateUpdate-72080969102262284-127.0.0.1:46533_solr-n_0000000000) [n:127.0.0.1:46533_solr] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:46533_solr 2> 3591688 INFO (jetty-closer-6772-thread-1) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null 2> 3591689 INFO (jetty-closer-6772-thread-1) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null 2> 3591699 INFO (jetty-closer-6772-thread-4) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null 2> 3591722 INFO (jetty-closer-6772-thread-4) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null 2> 3591844 INFO (jetty-closer-6772-thread-6) [] o.a.s.c.Overseer Overseer (id=72080969102262284-127.0.0.1:46533_solr-n_0000000000) closing 2> 3591885 INFO (closeThreadPool-6793-thread-1) [] o.a.s.c.Overseer Overseer (id=72080969102262284-127.0.0.1:46533_solr-n_0000000000) closing 2> 3592108 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkTestServer Shutting down ZkTestServer. 2> 3592356 WARN (ZkTestServer Run Thread) [] o.a.s.c.ZkTestServer Watch limit violations: 2> Maximum concurrent create/delete watches above limit: 2> 2> 11 /solr/collections/replacenodetest_coll/terms/shard1 2> 7 /solr/configs/conf1 2> 7 /solr/clusterprops.json 2> 7 /solr/aliases.json 2> 6 /solr/packages.json 2> 6 /solr/security.json 2> 6 /solr/collections/replacenodetest_coll/collectionprops.json 2> 2 /solr/collections/replacenodetest_coll/terms/shard3 2> 2 /solr/collections/replacenodetest_coll/terms/shard2 2> 2 /solr/collections/replacenodetest_coll/terms/shard5 2> 2 /solr/collections/replacenodetest_coll/terms/shard4 2> 2> Maximum concurrent data watches above limit: 2> 2> 262 /solr/collections/replacenodetest_coll/state.json 2> 2> Maximum concurrent children watches above limit: 2> 2> 55 /solr/overseer/queue 2> 49 /solr/live_nodes 2> 39 /solr/collections/replacenodetest_coll/state.json 2> 14 /solr/collections 2> 8 /solr/overseer/collection-queue-work 2> 2> 3592512 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003 2> 3592514 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER 2> 3592518 INFO (ZkTestServer Run Thread) [] o.a.s.c.ZkTestServer client port: 0.0.0.0/0.0.0.0:0 2> 3592519 INFO (ZkTestServer Run Thread) [] o.a.s.c.ZkTestServer Starting server 2> 3592531 WARN (ZkTestServer Run Thread) [] o.a.z.s.ServerCnxnFactory maxCnxns is not configured, using default value 0. 2> 3592620 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkTestServer start zk server on port: 34921 2> 3592973 WARN (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3593027 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3593091 INFO (zkConnectionManagerCallback-6797-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3593091 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3593093 WARN (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3593130 WARN (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3593204 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3593236 INFO (zkConnectionManagerCallback-6799-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3593237 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3593237 WARN (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3593238 WARN (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3593378 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3593436 INFO (zkConnectionManagerCallback-6801-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3593488 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3593489 WARN (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3593712 INFO (jetty-launcher-6802-thread-1) [] o.a.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0) 2> 3593714 INFO (jetty-launcher-6802-thread-1) [] o.a.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ... 2> 3593715 INFO (jetty-launcher-6802-thread-1) [] o.e.j.s.Server jetty-10.0.13; built: 2022-12-07T20:13:20.134Z; git: 1c2636ea05c0ca8de1ffd6ca7f3a98ac084c766d; jvm 11.0.20+8 2> 3593737 INFO (jetty-launcher-6802-thread-2) [] o.a.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0) 2> 3593737 INFO (jetty-launcher-6802-thread-2) [] o.a.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ... 2> 3593738 INFO (jetty-launcher-6802-thread-2) [] o.e.j.s.Server jetty-10.0.13; built: 2022-12-07T20:13:20.134Z; git: 1c2636ea05c0ca8de1ffd6ca7f3a98ac084c766d; jvm 11.0.20+8 2> 3593742 INFO (jetty-launcher-6802-thread-3) [] o.a.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0) 2> 3593743 INFO (jetty-launcher-6802-thread-3) [] o.a.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ... 2> 3593743 INFO (jetty-launcher-6802-thread-3) [] o.e.j.s.Server jetty-10.0.13; built: 2022-12-07T20:13:20.134Z; git: 1c2636ea05c0ca8de1ffd6ca7f3a98ac084c766d; jvm 11.0.20+8 2> 3593748 INFO (jetty-launcher-6802-thread-4) [] o.a.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0) 2> 3593748 INFO (jetty-launcher-6802-thread-4) [] o.a.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ... 2> 3593749 INFO (jetty-launcher-6802-thread-4) [] o.e.j.s.Server jetty-10.0.13; built: 2022-12-07T20:13:20.134Z; git: 1c2636ea05c0ca8de1ffd6ca7f3a98ac084c766d; jvm 11.0.20+8 2> 3593910 INFO (jetty-launcher-6802-thread-1) [] o.e.j.s.s.DefaultSessionIdManager Session workerName=node0 2> 3593930 INFO (jetty-launcher-6802-thread-1) [] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@21eaa628{/solr,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,AVAILABLE} 2> 3593934 INFO (jetty-launcher-6802-thread-1) [] o.e.j.s.AbstractConnector Started ServerConnector@d62b39a{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:35735} 2> 3593936 INFO (jetty-launcher-6802-thread-1) [] o.e.j.s.Server Started Server@7dd95547{STARTING}[10.0.13,sto=0] @3594283ms 2> 3593939 ERROR (jetty-launcher-6802-thread-1) [] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. 2> 3593940 INFO (jetty-launcher-6802-thread-1) [] o.a.s.s.CoreContainerProvider Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory 2> 3593942 INFO (jetty-launcher-6802-thread-1) [] o.a.s.s.CoreContainerProvider ___ _ Welcome to Apache Solr™ version 9.2.1 2> 3593944 INFO (jetty-launcher-6802-thread-1) [] o.a.s.s.CoreContainerProvider / __| ___| |_ _ Starting in cloud mode on port null 2> 3593949 INFO (jetty-launcher-6802-thread-1) [] o.a.s.s.CoreContainerProvider \__ \/ _ \ | '_| Install dir: /build/solr/src/solr-9.2.1/solr 2> 3593962 INFO (jetty-launcher-6802-thread-1) [] o.a.s.s.CoreContainerProvider |___/\___/_|_| Start time: 2023-10-03T08:48:31.966813Z 2> 3593965 INFO (jetty-launcher-6802-thread-1) [] o.a.s.s.CoreContainerProvider Solr Home: /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node1 (source: servlet config: solr.solr.home) 2> 3593977 WARN (jetty-launcher-6802-thread-1) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3594042 INFO (jetty-launcher-6802-thread-3) [] o.e.j.s.s.DefaultSessionIdManager Session workerName=node0 2> 3594061 INFO (jetty-launcher-6802-thread-3) [] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@24027d94{/solr,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,AVAILABLE} 2> 3594068 INFO (jetty-launcher-6802-thread-3) [] o.e.j.s.AbstractConnector Started ServerConnector@4dca7d85{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:36667} 2> 3594069 INFO (jetty-launcher-6802-thread-3) [] o.e.j.s.Server Started Server@3db1fb2e{STARTING}[10.0.13,sto=0] @3594415ms 2> 3594071 ERROR (jetty-launcher-6802-thread-3) [] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. 2> 3594071 INFO (jetty-launcher-6802-thread-3) [] o.a.s.s.CoreContainerProvider Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory 2> 3594072 INFO (jetty-launcher-6802-thread-3) [] o.a.s.s.CoreContainerProvider ___ _ Welcome to Apache Solr™ version 9.2.1 2> 3594083 INFO (jetty-launcher-6802-thread-3) [] o.a.s.s.CoreContainerProvider / __| ___| |_ _ Starting in cloud mode on port null 2> 3594083 INFO (jetty-launcher-6802-thread-3) [] o.a.s.s.CoreContainerProvider \__ \/ _ \ | '_| Install dir: /build/solr/src/solr-9.2.1/solr 2> 3594083 INFO (jetty-launcher-6802-thread-3) [] o.a.s.s.CoreContainerProvider |___/\___/_|_| Start time: 2023-10-03T08:48:32.087790Z 2> 3594089 INFO (jetty-launcher-6802-thread-3) [] o.a.s.s.CoreContainerProvider Solr Home: /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node3 (source: servlet config: solr.solr.home) 2> 3594090 WARN (jetty-launcher-6802-thread-3) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3594123 INFO (jetty-launcher-6802-thread-2) [] o.e.j.s.s.DefaultSessionIdManager Session workerName=node0 2> 3594130 INFO (jetty-launcher-6802-thread-2) [] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2fcd53f4{/solr,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,AVAILABLE} 2> 3594137 INFO (jetty-launcher-6802-thread-2) [] o.e.j.s.AbstractConnector Started ServerConnector@14c847c{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:34951} 2> 3594137 INFO (jetty-launcher-6802-thread-2) [] o.e.j.s.Server Started Server@71fb48b1{STARTING}[10.0.13,sto=0] @3594484ms 2> 3594139 ERROR (jetty-launcher-6802-thread-2) [] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. 2> 3594139 INFO (jetty-launcher-6802-thread-2) [] o.a.s.s.CoreContainerProvider Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory 2> 3594140 INFO (jetty-launcher-6802-thread-2) [] o.a.s.s.CoreContainerProvider ___ _ Welcome to Apache Solr™ version 9.2.1 2> 3594140 INFO (jetty-launcher-6802-thread-2) [] o.a.s.s.CoreContainerProvider / __| ___| |_ _ Starting in cloud mode on port null 2> 3594141 INFO (jetty-launcher-6802-thread-2) [] o.a.s.s.CoreContainerProvider \__ \/ _ \ | '_| Install dir: /build/solr/src/solr-9.2.1/solr 2> 3594141 INFO (jetty-launcher-6802-thread-2) [] o.a.s.s.CoreContainerProvider |___/\___/_|_| Start time: 2023-10-03T08:48:32.145237Z 2> 3594142 INFO (jetty-launcher-6802-thread-2) [] o.a.s.s.CoreContainerProvider Solr Home: /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node2 (source: servlet config: solr.solr.home) 2> 3594143 WARN (jetty-launcher-6802-thread-2) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3594169 INFO (jetty-launcher-6802-thread-4) [] o.e.j.s.s.DefaultSessionIdManager Session workerName=node0 2> 3594178 INFO (jetty-launcher-6802-thread-4) [] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@668605fe{/solr,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,AVAILABLE} 2> 3594181 INFO (jetty-launcher-6802-thread-4) [] o.e.j.s.AbstractConnector Started ServerConnector@449c60ba{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:46857} 2> 3594182 INFO (jetty-launcher-6802-thread-4) [] o.e.j.s.Server Started Server@34eea94b{STARTING}[10.0.13,sto=0] @3594528ms 2> 3594184 ERROR (jetty-launcher-6802-thread-4) [] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. 2> 3594184 INFO (jetty-launcher-6802-thread-4) [] o.a.s.s.CoreContainerProvider Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory 2> 3594184 INFO (jetty-launcher-6802-thread-4) [] o.a.s.s.CoreContainerProvider ___ _ Welcome to Apache Solr™ version 9.2.1 2> 3594185 INFO (jetty-launcher-6802-thread-4) [] o.a.s.s.CoreContainerProvider / __| ___| |_ _ Starting in cloud mode on port null 2> 3594185 INFO (jetty-launcher-6802-thread-4) [] o.a.s.s.CoreContainerProvider \__ \/ _ \ | '_| Install dir: /build/solr/src/solr-9.2.1/solr 2> 3594185 INFO (jetty-launcher-6802-thread-4) [] o.a.s.s.CoreContainerProvider |___/\___/_|_| Start time: 2023-10-03T08:48:32.189854Z 2> 3594186 INFO (jetty-launcher-6802-thread-4) [] o.a.s.s.CoreContainerProvider Solr Home: /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node4 (source: servlet config: solr.solr.home) 2> 3594196 WARN (jetty-launcher-6802-thread-4) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3594214 INFO (jetty-launcher-6802-thread-3) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3594257 INFO (jetty-launcher-6802-thread-4) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3594279 INFO (jetty-launcher-6802-thread-1) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3594296 INFO (zkConnectionManagerCallback-6806-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3594302 INFO (jetty-launcher-6802-thread-3) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3594303 WARN (jetty-launcher-6802-thread-3) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3594320 INFO (jetty-launcher-6802-thread-2) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3594355 INFO (zkConnectionManagerCallback-6804-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3594355 INFO (jetty-launcher-6802-thread-1) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3594356 WARN (jetty-launcher-6802-thread-1) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3594392 INFO (zkConnectionManagerCallback-6810-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3594400 INFO (jetty-launcher-6802-thread-4) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3594401 WARN (jetty-launcher-6802-thread-4) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3594403 INFO (jetty-launcher-6802-thread-3) [] o.a.s.c.NodeConfig solr.xml found in ZooKeeper. Loading... 2> 3594410 INFO (zkConnectionManagerCallback-6808-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3594410 INFO (jetty-launcher-6802-thread-2) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3594411 WARN (jetty-launcher-6802-thread-2) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3594429 INFO (jetty-launcher-6802-thread-4) [] o.a.s.c.NodeConfig solr.xml found in ZooKeeper. Loading... 2> 3594435 INFO (jetty-launcher-6802-thread-1) [] o.a.s.c.NodeConfig solr.xml found in ZooKeeper. Loading... 2> 3594476 INFO (jetty-launcher-6802-thread-2) [] o.a.s.c.NodeConfig solr.xml found in ZooKeeper. Loading... 2> 3595001 INFO (jetty-launcher-6802-thread-1) [] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34921/solr 2> 3595001 INFO (jetty-launcher-6802-thread-1) [] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=true. Solr will be using distributed cluster state updates. 2> 3595158 INFO (jetty-launcher-6802-thread-2) [] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34921/solr 2> 3595159 INFO (jetty-launcher-6802-thread-2) [] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=true. Solr will be using distributed cluster state updates. 2> 3595174 INFO (jetty-launcher-6802-thread-1) [n:127.0.0.1:35735_solr] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3595221 INFO (jetty-launcher-6802-thread-4) [] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34921/solr 2> 3595222 INFO (jetty-launcher-6802-thread-4) [] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=true. Solr will be using distributed cluster state updates. 2> 3595230 INFO (jetty-launcher-6802-thread-3) [] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34921/solr 2> 3595237 INFO (zkConnectionManagerCallback-6839-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3595244 INFO (jetty-launcher-6802-thread-1) [n:127.0.0.1:35735_solr] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3595260 INFO (jetty-launcher-6802-thread-3) [] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=true. Solr will be using distributed cluster state updates. 2> 3595266 INFO (jetty-launcher-6802-thread-2) [n:127.0.0.1:34951_solr] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3595317 INFO (jetty-launcher-6802-thread-4) [n:127.0.0.1:46857_solr] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3595342 INFO (jetty-launcher-6802-thread-3) [n:127.0.0.1:36667_solr] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 3595385 INFO (zkConnectionManagerCallback-6844-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3595386 INFO (jetty-launcher-6802-thread-2) [n:127.0.0.1:34951_solr] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3595396 INFO (zkConnectionManagerCallback-6847-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3595398 INFO (jetty-launcher-6802-thread-4) [n:127.0.0.1:46857_solr] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3595415 INFO (zkConnectionManagerCallback-6850-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3595416 INFO (jetty-launcher-6802-thread-3) [n:127.0.0.1:36667_solr] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3596349 WARN (jetty-launcher-6802-thread-2) [n:127.0.0.1:34951_solr] o.a.s.c.ZkController Contents of zookeeper /security.json are world-readable; consider setting up ACLs as described in https://solr.apache.org/guide/solr/latest/deployment-guide/zookeeper-access-control.html 2> 3596360 WARN (jetty-launcher-6802-thread-1) [n:127.0.0.1:35735_solr] o.a.s.c.ZkController Contents of zookeeper /security.json are world-readable; consider setting up ACLs as described in https://solr.apache.org/guide/solr/latest/deployment-guide/zookeeper-access-control.html 2> 3596392 WARN (jetty-launcher-6802-thread-3) [n:127.0.0.1:36667_solr] o.a.s.c.ZkController Contents of zookeeper /security.json are world-readable; consider setting up ACLs as described in https://solr.apache.org/guide/solr/latest/deployment-guide/zookeeper-access-control.html 2> 3596394 WARN (jetty-launcher-6802-thread-4) [n:127.0.0.1:46857_solr] o.a.s.c.ZkController Contents of zookeeper /security.json are world-readable; consider setting up ACLs as described in https://solr.apache.org/guide/solr/latest/deployment-guide/zookeeper-access-control.html 2> 3596501 INFO (jetty-launcher-6802-thread-4) [n:127.0.0.1:46857_solr] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=true. Solr will be using distributed cluster state updates. 2> 3596515 INFO (jetty-launcher-6802-thread-1) [n:127.0.0.1:35735_solr] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=true. Solr will be using distributed cluster state updates. 2> 3596534 INFO (jetty-launcher-6802-thread-2) [n:127.0.0.1:34951_solr] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=true. Solr will be using distributed cluster state updates. 2> 3596614 INFO (jetty-launcher-6802-thread-3) [n:127.0.0.1:36667_solr] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=true. Solr will be using distributed cluster state updates. 2> 3596615 INFO (jetty-launcher-6802-thread-4) [n:127.0.0.1:46857_solr] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46857_solr 2> 3596650 INFO (jetty-launcher-6802-thread-4) [n:127.0.0.1:46857_solr] o.a.s.c.Overseer Overseer (id=72080973716586505-127.0.0.1:46857_solr-n_0000000000) starting 2> 3596661 INFO (jetty-launcher-6802-thread-2) [n:127.0.0.1:34951_solr] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34951_solr 2> 3596691 INFO (jetty-launcher-6802-thread-1) [n:127.0.0.1:35735_solr] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35735_solr 2> 3596746 INFO (zkCallback-6849-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2) 2> 3596747 INFO (zkCallback-6838-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2) 2> 3596764 INFO (zkCallback-6843-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2) 2> 3596793 INFO (jetty-launcher-6802-thread-3) [n:127.0.0.1:36667_solr] o.a.s.c.ZkController Publish node=127.0.0.1:36667_solr as DOWN 2> 3596813 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2) 2> 3596819 INFO (jetty-launcher-6802-thread-3) [n:127.0.0.1:36667_solr] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36667_solr 2> 3596869 INFO (zkCallback-6849-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) 2> 3596883 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) 2> 3596896 INFO (zkCallback-6843-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) 2> 3596908 INFO (zkCallback-6838-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) 2> 3596944 WARN (jetty-launcher-6802-thread-1) [n:127.0.0.1:35735_solr] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info 2> 3596946 WARN (jetty-launcher-6802-thread-2) [n:127.0.0.1:34951_solr] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info 2> 3596964 WARN (jetty-launcher-6802-thread-3) [n:127.0.0.1:36667_solr] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info 2> 3597102 INFO (OverseerStateUpdate-72080973716586505-127.0.0.1:46857_solr-n_0000000000) [n:127.0.0.1:46857_solr] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:46857_solr 2> 3597118 INFO (jetty-launcher-6802-thread-4) [n:127.0.0.1:46857_solr] o.a.s.c.ZkController Publish node=127.0.0.1:46857_solr as DOWN 2> 3597160 INFO (jetty-launcher-6802-thread-4) [n:127.0.0.1:46857_solr] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46857_solr 2> 3597246 INFO (OverseerStateUpdate-72080973716586505-127.0.0.1:46857_solr-n_0000000000) [n:127.0.0.1:46857_solr] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4) 2> 3597251 INFO (zkCallback-6849-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4) 2> 3597256 INFO (zkCallback-6838-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4) 2> 3597258 INFO (zkCallback-6843-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4) 2> 3597419 WARN (jetty-launcher-6802-thread-4) [n:127.0.0.1:46857_solr] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info 2> 3598621 INFO (jetty-launcher-6802-thread-1) [n:127.0.0.1:35735_solr] o.a.s.c.a.c.DistributedCollectionConfigSetCommandRunner Creating DistributedCollectionConfigSetCommandRunner. Collection and ConfigSet APIs are running distributed (not Overseer based) 2> 3598622 INFO (jetty-launcher-6802-thread-1) [n:127.0.0.1:35735_solr] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=true. Solr will be using distributed cluster state updates. 2> 3600224 INFO (jetty-launcher-6802-thread-4) [n:127.0.0.1:46857_solr] o.a.s.c.a.c.DistributedCollectionConfigSetCommandRunner Creating DistributedCollectionConfigSetCommandRunner. Collection and ConfigSet APIs are running distributed (not Overseer based) 2> 3600224 INFO (jetty-launcher-6802-thread-4) [n:127.0.0.1:46857_solr] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=true. Solr will be using distributed cluster state updates. 2> 3600250 INFO (jetty-launcher-6802-thread-3) [n:127.0.0.1:36667_solr] o.a.s.c.a.c.DistributedCollectionConfigSetCommandRunner Creating DistributedCollectionConfigSetCommandRunner. Collection and ConfigSet APIs are running distributed (not Overseer based) 2> 3600250 INFO (jetty-launcher-6802-thread-3) [n:127.0.0.1:36667_solr] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=true. Solr will be using distributed cluster state updates. 2> 3600551 INFO (jetty-launcher-6802-thread-2) [n:127.0.0.1:34951_solr] o.a.s.c.a.c.DistributedCollectionConfigSetCommandRunner Creating DistributedCollectionConfigSetCommandRunner. Collection and ConfigSet APIs are running distributed (not Overseer based) 2> 3600551 INFO (jetty-launcher-6802-thread-2) [n:127.0.0.1:34951_solr] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=true. Solr will be using distributed cluster state updates. 2> 3603411 INFO (jetty-launcher-6802-thread-1) [n:127.0.0.1:35735_solr] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node1 2> 3605353 INFO (jetty-launcher-6802-thread-1) [n:127.0.0.1:35735_solr] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 3605369 INFO (jetty-launcher-6802-thread-4) [n:127.0.0.1:46857_solr] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node4 2> 3605504 INFO (jetty-launcher-6802-thread-3) [n:127.0.0.1:36667_solr] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node3 2> 3605544 INFO (jetty-launcher-6802-thread-2) [n:127.0.0.1:34951_solr] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node2 2> 3605847 INFO (jetty-launcher-6802-thread-1) [n:127.0.0.1:35735_solr] o.a.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35735, zkHost=127.0.0.1:34921/solr} 2> 3606518 INFO (jetty-launcher-6802-thread-4) [n:127.0.0.1:46857_solr] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 3606814 INFO (jetty-launcher-6802-thread-3) [n:127.0.0.1:36667_solr] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 3606838 INFO (jetty-launcher-6802-thread-4) [n:127.0.0.1:46857_solr] o.a.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46857, zkHost=127.0.0.1:34921/solr} 2> 3606882 INFO (jetty-launcher-6802-thread-2) [n:127.0.0.1:34951_solr] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 3607253 INFO (jetty-launcher-6802-thread-3) [n:127.0.0.1:36667_solr] o.a.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36667, zkHost=127.0.0.1:34921/solr} 2> 3607260 INFO (jetty-launcher-6802-thread-2) [n:127.0.0.1:34951_solr] o.a.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34951, zkHost=127.0.0.1:34921/solr} 2> 3607288 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForAllNodes: numServers=4 2> 3607291 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForNode: 127.0.0.1:35735_solr 2> 3607292 WARN (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3607342 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 15000ms for client to connect to ZooKeeper 2> 3607388 INFO (zkConnectionManagerCallback-6885-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3607392 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3607393 WARN (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3607402 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4) 2> 3607415 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34921/solr ready 2> 3607416 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForNode: 127.0.0.1:46857_solr 2> 3607416 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForNode: 127.0.0.1:36667_solr 2> 3607417 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForNode: 127.0.0.1:34951_solr 2> 3607418 WARN (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3607468 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 45000ms for client to connect to ZooKeeper 2> 3607491 INFO (zkConnectionManagerCallback-6887-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 3607493 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 3607494 WARN (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 3607662 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.ReplaceNodeTest total_jettys: 4 2> 3607681 INFO (qtp260799485-10566) [n:127.0.0.1:34951_solr] o.a.s.c.a.c.DistributedCollectionConfigSetCommandRunner Running Collection API locally for CREATE asyncId=null 2> 3607745 INFO (DistributedCollectionApiCommandExecutor-6872-thread-1-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr c:replacenodetest_coll] o.a.s.c.a.c.CreateCollectionCmd Create collection replacenodetest_coll 2> 3608112 INFO (DistributedCollectionApiCommandExecutor-6872-thread-1-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr c:replacenodetest_coll] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replacenodetest_coll_shard3_replica_n1", 2> "node_name":"127.0.0.1:36667_solr", 2> "base_url":"http://127.0.0.1:36667/solr", 2> "collection":"replacenodetest_coll", 2> "shard":"shard3", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 3608134 INFO (DistributedCollectionApiCommandExecutor-6872-thread-1-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr c:replacenodetest_coll] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replacenodetest_coll_shard3_replica_n2", 2> "node_name":"127.0.0.1:46857_solr", 2> "base_url":"http://127.0.0.1:46857/solr", 2> "collection":"replacenodetest_coll", 2> "shard":"shard3", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 3608162 INFO (DistributedCollectionApiCommandExecutor-6872-thread-1-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr c:replacenodetest_coll] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replacenodetest_coll_shard1_replica_n3", 2> "node_name":"127.0.0.1:36667_solr", 2> "base_url":"http://127.0.0.1:36667/solr", 2> "collection":"replacenodetest_coll", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 3608187 INFO (DistributedCollectionApiCommandExecutor-6872-thread-1-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr c:replacenodetest_coll] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replacenodetest_coll_shard2_replica_n4", 2> "node_name":"127.0.0.1:36667_solr", 2> "base_url":"http://127.0.0.1:36667/solr", 2> "collection":"replacenodetest_coll", 2> "shard":"shard2", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 3608220 INFO (DistributedCollectionApiCommandExecutor-6872-thread-1-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr c:replacenodetest_coll] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replacenodetest_coll_shard1_replica_n5", 2> "node_name":"127.0.0.1:46857_solr", 2> "base_url":"http://127.0.0.1:46857/solr", 2> "collection":"replacenodetest_coll", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 3608301 INFO (DistributedCollectionApiCommandExecutor-6872-thread-1-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr c:replacenodetest_coll] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replacenodetest_coll_shard2_replica_n6", 2> "node_name":"127.0.0.1:46857_solr", 2> "base_url":"http://127.0.0.1:46857/solr", 2> "collection":"replacenodetest_coll", 2> "shard":"shard2", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 3608528 INFO (qtp1124965268-10574) [n:127.0.0.1:36667_solr x:replacenodetest_coll_shard3_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_shard3_replica_n1&action=CREATE&numShards=3&collection=replacenodetest_coll&shard=shard3&wt=javabin&version=2&replicaType=NRT 2> 3608542 INFO (qtp1124965268-10578) [n:127.0.0.1:36667_solr x:replacenodetest_coll_shard2_replica_n4] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node10&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_shard2_replica_n4&action=CREATE&numShards=3&collection=replacenodetest_coll&shard=shard2&wt=javabin&version=2&replicaType=NRT 2> 3608553 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr x:replacenodetest_coll_shard3_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_shard3_replica_n2&action=CREATE&numShards=3&collection=replacenodetest_coll&shard=shard3&wt=javabin&version=2&replicaType=NRT 2> 3608562 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr x:replacenodetest_coll_shard1_replica_n5] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node11&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_shard1_replica_n5&action=CREATE&numShards=3&collection=replacenodetest_coll&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 3608552 INFO (qtp1124965268-10575) [n:127.0.0.1:36667_solr x:replacenodetest_coll_shard1_replica_n3] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node9&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_shard1_replica_n3&action=CREATE&numShards=3&collection=replacenodetest_coll&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 3608572 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr x:replacenodetest_coll_shard2_replica_n6] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node12&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_shard2_replica_n6&action=CREATE&numShards=3&collection=replacenodetest_coll&shard=shard2&wt=javabin&version=2&replicaType=NRT 2> 3608908 INFO (qtp1124965268-10578) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard2 r:core_node10 x:replacenodetest_coll_shard2_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 3608913 INFO (qtp1124965268-10575) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 3609035 INFO (qtp1124965268-10574) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 3609051 INFO (qtp1124965268-10574) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n1] o.a.s.s.IndexSchema Schema name=cloud-dynamic 2> 3609052 INFO (qtp1124965268-10575) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.s.IndexSchema Schema name=cloud-dynamic 2> 3609051 INFO (qtp1124965268-10578) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard2 r:core_node10 x:replacenodetest_coll_shard2_replica_n4] o.a.s.s.IndexSchema Schema name=cloud-dynamic 2> 3609144 INFO (qtp1124965268-10575) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id 2> 3609147 INFO (qtp1124965268-10575) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_shard1_replica_n3' using configuration from configset conf1, trusted=true 2> 3609149 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 3609157 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 3609160 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 3609173 INFO (qtp1124965268-10575) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node3/replacenodetest_coll_shard1_replica_n3], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node3/replacenodetest_coll_shard1_replica_n3/data/] 2> 3609223 INFO (qtp1124965268-10574) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id 2> 3609228 INFO (qtp1124965268-10578) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard2 r:core_node10 x:replacenodetest_coll_shard2_replica_n4] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id 2> 3609230 INFO (qtp1124965268-10578) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard2 r:core_node10 x:replacenodetest_coll_shard2_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_shard2_replica_n4' using configuration from configset conf1, trusted=true 2> 3609229 INFO (qtp1124965268-10574) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_shard3_replica_n1' using configuration from configset conf1, trusted=true 2> 3609253 INFO (qtp1124965268-10574) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node3/replacenodetest_coll_shard3_replica_n1], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node3/replacenodetest_coll_shard3_replica_n1/data/] 2> 3609256 INFO (qtp1124965268-10578) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard2 r:core_node10 x:replacenodetest_coll_shard2_replica_n4] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node3/replacenodetest_coll_shard2_replica_n4], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node3/replacenodetest_coll_shard2_replica_n4/data/] 2> 3609298 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.s.IndexSchema Schema name=cloud-dynamic 2> 3609311 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.s.IndexSchema Schema name=cloud-dynamic 2> 3609308 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.s.IndexSchema Schema name=cloud-dynamic 2> 3609489 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id 2> 3609490 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_shard2_replica_n6' using configuration from configset conf1, trusted=true 2> 3609499 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node4/replacenodetest_coll_shard2_replica_n6], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node4/replacenodetest_coll_shard2_replica_n6/data/] 2> 3609521 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id 2> 3609521 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_shard3_replica_n2' using configuration from configset conf1, trusted=true 2> 3609530 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node4/replacenodetest_coll_shard3_replica_n2], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node4/replacenodetest_coll_shard3_replica_n2/data/] 2> 3609543 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id 2> 3609548 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_shard1_replica_n5' using configuration from configset conf1, trusted=true 2> 3609568 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node4/replacenodetest_coll_shard1_replica_n5], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node4/replacenodetest_coll_shard1_replica_n5/data/] 2> 3612376 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 3612392 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 3612393 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 3612421 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 3612422 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 3612429 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 3612429 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 3612449 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 3612449 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 3612500 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 3612501 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 3612511 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 3612516 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 3612523 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 3612524 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723513201328128 2> 3612529 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 3612529 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 3612634 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replacenodetest_coll/terms/shard3 to Terms{values={core_node8=0}, version=0} for registerTerm 2> 3612635 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replacenodetest_coll/leaders/shard3 2> 3612651 INFO (searcherExecutor-6897-thread-1-processing-127.0.0.1:46857_solr replacenodetest_coll_shard3_replica_n2 replacenodetest_coll shard3 core_node8) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 3612696 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 3612712 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 3612740 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 3612749 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 3612761 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 3612762 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723513450889216 2> 3612780 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard3: total=2 found=1 timeoute in=9974ms 2> 3612814 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 3612815 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723513506463744 2> 3612821 INFO (qtp1124965268-10575) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 3612986 INFO (qtp1124965268-10575) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 3612986 INFO (qtp1124965268-10575) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 3613038 INFO (qtp1124965268-10578) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard2 r:core_node10 x:replacenodetest_coll_shard2_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 3613044 INFO (qtp1124965268-10578) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard2 r:core_node10 x:replacenodetest_coll_shard2_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 3613086 INFO (qtp1124965268-10574) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 3613086 INFO (qtp1124965268-10574) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 3613094 INFO (searcherExecutor-6895-thread-1-processing-127.0.0.1:46857_solr replacenodetest_coll_shard2_replica_n6 replacenodetest_coll shard2 core_node12) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 3613197 INFO (searcherExecutor-6899-thread-1-processing-127.0.0.1:46857_solr replacenodetest_coll_shard1_replica_n5 replacenodetest_coll shard1 core_node11) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 3613286 INFO (qtp1124965268-10575) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 3613286 INFO (qtp1124965268-10575) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 3613296 INFO (qtp1124965268-10578) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard2 r:core_node10 x:replacenodetest_coll_shard2_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 3613313 INFO (qtp1124965268-10578) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard2 r:core_node10 x:replacenodetest_coll_shard2_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 3613381 INFO (qtp1124965268-10574) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 3613381 INFO (qtp1124965268-10574) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 3613428 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replacenodetest_coll/terms/shard1 to Terms{values={core_node11=0}, version=0} for registerTerm 2> 3613459 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replacenodetest_coll/leaders/shard1 2> 3613516 INFO (qtp1124965268-10575) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 3613540 INFO (qtp1124965268-10575) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 3613591 INFO (qtp1124965268-10575) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 3613627 INFO (qtp1124965268-10575) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723514357907456 2> 3613699 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replacenodetest_coll/terms/shard2 to Terms{values={core_node12=0}, version=0} for registerTerm 2> 3613699 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replacenodetest_coll/leaders/shard2 2> 3613764 INFO (qtp1124965268-10574) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 3613794 INFO (qtp1124965268-10574) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 3613802 INFO (qtp1124965268-10574) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 3613805 INFO (qtp1124965268-10574) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723514544553984 2> 3613900 INFO (qtp1124965268-10578) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard2 r:core_node10 x:replacenodetest_coll_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 3613932 INFO (searcherExecutor-6889-thread-1-processing-127.0.0.1:36667_solr replacenodetest_coll_shard1_replica_n3 replacenodetest_coll shard1 core_node9) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 3613939 INFO (qtp1124965268-10578) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard2 r:core_node10 x:replacenodetest_coll_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 3613946 INFO (zkCallback-6849-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3613962 INFO (qtp1124965268-10578) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard2 r:core_node10 x:replacenodetest_coll_shard2_replica_n4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 3613963 INFO (qtp1124965268-10578) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard2 r:core_node10 x:replacenodetest_coll_shard2_replica_n4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723514709180416 2> 3614049 INFO (searcherExecutor-6891-thread-1-processing-127.0.0.1:36667_solr replacenodetest_coll_shard3_replica_n1 replacenodetest_coll shard3 core_node7) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 3614061 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoute in=9902ms 2> 3614152 INFO (qtp1124965268-10575) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replacenodetest_coll/terms/shard1 to Terms{values={core_node11=0, core_node9=0}, version=1} for registerTerm 2> 3614152 INFO (qtp1124965268-10575) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replacenodetest_coll/leaders/shard1 2> 3614260 INFO (zkCallback-6849-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3614285 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoute in=9944ms 2> 3614331 INFO (qtp1124965268-10574) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replacenodetest_coll/terms/shard3 to Terms{values={core_node7=0, core_node8=0}, version=1} for registerTerm 2> 3614373 INFO (qtp1124965268-10574) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replacenodetest_coll/leaders/shard3 2> 3614400 INFO (searcherExecutor-6893-thread-1-processing-127.0.0.1:36667_solr replacenodetest_coll_shard2_replica_n4 replacenodetest_coll shard2 core_node10) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard2 r:core_node10 x:replacenodetest_coll_shard2_replica_n4] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 3614527 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 3614528 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 3614528 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:46857/solr/replacenodetest_coll_shard3_replica_n2/ 2> 3614549 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.u.PeerSync PeerSync: core=replacenodetest_coll_shard3_replica_n2 url=http://127.0.0.1:46857/solr START replicas=[http://127.0.0.1:36667/solr/replacenodetest_coll_shard3_replica_n1/] nUpdates=100 2> 3614561 INFO (qtp1124965268-10578) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard2 r:core_node10 x:replacenodetest_coll_shard2_replica_n4] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replacenodetest_coll/terms/shard2 to Terms{values={core_node10=0, core_node12=0}, version=1} for registerTerm 2> 3614561 INFO (qtp1124965268-10578) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard2 r:core_node10 x:replacenodetest_coll_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replacenodetest_coll/leaders/shard2 2> 3614576 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.u.PeerSync PeerSync: core=replacenodetest_coll_shard3_replica_n2 url=http://127.0.0.1:46857/solr DONE. We have no versions. sync failed. 2> 3614613 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 3614613 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 3614613 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:46857/solr/replacenodetest_coll_shard1_replica_n5/ 2> 3614628 INFO (qtp1124965268-10573) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n1] o.a.s.c.S.Request webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=2 2> 3614634 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate 2> 3614635 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway 2> 3614635 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/replacenodetest_coll/leaders/shard3/leader after winning as /collections/replacenodetest_coll/leader_elect/shard3/election/72080973716586505-core_node8-n_0000000000 2> 3614668 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.u.PeerSync PeerSync: core=replacenodetest_coll_shard1_replica_n5 url=http://127.0.0.1:46857/solr START replicas=[http://127.0.0.1:36667/solr/replacenodetest_coll_shard1_replica_n3/] nUpdates=100 2> 3614686 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.u.PeerSync PeerSync: core=replacenodetest_coll_shard1_replica_n5 url=http://127.0.0.1:46857/solr DONE. We have no versions. sync failed. 2> 3614707 INFO (qtp1124965268-10579) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.c.S.Request webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=12 2> 3614717 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate 2> 3614718 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway 2> 3614718 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/replacenodetest_coll/leaders/shard1/leader after winning as /collections/replacenodetest_coll/leader_elect/shard1/election/72080973716586505-core_node11-n_0000000000 2> 3614775 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:46857/solr/replacenodetest_coll_shard3_replica_n2/ shard3 2> 3614817 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 3614817 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 3614817 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:46857/solr/replacenodetest_coll_shard2_replica_n6/ 2> 3614864 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:46857/solr/replacenodetest_coll_shard1_replica_n5/ shard1 2> 3614887 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.u.PeerSync PeerSync: core=replacenodetest_coll_shard2_replica_n6 url=http://127.0.0.1:46857/solr START replicas=[http://127.0.0.1:36667/solr/replacenodetest_coll_shard2_replica_n4/] nUpdates=100 2> 3614911 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.u.PeerSync PeerSync: core=replacenodetest_coll_shard2_replica_n6 url=http://127.0.0.1:46857/solr DONE. We have no versions. sync failed. 2> 3614933 INFO (qtp1124965268-10572) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard2 r:core_node10 x:replacenodetest_coll_shard2_replica_n4] o.a.s.c.S.Request webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=11 2> 3614947 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate 2> 3614950 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 3614956 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway 2> 3614957 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/replacenodetest_coll/leaders/shard2/leader after winning as /collections/replacenodetest_coll/leader_elect/shard2/election/72080973716586505-core_node12-n_0000000000 2> 3615172 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:46857/solr/replacenodetest_coll_shard2_replica_n6/ shard2 2> 3615216 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 3615272 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 3615273 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_shard3_replica_n2&action=CREATE&numShards=3&collection=replacenodetest_coll&shard=shard3&wt=javabin&version=2&replicaType=NRT} status=0 QTime=6723 2> 3615350 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3615473 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node12&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_shard2_replica_n6&action=CREATE&numShards=3&collection=replacenodetest_coll&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=6913 2> 3615485 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3615566 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node11&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_shard1_replica_n5&action=CREATE&numShards=3&collection=replacenodetest_coll&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=6960 2> 3615692 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3615812 INFO (qtp1124965268-10575) [n:127.0.0.1:36667_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node9&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_shard1_replica_n3&action=CREATE&numShards=3&collection=replacenodetest_coll&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=7274 2> 3615825 INFO (zkCallback-6849-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3615830 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3615890 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3615932 INFO (zkCallback-6849-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3615977 INFO (qtp1124965268-10578) [n:127.0.0.1:36667_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node10&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_shard2_replica_n4&action=CREATE&numShards=3&collection=replacenodetest_coll&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=7437 2> 3615989 INFO (qtp1124965268-10574) [n:127.0.0.1:36667_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_shard3_replica_n1&action=CREATE&numShards=3&collection=replacenodetest_coll&shard=shard3&wt=javabin&version=2&replicaType=NRT} status=0 QTime=7463 2> 3616014 INFO (qtp260799485-10566) [n:127.0.0.1:34951_solr] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas 2> 3616096 INFO (qtp260799485-10566) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&collection.configName=conf1&name=replacenodetest_coll&nrtReplicas=2&action=CREATE&numShards=3&tlogReplicas=0&createNodeSet=127.0.0.1:46857_solr,127.0.0.1:36667_solr&wt=javabin&version=2} status=0 QTime=8417 2> 3616098 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForActiveCollection: replacenodetest_coll 2> 3616217 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.c.ReplaceNodeTest excluded_nodes : [127.0.0.1:35735_solr, 127.0.0.1:34951_solr] 2> 3616240 INFO (qtp260799485-10581) [n:127.0.0.1:34951_solr] o.a.s.c.a.c.DistributedCollectionConfigSetCommandRunner Running Collection API locally for REPLACENODE asyncId=000 2> 3616405 INFO (qtp260799485-10581) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=000&sourceNode=127.0.0.1:46857_solr¶llel=true&action=REPLACENODE&wt=javabin&version=2} status=0 QTime=175 2> 3616454 INFO (qtp260799485-10582) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=000&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=41 2> 3616508 INFO (DistributedCollectionApiCommandExecutor-6872-thread-2-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr] o.a.s.c.a.c.ReplaceNodeCmd Going to create replica for collection=replacenodetest_coll shard=shard1 on node=null 2> 3616555 INFO (DistributedCollectionApiCommandExecutor-6872-thread-2-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:34951_solr for creating new replica of shard shard1 for collection replacenodetest_coll 2> 3616622 INFO (DistributedCollectionApiCommandExecutor-6872-thread-2-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command. 2> 3616644 INFO (DistributedCollectionApiCommandExecutor-6872-thread-2-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replacenodetest_coll_shard1_replica_n13", 2> "node_name":"127.0.0.1:34951_solr", 2> "base_url":"http://127.0.0.1:34951/solr", 2> "collection":"replacenodetest_coll", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"addreplica"} 2> 3616744 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3616748 INFO (zkCallback-6843-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3616752 INFO (zkCallback-6849-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3616954 INFO (qtp260799485-10565) [n:127.0.0.1:34951_solr x:replacenodetest_coll_shard1_replica_n13] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=000356769838042625&qt=/admin/cores&coreNodeName=core_node14&collection.configName=conf1&name=replacenodetest_coll_shard1_replica_n13&action=CREATE&collection=replacenodetest_coll&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=20 2> 3616957 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-1-processing-127.0.0.1:34951_solr replacenodetest_coll_shard1_replica_n13 000356769838042625 create) [n:127.0.0.1:34951_solr x:replacenodetest_coll_shard1_replica_n13] o.a.s.h.a.CoreAdminOperation core create command async=000356769838042625&qt=/admin/cores&coreNodeName=core_node14&collection.configName=conf1&name=replacenodetest_coll_shard1_replica_n13&action=CREATE&collection=replacenodetest_coll&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 3616976 INFO (qtp260799485-10566) [n:127.0.0.1:34951_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 000356769838042625 2> 3616986 INFO (qtp260799485-10566) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=000356769838042625&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=13 2> 3617185 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3617186 INFO (zkCallback-6849-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3617518 INFO (qtp260799485-10581) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=000&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=53 2> 3617565 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-1-processing-127.0.0.1:34951_solr replacenodetest_coll_shard1_replica_n13 000356769838042625 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 3617648 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-1-processing-127.0.0.1:34951_solr replacenodetest_coll_shard1_replica_n13 000356769838042625 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.s.IndexSchema Schema name=cloud-dynamic 2> 3617836 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-1-processing-127.0.0.1:34951_solr replacenodetest_coll_shard1_replica_n13 000356769838042625 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id 2> 3617836 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-1-processing-127.0.0.1:34951_solr replacenodetest_coll_shard1_replica_n13 000356769838042625 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_shard1_replica_n13' using configuration from configset conf1, trusted=true 2> 3617839 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-1-processing-127.0.0.1:34951_solr replacenodetest_coll_shard1_replica_n13 000356769838042625 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node2/replacenodetest_coll_shard1_replica_n13], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node2/replacenodetest_coll_shard1_replica_n13/data/] 2> 3618007 INFO (qtp260799485-10582) [n:127.0.0.1:34951_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 000356769838042625 2> 3618007 INFO (qtp260799485-10582) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=000356769838042625&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 3618580 INFO (qtp260799485-10583) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=000&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=43 2> 3620536 INFO (qtp260799485-10565) [n:127.0.0.1:34951_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 000356769838042625 2> 3620542 INFO (qtp260799485-10565) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=000356769838042625&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=8 2> 3620612 INFO (qtp260799485-10584) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=000&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=78 2> 3621564 INFO (qtp260799485-10566) [n:127.0.0.1:34951_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 000356769838042625 2> 3621564 INFO (qtp260799485-10566) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=000356769838042625&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 3621685 INFO (qtp260799485-10581) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=000&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=60 2> 3622488 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-1-processing-127.0.0.1:34951_solr replacenodetest_coll_shard1_replica_n13 000356769838042625 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 3622562 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-1-processing-127.0.0.1:34951_solr replacenodetest_coll_shard1_replica_n13 000356769838042625 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 3622574 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-1-processing-127.0.0.1:34951_solr replacenodetest_coll_shard1_replica_n13 000356769838042625 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 3622590 INFO (qtp260799485-10582) [n:127.0.0.1:34951_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 000356769838042625 2> 3622590 INFO (qtp260799485-10582) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=000356769838042625&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 3622675 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-1-processing-127.0.0.1:34951_solr replacenodetest_coll_shard1_replica_n13 000356769838042625 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 3622675 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-1-processing-127.0.0.1:34951_solr replacenodetest_coll_shard1_replica_n13 000356769838042625 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 3622756 INFO (qtp260799485-10583) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=000&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=63 2> 3622836 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-1-processing-127.0.0.1:34951_solr replacenodetest_coll_shard1_replica_n13 000356769838042625 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 3622848 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-1-processing-127.0.0.1:34951_solr replacenodetest_coll_shard1_replica_n13 000356769838042625 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 3622873 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-1-processing-127.0.0.1:34951_solr replacenodetest_coll_shard1_replica_n13 000356769838042625 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 3622893 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-1-processing-127.0.0.1:34951_solr replacenodetest_coll_shard1_replica_n13 000356769838042625 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723524074012672 2> 3622998 INFO (searcherExecutor-6925-thread-1-processing-replacenodetest_coll_shard1_replica_n13 000356769838042625 core_node14 create 127.0.0.1:34951_solr replacenodetest_coll shard1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 3623118 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-1-processing-127.0.0.1:34951_solr replacenodetest_coll_shard1_replica_n13 000356769838042625 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replacenodetest_coll/terms/shard1 to Terms{values={core_node14=0, core_node11=0, core_node9=0}, version=2} for registerTerm 2> 3623119 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-1-processing-127.0.0.1:34951_solr replacenodetest_coll_shard1_replica_n13 000356769838042625 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replacenodetest_coll/leaders/shard1 2> 3623346 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-1-processing-127.0.0.1:34951_solr replacenodetest_coll_shard1_replica_n13 000356769838042625 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.ZkController Core needs to recover:replacenodetest_coll_shard1_replica_n13 2> 3623349 INFO (updateExecutor-6832-thread-1-processing-replacenodetest_coll_shard1_replica_n13 000356769838042625 core_node14 create 127.0.0.1:34951_solr replacenodetest_coll shard1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.DefaultSolrCoreState Running recovery 2> 3623351 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard1_replica_n13 000356769838042625 core_node14 create 127.0.0.1:34951_solr replacenodetest_coll shard1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true 2> 3623352 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard1_replica_n13 000356769838042625 core_node14 create 127.0.0.1:34951_solr replacenodetest_coll shard1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy startupVersions is empty 2> 3623407 INFO (qtp1247512449-10560) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.c.S.Request webapp=/solr path=/admin/ping params={wt=javabin&version=2} hits=0 status=0 QTime=2 2> 3623418 INFO (qtp1247512449-10560) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.c.S.Request webapp=/solr path=/admin/ping params={wt=javabin&version=2} status=0 QTime=13 2> 3623420 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard1_replica_n13 000356769838042625 core_node14 create 127.0.0.1:34951_solr replacenodetest_coll shard1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[replacenodetest_coll_shard1_replica_n13] 2> 3623427 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard1_replica_n13 000356769838042625 core_node14 create 127.0.0.1:34951_solr replacenodetest_coll shard1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null} 2> 3623427 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard1_replica_n13 000356769838042625 core_node14 create 127.0.0.1:34951_solr replacenodetest_coll shard1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy Publishing state of core [replacenodetest_coll_shard1_replica_n13] as recovering, leader is [http://127.0.0.1:46857/solr/replacenodetest_coll_shard1_replica_n5/] and I am [http://127.0.0.1:34951/solr/replacenodetest_coll_shard1_replica_n13/] 2> 3623468 INFO (zkCallback-6843-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3623468 INFO (zkCallback-6849-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3623468 INFO (zkCallback-6843-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3623469 INFO (zkCallback-6843-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3623473 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3623474 INFO (zkCallback-6843-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3623485 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard1_replica_n13 000356769838042625 core_node14 create 127.0.0.1:34951_solr replacenodetest_coll shard1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:46857/solr]; [WaitForState: action=PREPRECOVERY&core=replacenodetest_coll_shard1_replica_n5&nodeName=127.0.0.1:34951_solr&coreNodeName=core_node14&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true] 2> 3623495 INFO (qtp1247512449-10561) [n:127.0.0.1:46857_solr x:replacenodetest_coll_shard1_replica_n5] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node14, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true 2> 3623497 INFO (qtp1247512449-10561) [n:127.0.0.1:46857_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=replacenodetest_coll, shard=shard1, thisCore=replacenodetest_coll_shard1_replica_n5, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:34951_solr, coreNodeName=core_node14, onlyIfActiveCheckResult=false, nodeProps: core_node14:{ 2> "core":"replacenodetest_coll_shard1_replica_n13", 2> "node_name":"127.0.0.1:34951_solr", 2> "base_url":"http://127.0.0.1:34951/solr", 2> "state":"recovering", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 3623505 INFO (qtp1247512449-10561) [n:127.0.0.1:46857_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=replacenodetest_coll, shard=shard1, thisCore=replacenodetest_coll_shard1_replica_n5, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:34951_solr, coreNodeName=core_node14, onlyIfActiveCheckResult=false, nodeProps: core_node14:{ 2> "core":"replacenodetest_coll_shard1_replica_n13", 2> "node_name":"127.0.0.1:34951_solr", 2> "base_url":"http://127.0.0.1:34951/solr", 2> "state":"recovering", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 3623506 INFO (qtp1247512449-10561) [n:127.0.0.1:46857_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=replacenodetest_coll, shard=shard1, thisCore=replacenodetest_coll_shard1_replica_n5, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:34951_solr, coreNodeName=core_node14, onlyIfActiveCheckResult=false, nodeProps: core_node14:{ 2> "core":"replacenodetest_coll_shard1_replica_n13", 2> "node_name":"127.0.0.1:34951_solr", 2> "base_url":"http://127.0.0.1:34951/solr", 2> "state":"recovering", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 3623506 INFO (qtp1247512449-10561) [n:127.0.0.1:46857_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:34951_solr&onlyIfLeaderActive=true&core=replacenodetest_coll_shard1_replica_n5&coreNodeName=core_node14&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=13 2> 3623618 INFO (qtp260799485-10565) [n:127.0.0.1:34951_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 000356769838042625 2> 3623618 INFO (qtp260799485-10565) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=000356769838042625&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 3623628 INFO (DistributedCollectionApiCommandExecutor-6872-thread-2-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr] o.a.s.c.a.c.ReplaceNodeCmd Going to create replica for collection=replacenodetest_coll shard=shard2 on node=null 2> 3623663 INFO (DistributedCollectionApiCommandExecutor-6872-thread-2-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:35735_solr for creating new replica of shard shard2 for collection replacenodetest_coll 2> 3623712 INFO (DistributedCollectionApiCommandExecutor-6872-thread-2-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command. 2> 3623720 INFO (DistributedCollectionApiCommandExecutor-6872-thread-2-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replacenodetest_coll_shard2_replica_n15", 2> "node_name":"127.0.0.1:35735_solr", 2> "base_url":"http://127.0.0.1:35735/solr", 2> "collection":"replacenodetest_coll", 2> "shard":"shard2", 2> "state":"down", 2> "type":"NRT", 2> "operation":"addreplica"} 2> 3623770 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3623770 INFO (zkCallback-6843-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3623771 INFO (zkCallback-6843-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3623772 INFO (zkCallback-6849-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3623776 INFO (zkCallback-6843-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3623776 INFO (zkCallback-6843-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3623836 INFO (qtp260799485-10584) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=000&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=69 2> 3623876 INFO (qtp61164958-10558) [n:127.0.0.1:35735_solr x:replacenodetest_coll_shard2_replica_n15] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=000356776774238633&qt=/admin/cores&coreNodeName=core_node16&collection.configName=conf1&name=replacenodetest_coll_shard2_replica_n15&action=CREATE&collection=replacenodetest_coll&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=20 2> 3623877 INFO (parallelCoreAdminAPIBaseExecutor-6865-thread-1-processing-127.0.0.1:35735_solr replacenodetest_coll_shard2_replica_n15 000356776774238633 create) [n:127.0.0.1:35735_solr x:replacenodetest_coll_shard2_replica_n15] o.a.s.h.a.CoreAdminOperation core create command async=000356776774238633&qt=/admin/cores&coreNodeName=core_node16&collection.configName=conf1&name=replacenodetest_coll_shard2_replica_n15&action=CREATE&collection=replacenodetest_coll&shard=shard2&wt=javabin&version=2&replicaType=NRT 2> 3623891 INFO (qtp61164958-10567) [n:127.0.0.1:35735_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 000356776774238633 2> 3623893 INFO (qtp61164958-10567) [n:127.0.0.1:35735_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=000356776774238633&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=4 2> 3624008 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard1_replica_n13 000356769838042625 core_node14 create 127.0.0.1:34951_solr replacenodetest_coll shard1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:46857/solr/replacenodetest_coll_shard1_replica_n5/] - recoveringAfterStartup=[true] 2> 3624011 WARN (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard1_replica_n13 000356769838042625 core_node14 create 127.0.0.1:34951_solr replacenodetest_coll shard1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.PeerSyncWithLeader no frame of reference to tell if we've missed updates 2> 3624012 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard1_replica_n13 000356769838042625 core_node14 create 127.0.0.1:34951_solr replacenodetest_coll shard1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy PeerSync Recovery was not successful - trying replication. 2> 3624012 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard1_replica_n13 000356769838042625 core_node14 create 127.0.0.1:34951_solr replacenodetest_coll shard1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy Starting Replication Recovery. 2> 3624013 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard1_replica_n13 000356769838042625 core_node14 create 127.0.0.1:34951_solr replacenodetest_coll shard1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy Attempting to replicate from [http://127.0.0.1:46857/solr/replacenodetest_coll_shard1_replica_n5/]. 2> 3624085 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3624085 INFO (zkCallback-6843-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3624085 INFO (zkCallback-6843-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3624086 INFO (zkCallback-6843-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3624086 INFO (zkCallback-6843-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3624087 INFO (zkCallback-6849-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3624380 INFO (qtp1124965268-10572) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46857/solr/replacenodetest_coll_shard3_replica_n2/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 110 2> 3624383 INFO (qtp1124965268-10575) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard2 r:core_node10 x:replacenodetest_coll_shard2_replica_n4] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46857/solr/replacenodetest_coll_shard2_replica_n6/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 103 2> 3624386 INFO (qtp1247512449-10591) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46857/solr/replacenodetest_coll_shard1_replica_n5/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 225 2> 3624391 INFO (qtp1247512449-10560) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46857/solr/replacenodetest_coll_shard1_replica_n5/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 230 2> 3624419 INFO (qtp1124965268-10578) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46857/solr/replacenodetest_coll_shard1_replica_n5/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 3 2> 3624420 INFO (parallelCoreAdminAPIBaseExecutor-6865-thread-1-processing-127.0.0.1:35735_solr replacenodetest_coll_shard2_replica_n15 000356776774238633 create) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 3624467 INFO (parallelCoreAdminAPIBaseExecutor-6865-thread-1-processing-127.0.0.1:35735_solr replacenodetest_coll_shard2_replica_n15 000356776774238633 create) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.s.IndexSchema Schema name=cloud-dynamic 2> 3624548 INFO (qtp260799485-10582) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.p.DistributedUpdateProcessor Ignoring commit while not ACTIVE - state: BUFFERING replay: false 2> 3624549 INFO (qtp260799485-10582) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46857/solr/replacenodetest_coll_shard1_replica_n5/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 55 2> 3624551 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 533 2> 3624563 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 3624577 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard1_replica_n13 000356769838042625 core_node14 create 127.0.0.1:34951_solr replacenodetest_coll shard1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.h.IndexFetcher Leader's generation: 1 2> 3624577 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard1_replica_n13 000356769838042625 core_node14 create 127.0.0.1:34951_solr replacenodetest_coll shard1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.h.IndexFetcher Leader's version: 0 2> 3624577 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard1_replica_n13 000356769838042625 core_node14 create 127.0.0.1:34951_solr replacenodetest_coll shard1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.h.IndexFetcher Follower's generation: 1 2> 3624577 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard1_replica_n13 000356769838042625 core_node14 create 127.0.0.1:34951_solr replacenodetest_coll shard1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.h.IndexFetcher Follower's version: 0 2> 3624578 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard1_replica_n13 000356769838042625 core_node14 create 127.0.0.1:34951_solr replacenodetest_coll shard1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy No replay needed. 2> 3624583 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard1_replica_n13 000356769838042625 core_node14 create 127.0.0.1:34951_solr replacenodetest_coll shard1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy Replication Recovery was successful. 2> 3624583 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard1_replica_n13 000356769838042625 core_node14 create 127.0.0.1:34951_solr replacenodetest_coll shard1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy Registering as Active after recovery. 2> 3624603 INFO (parallelCoreAdminAPIBaseExecutor-6865-thread-1-processing-127.0.0.1:35735_solr replacenodetest_coll_shard2_replica_n15 000356776774238633 create) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id 2> 3624604 INFO (parallelCoreAdminAPIBaseExecutor-6865-thread-1-processing-127.0.0.1:35735_solr replacenodetest_coll_shard2_replica_n15 000356776774238633 create) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_shard2_replica_n15' using configuration from configset conf1, trusted=true 2> 3624624 INFO (parallelCoreAdminAPIBaseExecutor-6865-thread-1-processing-127.0.0.1:35735_solr replacenodetest_coll_shard2_replica_n15 000356776774238633 create) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node1/replacenodetest_coll_shard2_replica_n15], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node1/replacenodetest_coll_shard2_replica_n15/data/] 2> 3624633 INFO (zkCallback-6849-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3624636 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3624641 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard1_replica_n13 000356769838042625 core_node14 create 127.0.0.1:34951_solr replacenodetest_coll shard1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy Updating version bucket highest from index after successful recovery. 2> 3624641 INFO (zkCallback-6843-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3624641 INFO (zkCallback-6843-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3624641 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard1_replica_n13 000356769838042625 core_node14 create 127.0.0.1:34951_solr replacenodetest_coll shard1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723525906923520 2> 3624641 INFO (zkCallback-6843-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3624644 INFO (zkCallback-6843-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3624668 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard1_replica_n13 000356769838042625 core_node14 create 127.0.0.1:34951_solr replacenodetest_coll shard1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy Finished recovery process, successful=[true] msTimeTaken=1315.0 2> 3624668 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard1_replica_n13 000356769838042625 core_node14 create 127.0.0.1:34951_solr replacenodetest_coll shard1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.RecoveryStrategy Finished recovery process. recoveringAfterStartup=true msTimeTaken=1315.0 2> 3624936 INFO (qtp260799485-10583) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=000&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=85 2> 3624980 INFO (qtp61164958-10568) [n:127.0.0.1:35735_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 000356776774238633 2> 3624981 INFO (qtp61164958-10568) [n:127.0.0.1:35735_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=000356776774238633&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 3625979 INFO (qtp260799485-10565) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=000&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=31 2> 3625998 INFO (qtp61164958-10569) [n:127.0.0.1:35735_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 000356776774238633 2> 3625998 INFO (qtp61164958-10569) [n:127.0.0.1:35735_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=000356776774238633&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=11 2> 3626427 INFO (parallelCoreAdminAPIBaseExecutor-6865-thread-1-processing-127.0.0.1:35735_solr replacenodetest_coll_shard2_replica_n15 000356776774238633 create) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 3626497 INFO (parallelCoreAdminAPIBaseExecutor-6865-thread-1-processing-127.0.0.1:35735_solr replacenodetest_coll_shard2_replica_n15 000356776774238633 create) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 3626498 INFO (parallelCoreAdminAPIBaseExecutor-6865-thread-1-processing-127.0.0.1:35735_solr replacenodetest_coll_shard2_replica_n15 000356776774238633 create) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 3626558 INFO (parallelCoreAdminAPIBaseExecutor-6865-thread-1-processing-127.0.0.1:35735_solr replacenodetest_coll_shard2_replica_n15 000356776774238633 create) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 3626559 INFO (parallelCoreAdminAPIBaseExecutor-6865-thread-1-processing-127.0.0.1:35735_solr replacenodetest_coll_shard2_replica_n15 000356776774238633 create) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 3626656 INFO (parallelCoreAdminAPIBaseExecutor-6865-thread-1-processing-127.0.0.1:35735_solr replacenodetest_coll_shard2_replica_n15 000356776774238633 create) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 3626660 INFO (parallelCoreAdminAPIBaseExecutor-6865-thread-1-processing-127.0.0.1:35735_solr replacenodetest_coll_shard2_replica_n15 000356776774238633 create) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 3626666 INFO (parallelCoreAdminAPIBaseExecutor-6865-thread-1-processing-127.0.0.1:35735_solr replacenodetest_coll_shard2_replica_n15 000356776774238633 create) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 3626667 INFO (parallelCoreAdminAPIBaseExecutor-6865-thread-1-processing-127.0.0.1:35735_solr replacenodetest_coll_shard2_replica_n15 000356776774238633 create) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723528031338496 2> 3626779 INFO (searcherExecutor-6932-thread-1-processing-replacenodetest_coll_shard2_replica_n15 000356776774238633 core_node16 create 127.0.0.1:35735_solr replacenodetest_coll shard2) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 3626801 INFO (parallelCoreAdminAPIBaseExecutor-6865-thread-1-processing-127.0.0.1:35735_solr replacenodetest_coll_shard2_replica_n15 000356776774238633 create) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replacenodetest_coll/terms/shard2 to Terms{values={core_node10=0, core_node12=0, core_node16=0}, version=2} for registerTerm 2> 3626801 INFO (parallelCoreAdminAPIBaseExecutor-6865-thread-1-processing-127.0.0.1:35735_solr replacenodetest_coll_shard2_replica_n15 000356776774238633 create) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replacenodetest_coll/leaders/shard2 2> 3626941 INFO (parallelCoreAdminAPIBaseExecutor-6865-thread-1-processing-127.0.0.1:35735_solr replacenodetest_coll_shard2_replica_n15 000356776774238633 create) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.c.ZkController Core needs to recover:replacenodetest_coll_shard2_replica_n15 2> 3626949 INFO (updateExecutor-6831-thread-1-processing-replacenodetest_coll_shard2_replica_n15 000356776774238633 core_node16 create 127.0.0.1:35735_solr replacenodetest_coll shard2) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.u.DefaultSolrCoreState Running recovery 2> 3626961 INFO (recoveryExecutor-6837-thread-1-processing-replacenodetest_coll_shard2_replica_n15 000356776774238633 core_node16 create 127.0.0.1:35735_solr replacenodetest_coll shard2) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true 2> 3626962 INFO (recoveryExecutor-6837-thread-1-processing-replacenodetest_coll_shard2_replica_n15 000356776774238633 core_node16 create 127.0.0.1:35735_solr replacenodetest_coll shard2) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.c.RecoveryStrategy startupVersions is empty 2> 3627042 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.c.S.Request webapp=/solr path=/admin/ping params={wt=javabin&version=2} hits=0 status=0 QTime=40 2> 3627050 INFO (qtp61164958-10554) [n:127.0.0.1:35735_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 000356776774238633 2> 3627043 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.c.S.Request webapp=/solr path=/admin/ping params={wt=javabin&version=2} status=0 QTime=40 2> 3627059 INFO (recoveryExecutor-6837-thread-1-processing-replacenodetest_coll_shard2_replica_n15 000356776774238633 core_node16 create 127.0.0.1:35735_solr replacenodetest_coll shard2) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[replacenodetest_coll_shard2_replica_n15] 2> 3627063 INFO (recoveryExecutor-6837-thread-1-processing-replacenodetest_coll_shard2_replica_n15 000356776774238633 core_node16 create 127.0.0.1:35735_solr replacenodetest_coll shard2) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null} 2> 3627071 INFO (recoveryExecutor-6837-thread-1-processing-replacenodetest_coll_shard2_replica_n15 000356776774238633 core_node16 create 127.0.0.1:35735_solr replacenodetest_coll shard2) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.c.RecoveryStrategy Publishing state of core [replacenodetest_coll_shard2_replica_n15] as recovering, leader is [http://127.0.0.1:46857/solr/replacenodetest_coll_shard2_replica_n6/] and I am [http://127.0.0.1:35735/solr/replacenodetest_coll_shard2_replica_n15/] 2> 3627071 INFO (qtp61164958-10554) [n:127.0.0.1:35735_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=000356776774238633&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=22 2> 3627124 INFO (zkCallback-6849-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3627126 INFO (zkCallback-6843-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3627126 INFO (zkCallback-6843-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3627126 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3627126 INFO (zkCallback-6843-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3627126 INFO (qtp260799485-10584) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=000&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=141 2> 3627126 INFO (zkCallback-6838-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3627126 INFO (zkCallback-6843-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3627144 INFO (zkCallback-6838-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3627146 INFO (zkCallback-6838-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3627158 INFO (recoveryExecutor-6837-thread-1-processing-replacenodetest_coll_shard2_replica_n15 000356776774238633 core_node16 create 127.0.0.1:35735_solr replacenodetest_coll shard2) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:46857/solr]; [WaitForState: action=PREPRECOVERY&core=replacenodetest_coll_shard2_replica_n6&nodeName=127.0.0.1:35735_solr&coreNodeName=core_node16&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true] 2> 3627171 INFO (qtp1247512449-10591) [n:127.0.0.1:46857_solr x:replacenodetest_coll_shard2_replica_n6] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node16, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true 2> 3627176 INFO (qtp1247512449-10591) [n:127.0.0.1:46857_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=replacenodetest_coll, shard=shard2, thisCore=replacenodetest_coll_shard2_replica_n6, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:35735_solr, coreNodeName=core_node16, onlyIfActiveCheckResult=false, nodeProps: core_node16:{ 2> "core":"replacenodetest_coll_shard2_replica_n15", 2> "node_name":"127.0.0.1:35735_solr", 2> "base_url":"http://127.0.0.1:35735/solr", 2> "state":"recovering", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 3627189 INFO (qtp1247512449-10591) [n:127.0.0.1:46857_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=replacenodetest_coll, shard=shard2, thisCore=replacenodetest_coll_shard2_replica_n6, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:35735_solr, coreNodeName=core_node16, onlyIfActiveCheckResult=false, nodeProps: core_node16:{ 2> "core":"replacenodetest_coll_shard2_replica_n15", 2> "node_name":"127.0.0.1:35735_solr", 2> "base_url":"http://127.0.0.1:35735/solr", 2> "state":"recovering", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 3627190 INFO (qtp1247512449-10591) [n:127.0.0.1:46857_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=replacenodetest_coll, shard=shard2, thisCore=replacenodetest_coll_shard2_replica_n6, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:35735_solr, coreNodeName=core_node16, onlyIfActiveCheckResult=false, nodeProps: core_node16:{ 2> "core":"replacenodetest_coll_shard2_replica_n15", 2> "node_name":"127.0.0.1:35735_solr", 2> "base_url":"http://127.0.0.1:35735/solr", 2> "state":"recovering", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 3627190 INFO (qtp1247512449-10591) [n:127.0.0.1:46857_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:35735_solr&onlyIfLeaderActive=true&core=replacenodetest_coll_shard2_replica_n6&coreNodeName=core_node16&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=21 2> 3627706 INFO (recoveryExecutor-6837-thread-1-processing-replacenodetest_coll_shard2_replica_n15 000356776774238633 core_node16 create 127.0.0.1:35735_solr replacenodetest_coll shard2) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:46857/solr/replacenodetest_coll_shard2_replica_n6/] - recoveringAfterStartup=[true] 2> 3627710 WARN (recoveryExecutor-6837-thread-1-processing-replacenodetest_coll_shard2_replica_n15 000356776774238633 core_node16 create 127.0.0.1:35735_solr replacenodetest_coll shard2) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.u.PeerSyncWithLeader no frame of reference to tell if we've missed updates 2> 3627710 INFO (recoveryExecutor-6837-thread-1-processing-replacenodetest_coll_shard2_replica_n15 000356776774238633 core_node16 create 127.0.0.1:35735_solr replacenodetest_coll shard2) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.c.RecoveryStrategy PeerSync Recovery was not successful - trying replication. 2> 3627711 INFO (recoveryExecutor-6837-thread-1-processing-replacenodetest_coll_shard2_replica_n15 000356776774238633 core_node16 create 127.0.0.1:35735_solr replacenodetest_coll shard2) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.c.RecoveryStrategy Starting Replication Recovery. 2> 3627711 INFO (recoveryExecutor-6837-thread-1-processing-replacenodetest_coll_shard2_replica_n15 000356776774238633 core_node16 create 127.0.0.1:35735_solr replacenodetest_coll shard2) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.c.RecoveryStrategy Attempting to replicate from [http://127.0.0.1:46857/solr/replacenodetest_coll_shard2_replica_n6/]. 2> 3627885 INFO (qtp1124965268-10574) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46857/solr/replacenodetest_coll_shard1_replica_n5/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 62 2> 3627900 INFO (qtp1124965268-10573) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46857/solr/replacenodetest_coll_shard3_replica_n2/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 11 2> 3627915 INFO (qtp260799485-10566) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46857/solr/replacenodetest_coll_shard1_replica_n5/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 89 2> 3627917 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46857/solr/replacenodetest_coll_shard2_replica_n6/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 156 2> 3627921 INFO (qtp1247512449-10561) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46857/solr/replacenodetest_coll_shard2_replica_n6/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 167 2> 3627994 INFO (qtp1124965268-10579) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard2 r:core_node10 x:replacenodetest_coll_shard2_replica_n4] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46857/solr/replacenodetest_coll_shard2_replica_n6/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 66 2> 3628044 INFO (qtp61164958-10567) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.u.p.DistributedUpdateProcessor Ignoring commit while not ACTIVE - state: BUFFERING replay: false 2> 3628044 INFO (qtp61164958-10567) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46857/solr/replacenodetest_coll_shard2_replica_n6/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 31 2> 3628047 INFO (qtp1247512449-10560) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 322 2> 3628072 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 3628095 INFO (qtp61164958-10568) [n:127.0.0.1:35735_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 000356776774238633 2> 3628095 INFO (qtp61164958-10568) [n:127.0.0.1:35735_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=000356776774238633&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 3628108 INFO (recoveryExecutor-6837-thread-1-processing-replacenodetest_coll_shard2_replica_n15 000356776774238633 core_node16 create 127.0.0.1:35735_solr replacenodetest_coll shard2) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.h.IndexFetcher Leader's generation: 1 2> 3628108 INFO (recoveryExecutor-6837-thread-1-processing-replacenodetest_coll_shard2_replica_n15 000356776774238633 core_node16 create 127.0.0.1:35735_solr replacenodetest_coll shard2) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.h.IndexFetcher Leader's version: 0 2> 3628109 INFO (recoveryExecutor-6837-thread-1-processing-replacenodetest_coll_shard2_replica_n15 000356776774238633 core_node16 create 127.0.0.1:35735_solr replacenodetest_coll shard2) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.h.IndexFetcher Follower's generation: 1 2> 3628109 INFO (recoveryExecutor-6837-thread-1-processing-replacenodetest_coll_shard2_replica_n15 000356776774238633 core_node16 create 127.0.0.1:35735_solr replacenodetest_coll shard2) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.h.IndexFetcher Follower's version: 0 2> 3628109 INFO (recoveryExecutor-6837-thread-1-processing-replacenodetest_coll_shard2_replica_n15 000356776774238633 core_node16 create 127.0.0.1:35735_solr replacenodetest_coll shard2) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.c.RecoveryStrategy No replay needed. 2> 3628115 INFO (DistributedCollectionApiCommandExecutor-6872-thread-2-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr] o.a.s.c.a.c.ReplaceNodeCmd Going to create replica for collection=replacenodetest_coll shard=shard3 on node=null 2> 3628128 INFO (recoveryExecutor-6837-thread-1-processing-replacenodetest_coll_shard2_replica_n15 000356776774238633 core_node16 create 127.0.0.1:35735_solr replacenodetest_coll shard2) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.c.RecoveryStrategy Replication Recovery was successful. 2> 3628128 INFO (recoveryExecutor-6837-thread-1-processing-replacenodetest_coll_shard2_replica_n15 000356776774238633 core_node16 create 127.0.0.1:35735_solr replacenodetest_coll shard2) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.c.RecoveryStrategy Registering as Active after recovery. 2> 3628170 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628170 INFO (zkCallback-6838-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628172 INFO (recoveryExecutor-6837-thread-1-processing-replacenodetest_coll_shard2_replica_n15 000356776774238633 core_node16 create 127.0.0.1:35735_solr replacenodetest_coll shard2) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.c.RecoveryStrategy Updating version bucket highest from index after successful recovery. 2> 3628172 INFO (recoveryExecutor-6837-thread-1-processing-replacenodetest_coll_shard2_replica_n15 000356776774238633 core_node16 create 127.0.0.1:35735_solr replacenodetest_coll shard2) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723529609445376 2> 3628177 INFO (zkCallback-6838-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628178 INFO (zkCallback-6843-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628179 INFO (zkCallback-6849-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628188 INFO (zkCallback-6843-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628188 INFO (zkCallback-6843-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628192 INFO (zkCallback-6843-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628206 INFO (DistributedCollectionApiCommandExecutor-6872-thread-2-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:34951_solr for creating new replica of shard shard3 for collection replacenodetest_coll 2> 3628209 INFO (recoveryExecutor-6837-thread-1-processing-replacenodetest_coll_shard2_replica_n15 000356776774238633 core_node16 create 127.0.0.1:35735_solr replacenodetest_coll shard2) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.c.RecoveryStrategy Finished recovery process, successful=[true] msTimeTaken=1247.0 2> 3628209 INFO (recoveryExecutor-6837-thread-1-processing-replacenodetest_coll_shard2_replica_n15 000356776774238633 core_node16 create 127.0.0.1:35735_solr replacenodetest_coll shard2) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.c.RecoveryStrategy Finished recovery process. recoveringAfterStartup=true msTimeTaken=1247.0 2> 3628225 INFO (qtp260799485-10581) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=000&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=91 2> 3628225 INFO (zkCallback-6838-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628316 INFO (DistributedCollectionApiCommandExecutor-6872-thread-2-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command. 2> 3628377 INFO (DistributedCollectionApiCommandExecutor-6872-thread-2-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replacenodetest_coll_shard3_replica_n17", 2> "node_name":"127.0.0.1:34951_solr", 2> "base_url":"http://127.0.0.1:34951/solr", 2> "collection":"replacenodetest_coll", 2> "shard":"shard3", 2> "state":"down", 2> "type":"NRT", 2> "operation":"addreplica"} 2> 3628435 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628435 INFO (zkCallback-6838-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628439 INFO (zkCallback-6838-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628439 INFO (zkCallback-6843-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628439 INFO (zkCallback-6838-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628439 INFO (zkCallback-6843-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628440 INFO (zkCallback-6843-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628439 INFO (zkCallback-6843-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628448 INFO (zkCallback-6849-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628498 INFO (qtp260799485-10582) [n:127.0.0.1:34951_solr x:replacenodetest_coll_shard3_replica_n17] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=000356781419807301&qt=/admin/cores&coreNodeName=core_node18&collection.configName=conf1&name=replacenodetest_coll_shard3_replica_n17&action=CREATE&collection=replacenodetest_coll&shard=shard3&wt=javabin&version=2&replicaType=NRT} status=0 QTime=35 2> 3628506 INFO (qtp260799485-10583) [n:127.0.0.1:34951_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 000356781419807301 2> 3628507 INFO (qtp260799485-10583) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=000356781419807301&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 3628510 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-2-processing-127.0.0.1:34951_solr replacenodetest_coll_shard3_replica_n17 000356781419807301 create) [n:127.0.0.1:34951_solr x:replacenodetest_coll_shard3_replica_n17] o.a.s.h.a.CoreAdminOperation core create command async=000356781419807301&qt=/admin/cores&coreNodeName=core_node18&collection.configName=conf1&name=replacenodetest_coll_shard3_replica_n17&action=CREATE&collection=replacenodetest_coll&shard=shard3&wt=javabin&version=2&replicaType=NRT 2> 3628535 INFO (zkCallback-6838-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628536 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628537 INFO (zkCallback-6843-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628537 INFO (zkCallback-6843-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628537 INFO (zkCallback-6843-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628538 INFO (zkCallback-6849-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628537 INFO (zkCallback-6838-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628548 INFO (zkCallback-6838-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628548 INFO (zkCallback-6843-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3628629 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-2-processing-127.0.0.1:34951_solr replacenodetest_coll_shard3_replica_n17 000356781419807301 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 3628663 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-2-processing-127.0.0.1:34951_solr replacenodetest_coll_shard3_replica_n17 000356781419807301 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.s.IndexSchema Schema name=cloud-dynamic 2> 3628818 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-2-processing-127.0.0.1:34951_solr replacenodetest_coll_shard3_replica_n17 000356781419807301 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id 2> 3628821 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-2-processing-127.0.0.1:34951_solr replacenodetest_coll_shard3_replica_n17 000356781419807301 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_shard3_replica_n17' using configuration from configset conf1, trusted=true 2> 3628834 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-2-processing-127.0.0.1:34951_solr replacenodetest_coll_shard3_replica_n17 000356781419807301 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node2/replacenodetest_coll_shard3_replica_n17], dataDir=[/build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001/tempDir-003/node2/replacenodetest_coll_shard3_replica_n17/data/] 2> 3629296 INFO (qtp260799485-10565) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=000&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=41 2> 3629530 INFO (qtp260799485-10584) [n:127.0.0.1:34951_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 000356781419807301 2> 3629530 INFO (qtp260799485-10584) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=000356781419807301&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 3630187 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-2-processing-127.0.0.1:34951_solr replacenodetest_coll_shard3_replica_n17 000356781419807301 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 3630187 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-2-processing-127.0.0.1:34951_solr replacenodetest_coll_shard3_replica_n17 000356781419807301 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 3630207 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-2-processing-127.0.0.1:34951_solr replacenodetest_coll_shard3_replica_n17 000356781419807301 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 3630207 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-2-processing-127.0.0.1:34951_solr replacenodetest_coll_shard3_replica_n17 000356781419807301 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 3630230 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-2-processing-127.0.0.1:34951_solr replacenodetest_coll_shard3_replica_n17 000356781419807301 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 2> 3630233 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-2-processing-127.0.0.1:34951_solr replacenodetest_coll_shard3_replica_n17 000356781419807301 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 2> 3630240 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-2-processing-127.0.0.1:34951_solr replacenodetest_coll_shard3_replica_n17 000356781419807301 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 3630241 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-2-processing-127.0.0.1:34951_solr replacenodetest_coll_shard3_replica_n17 000356781419807301 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723531778949120 2> 3630280 INFO (searcherExecutor-6939-thread-1-processing-replacenodetest_coll_shard3_replica_n17 000356781419807301 core_node18 create 127.0.0.1:34951_solr replacenodetest_coll shard3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 3630413 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-2-processing-127.0.0.1:34951_solr replacenodetest_coll_shard3_replica_n17 000356781419807301 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replacenodetest_coll/terms/shard3 to Terms{values={core_node18=0, core_node7=0, core_node8=0}, version=2} for registerTerm 2> 3630416 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-2-processing-127.0.0.1:34951_solr replacenodetest_coll_shard3_replica_n17 000356781419807301 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replacenodetest_coll/leaders/shard3 2> 3630449 INFO (qtp260799485-10566) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=000&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=115 2> 3630529 INFO (parallelCoreAdminAPIBaseExecutor-6876-thread-2-processing-127.0.0.1:34951_solr replacenodetest_coll_shard3_replica_n17 000356781419807301 create) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.ZkController Core needs to recover:replacenodetest_coll_shard3_replica_n17 2> 3630530 INFO (updateExecutor-6832-thread-1-processing-replacenodetest_coll_shard3_replica_n17 000356781419807301 core_node18 create 127.0.0.1:34951_solr replacenodetest_coll shard3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.u.DefaultSolrCoreState Running recovery 2> 3630532 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard3_replica_n17 000356781419807301 core_node18 create 127.0.0.1:34951_solr replacenodetest_coll shard3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true 2> 3630532 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard3_replica_n17 000356781419807301 core_node18 create 127.0.0.1:34951_solr replacenodetest_coll shard3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.RecoveryStrategy startupVersions is empty 2> 3630542 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.c.S.Request webapp=/solr path=/admin/ping params={wt=javabin&version=2} hits=0 status=0 QTime=3 2> 3630542 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.c.S.Request webapp=/solr path=/admin/ping params={wt=javabin&version=2} status=0 QTime=3 2> 3630544 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard3_replica_n17 000356781419807301 core_node18 create 127.0.0.1:34951_solr replacenodetest_coll shard3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[replacenodetest_coll_shard3_replica_n17] 2> 3630547 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard3_replica_n17 000356781419807301 core_node18 create 127.0.0.1:34951_solr replacenodetest_coll shard3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null} 2> 3630547 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard3_replica_n17 000356781419807301 core_node18 create 127.0.0.1:34951_solr replacenodetest_coll shard3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.RecoveryStrategy Publishing state of core [replacenodetest_coll_shard3_replica_n17] as recovering, leader is [http://127.0.0.1:46857/solr/replacenodetest_coll_shard3_replica_n2/] and I am [http://127.0.0.1:34951/solr/replacenodetest_coll_shard3_replica_n17/] 2> 3630567 INFO (qtp260799485-10581) [n:127.0.0.1:34951_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 000356781419807301 2> 3630567 INFO (qtp260799485-10581) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=000356781419807301&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 3630612 INFO (zkCallback-6838-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3630612 INFO (zkCallback-6838-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3630616 INFO (zkCallback-6838-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3630621 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3630624 INFO (zkCallback-6843-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3630624 INFO (zkCallback-6843-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3630624 INFO (zkCallback-6843-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3630624 INFO (zkCallback-6843-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3630624 INFO (zkCallback-6849-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3630632 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard3_replica_n17 000356781419807301 core_node18 create 127.0.0.1:34951_solr replacenodetest_coll shard3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:46857/solr]; [WaitForState: action=PREPRECOVERY&core=replacenodetest_coll_shard3_replica_n2&nodeName=127.0.0.1:34951_solr&coreNodeName=core_node18&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true] 2> 3630648 INFO (qtp1247512449-10591) [n:127.0.0.1:46857_solr x:replacenodetest_coll_shard3_replica_n2] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node18, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true 2> 3630649 INFO (qtp1247512449-10591) [n:127.0.0.1:46857_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=replacenodetest_coll, shard=shard3, thisCore=replacenodetest_coll_shard3_replica_n2, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:34951_solr, coreNodeName=core_node18, onlyIfActiveCheckResult=false, nodeProps: core_node18:{ 2> "core":"replacenodetest_coll_shard3_replica_n17", 2> "node_name":"127.0.0.1:34951_solr", 2> "base_url":"http://127.0.0.1:34951/solr", 2> "state":"down", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 3630650 INFO (qtp1247512449-10591) [n:127.0.0.1:46857_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=replacenodetest_coll, shard=shard3, thisCore=replacenodetest_coll_shard3_replica_n2, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:34951_solr, coreNodeName=core_node18, onlyIfActiveCheckResult=false, nodeProps: core_node18:{ 2> "core":"replacenodetest_coll_shard3_replica_n17", 2> "node_name":"127.0.0.1:34951_solr", 2> "base_url":"http://127.0.0.1:34951/solr", 2> "state":"down", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 3630651 INFO (qtp1247512449-10591) [n:127.0.0.1:46857_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=replacenodetest_coll, shard=shard3, thisCore=replacenodetest_coll_shard3_replica_n2, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:34951_solr, coreNodeName=core_node18, onlyIfActiveCheckResult=false, nodeProps: core_node18:{ 2> "core":"replacenodetest_coll_shard3_replica_n17", 2> "node_name":"127.0.0.1:34951_solr", 2> "base_url":"http://127.0.0.1:34951/solr", 2> "state":"down", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 3630663 INFO (watches-6853-thread-1) [n:127.0.0.1:46857_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=replacenodetest_coll, shard=shard3, thisCore=replacenodetest_coll_shard3_replica_n2, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:34951_solr, coreNodeName=core_node18, onlyIfActiveCheckResult=false, nodeProps: core_node18:{ 2> "core":"replacenodetest_coll_shard3_replica_n17", 2> "node_name":"127.0.0.1:34951_solr", 2> "base_url":"http://127.0.0.1:34951/solr", 2> "state":"recovering", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 3630664 INFO (qtp1247512449-10591) [n:127.0.0.1:46857_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:34951_solr&onlyIfLeaderActive=true&core=replacenodetest_coll_shard3_replica_n2&coreNodeName=core_node18&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=18 2> 3631174 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard3_replica_n17 000356781419807301 core_node18 create 127.0.0.1:34951_solr replacenodetest_coll shard3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:46857/solr/replacenodetest_coll_shard3_replica_n2/] - recoveringAfterStartup=[true] 2> 3631178 WARN (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard3_replica_n17 000356781419807301 core_node18 create 127.0.0.1:34951_solr replacenodetest_coll shard3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.u.PeerSyncWithLeader no frame of reference to tell if we've missed updates 2> 3631179 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard3_replica_n17 000356781419807301 core_node18 create 127.0.0.1:34951_solr replacenodetest_coll shard3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.RecoveryStrategy PeerSync Recovery was not successful - trying replication. 2> 3631179 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard3_replica_n17 000356781419807301 core_node18 create 127.0.0.1:34951_solr replacenodetest_coll shard3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.RecoveryStrategy Starting Replication Recovery. 2> 3631179 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard3_replica_n17 000356781419807301 core_node18 create 127.0.0.1:34951_solr replacenodetest_coll shard3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.RecoveryStrategy Attempting to replicate from [http://127.0.0.1:46857/solr/replacenodetest_coll_shard3_replica_n2/]. 2> 3631218 INFO (qtp1124965268-10572) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard2 r:core_node10 x:replacenodetest_coll_shard2_replica_n4] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46857/solr/replacenodetest_coll_shard2_replica_n6/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 7 2> 3631219 INFO (qtp61164958-10569) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46857/solr/replacenodetest_coll_shard2_replica_n6/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 8 2> 3631225 INFO (qtp1247512449-10561) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46857/solr/replacenodetest_coll_shard3_replica_n2/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 26 2> 3631237 INFO (qtp1124965268-10575) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46857/solr/replacenodetest_coll_shard1_replica_n5/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 4 2> 3631237 INFO (qtp260799485-10582) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46857/solr/replacenodetest_coll_shard1_replica_n5/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 4 2> 3631249 INFO (qtp1247512449-10560) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard1 r:core_node11 x:replacenodetest_coll_shard1_replica_n5] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46857/solr/replacenodetest_coll_shard3_replica_n2/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 48 2> 3631260 INFO (qtp1124965268-10578) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46857/solr/replacenodetest_coll_shard3_replica_n2/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 4 2> 3631263 INFO (qtp260799485-10583) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.u.p.DistributedUpdateProcessor Ignoring commit while not ACTIVE - state: BUFFERING replay: false 2> 3631263 INFO (qtp260799485-10583) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:46857/solr/replacenodetest_coll_shard3_replica_n2/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 4 2> 3631270 INFO (qtp1247512449-10562) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 82 2> 3631291 INFO (qtp1247512449-10588) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=2 2> 3631296 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard3_replica_n17 000356781419807301 core_node18 create 127.0.0.1:34951_solr replacenodetest_coll shard3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.h.IndexFetcher Leader's generation: 1 2> 3631297 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard3_replica_n17 000356781419807301 core_node18 create 127.0.0.1:34951_solr replacenodetest_coll shard3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.h.IndexFetcher Leader's version: 0 2> 3631298 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard3_replica_n17 000356781419807301 core_node18 create 127.0.0.1:34951_solr replacenodetest_coll shard3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.h.IndexFetcher Follower's generation: 1 2> 3631299 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard3_replica_n17 000356781419807301 core_node18 create 127.0.0.1:34951_solr replacenodetest_coll shard3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.h.IndexFetcher Follower's version: 0 2> 3631300 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard3_replica_n17 000356781419807301 core_node18 create 127.0.0.1:34951_solr replacenodetest_coll shard3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.RecoveryStrategy No replay needed. 2> 3631337 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard3_replica_n17 000356781419807301 core_node18 create 127.0.0.1:34951_solr replacenodetest_coll shard3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.RecoveryStrategy Replication Recovery was successful. 2> 3631337 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard3_replica_n17 000356781419807301 core_node18 create 127.0.0.1:34951_solr replacenodetest_coll shard3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.RecoveryStrategy Registering as Active after recovery. 2> 3631420 INFO (zkCallback-6838-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3631420 INFO (zkCallback-6838-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3631420 INFO (zkCallback-6838-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3631420 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3631424 INFO (zkCallback-6849-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3631434 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard3_replica_n17 000356781419807301 core_node18 create 127.0.0.1:34951_solr replacenodetest_coll shard3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.RecoveryStrategy Updating version bucket highest from index after successful recovery. 2> 3631434 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard3_replica_n17 000356781419807301 core_node18 create 127.0.0.1:34951_solr replacenodetest_coll shard3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778723533029900288 2> 3631436 INFO (zkCallback-6843-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3631436 INFO (zkCallback-6843-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3631436 INFO (zkCallback-6843-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3631437 INFO (zkCallback-6843-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [4]) 2> 3631496 INFO (TEST-ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Ending testGoodSpreadDuringAssignWithNoTarget 2> 3631516 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard3_replica_n17 000356781419807301 core_node18 create 127.0.0.1:34951_solr replacenodetest_coll shard3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.RecoveryStrategy Finished recovery process, successful=[true] msTimeTaken=983.0 2> 3631516 INFO (recoveryExecutor-6842-thread-1-processing-replacenodetest_coll_shard3_replica_n17 000356781419807301 core_node18 create 127.0.0.1:34951_solr replacenodetest_coll shard3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.RecoveryStrategy Finished recovery process. recoveringAfterStartup=true msTimeTaken=984.0 > java.lang.AssertionError: There should be no more replicas on the sourceNode after a replaceNode request. expected:<[]> but was:<[core_node11:{ > "core":"replacenodetest_coll_shard1_replica_n5", > "leader":"true", > "node_name":"127.0.0.1:46857_solr", > "base_url":"http://127.0.0.1:46857/solr", > "state":"active", > "type":"NRT", > "force_set_state":"false"}, core_node12:{ > "core":"replacenodetest_coll_shard2_replica_n6", > "leader":"true", > "node_name":"127.0.0.1:46857_solr", > "base_url":"http://127.0.0.1:46857/solr", > "state":"active", > "type":"NRT", > "force_set_state":"false"}, core_node8:{ > "core":"replacenodetest_coll_shard3_replica_n2", > "leader":"true", > "node_name":"127.0.0.1:46857_solr", > "base_url":"http://127.0.0.1:46857/solr", > "state":"active", > "type":"NRT", > "force_set_state":"false"}]> > at __randomizedtesting.SeedInfo.seed([9CAA27B60427D55F:B2C1F38C3144042B]:0) > at org.junit.Assert.fail(Assert.java:89) > at org.junit.Assert.failNotEquals(Assert.java:835) > at org.junit.Assert.assertEquals(Assert.java:120) > at org.apache.solr.cloud.ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget(ReplaceNodeTest.java:274) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) > at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) > at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) > at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) > at org.junit.rules.RunRules.evaluate(RunRules.java:20) > at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:44) > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) > at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) > at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) > at org.junit.rules.RunRules.evaluate(RunRules.java:20) > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) > at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) > at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) > at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) > at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) > at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) > at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) > at org.junit.rules.RunRules.evaluate(RunRules.java:20) > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) > at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) > at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) > at org.junit.rules.RunRules.evaluate(RunRules.java:20) > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) > at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) > at java.base/java.lang.Thread.run(Thread.java:829) 2> NOTE: reproduce with: gradlew test --tests ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget -Dtests.seed=9CAA27B60427D55F -Dtests.badapples=true -Dtests.locale=en-GH -Dtests.timezone=Iran -Dtests.asserts=true -Dtests.file.encoding=UTF-8 2> 3631607 INFO (qtp260799485-10565) [n:127.0.0.1:34951_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 000356781419807301 2> 3631607 INFO (qtp260799485-10565) [n:127.0.0.1:34951_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=000356781419807301&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 3631634 INFO (DistributedCollectionApiCommandExecutor-6872-thread-2-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr] o.a.s.c.a.c.DeleteNodeCmd Deleting replica type=NRT for collection=replacenodetest_coll shard=shard1 on node=127.0.0.1:46857_solr 2> 3631725 INFO (jetty-closer-6945-thread-1) [] o.e.j.s.Server Stopped Server@7dd95547{STOPPING}[10.0.13,sto=0] 2> 3631729 INFO (jetty-closer-6945-thread-1) [] o.e.j.s.AbstractConnector Stopped ServerConnector@d62b39a{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0} 2> 3631730 INFO (jetty-closer-6945-thread-1) [] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@21eaa628{/solr,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,STOPPED} 2> 3631746 INFO (jetty-closer-6945-thread-2) [] o.e.j.s.Server Stopped Server@34eea94b{STOPPING}[10.0.13,sto=0] 2> 3631748 INFO (qtp1247512449-10563) [n:127.0.0.1:46857_solr x:replacenodetest_coll_shard1_replica_n5] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=replacenodetest_coll_shard1_replica_n5&async=000356784677406883&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2&deleteIndex=true} status=0 QTime=27 2> 3631751 INFO (parallelCoreAdminAPIBaseExecutor-6874-thread-1-processing-127.0.0.1:46857_solr replacenodetest_coll_shard1_replica_n5 000356784677406883 unload) [n:127.0.0.1:46857_solr x:replacenodetest_coll_shard1_replica_n5] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replacenodetest_coll.shard1.replica_n5 tag=null 2> 3631755 INFO (jetty-closer-6945-thread-2) [] o.e.j.s.AbstractConnector Stopped ServerConnector@449c60ba{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0} 2> 3631757 INFO (jetty-closer-6945-thread-2) [] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@668605fe{/solr,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,STOPPED} 2> 3631761 INFO (DistributedCollectionApiCommandExecutor-6872-thread-2-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr] o.a.s.c.a.c.DeleteNodeCmd Deleting replica type=NRT for collection=replacenodetest_coll shard=shard2 on node=127.0.0.1:46857_solr 2> 3631905 INFO (jetty-closer-6945-thread-3) [] o.e.j.s.Server Stopped Server@3db1fb2e{STOPPING}[10.0.13,sto=0] 2> 3631914 INFO (jetty-closer-6945-thread-3) [] o.e.j.s.AbstractConnector Stopped ServerConnector@4dca7d85{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0} 2> 3631915 INFO (jetty-closer-6945-thread-3) [] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@24027d94{/solr,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,STOPPED} 2> 3631940 INFO (jetty-closer-6945-thread-4) [] o.e.j.s.Server Stopped Server@71fb48b1{STOPPING}[10.0.13,sto=0] 2> 3631969 INFO (jetty-closer-6945-thread-4) [] o.e.j.s.AbstractConnector Stopped ServerConnector@14c847c{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0} 2> 3631983 INFO (jetty-closer-6945-thread-4) [] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2fcd53f4{/solr,file:///build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-cwd/,STOPPED} 2> 3632090 INFO (DistributedCollectionApiCommandExecutor-6872-thread-2-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr] o.a.s.c.a.c.DeleteNodeCmd Deleting replica type=NRT for collection=replacenodetest_coll shard=shard3 on node=127.0.0.1:46857_solr 2> 3632170 INFO (jetty-closer-6945-thread-2) [] o.a.s.c.CoreContainer Shutting down CoreContainer instance=450436536 2> 3632170 INFO (jetty-closer-6945-thread-2) [] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:46857_solr 2> 3632171 INFO (jetty-closer-6945-thread-3) [] o.a.s.c.CoreContainer Shutting down CoreContainer instance=273819679 2> 3632204 INFO (jetty-closer-6945-thread-3) [] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:36667_solr 2> 3632216 INFO (jetty-closer-6945-thread-2) [] o.a.s.c.ZkController Publish this node as DOWN... 2> 3632216 INFO (jetty-closer-6945-thread-2) [] o.a.s.c.ZkController Publish node=127.0.0.1:46857_solr as DOWN 2> 3632257 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (2) 2> 3632258 INFO (jetty-closer-6945-thread-3) [] o.a.s.c.ZkController Publish this node as DOWN... 2> 3632258 INFO (jetty-closer-6945-thread-3) [] o.a.s.c.ZkController Publish node=127.0.0.1:36667_solr as DOWN 2> 3632297 INFO (zkCallback-6838-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (2) 2> 3632215 ERROR (DistributedCollectionApiExecutorService-6871-thread-2-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr] o.a.s.c.a.c.CollectionHandlingUtils Error from shard: http://127.0.0.1:46857/solr 2> => org.apache.solr.client.solrj.SolrServerException: Connection refused 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) 2> org.apache.solr.client.solrj.SolrServerException: Connection refused 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) ~[solr-solrj-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:192) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:184) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpExchange.notifyFailureComplete(HttpExchange.java:284) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpExchange.abort(HttpExchange.java:256) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpConversation.abort(HttpConversation.java:159) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpRequest.abort(HttpRequest.java:865) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpDestination.abort(HttpDestination.java:530) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpDestination.failed(HttpDestination.java:294) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.AbstractConnectionPool$FutureConnection.failed(AbstractConnectionPool.java:574) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.Promise$Wrapper.failed(Promise.java:169) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1$1.failed(HttpClient.java:628) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HTTPSessionListenerPromise.failConnectionPromise(HTTPSessionListenerPromise.java:136) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HTTPSessionListenerPromise.failed(HTTPSessionListenerPromise.java:51) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.Promise$1.failed(Promise.java:94) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.ClientConnector.connectFailed(ClientConnector.java:539) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.ClientConnector.connect(ClientConnector.java:455) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.HTTP2Client.connect(HTTP2Client.java:406) ~[http2-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2.connect(HttpClientTransportOverHTTP2.java:146) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2.connect(HttpClientTransportOverHTTP2.java:135) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.connect(HttpClient.java:633) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.succeeded(HttpClient.java:610) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.succeeded(HttpClient.java:606) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.SocketAddressResolver$Async.lambda$resolve$1(SocketAddressResolver.java:180) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289) [solr-solrj-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> Caused by: java.net.ConnectException: Connection refused 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:774) ~[?:?] 2> at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:120) ~[?:?] 2> at org.eclipse.jetty.io.ClientConnector.connect(ClientConnector.java:427) ~[jetty-io-10.0.13.jar:10.0.13] 2> ... 11 more 2> 3632170 ERROR (DistributedCollectionApiExecutorService-6871-thread-1-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr] o.a.s.c.a.c.CollectionHandlingUtils Error from shard: http://127.0.0.1:46857/solr 2> => org.apache.solr.client.solrj.SolrServerException: Connection refused 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) 2> org.apache.solr.client.solrj.SolrServerException: Connection refused 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) ~[solr-solrj-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:192) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:184) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpExchange.notifyFailureComplete(HttpExchange.java:284) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpExchange.abort(HttpExchange.java:256) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpConversation.abort(HttpConversation.java:159) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpRequest.abort(HttpRequest.java:865) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpDestination.abort(HttpDestination.java:530) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpDestination.failed(HttpDestination.java:294) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.AbstractConnectionPool$FutureConnection.failed(AbstractConnectionPool.java:574) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.Promise$Wrapper.failed(Promise.java:169) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1$1.failed(HttpClient.java:628) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HTTPSessionListenerPromise.failConnectionPromise(HTTPSessionListenerPromise.java:136) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HTTPSessionListenerPromise.failed(HTTPSessionListenerPromise.java:51) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.Promise$1.failed(Promise.java:94) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.ClientConnector.connectFailed(ClientConnector.java:539) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.ClientConnector.connect(ClientConnector.java:455) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.HTTP2Client.connect(HTTP2Client.java:406) ~[http2-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2.connect(HttpClientTransportOverHTTP2.java:146) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2.connect(HttpClientTransportOverHTTP2.java:135) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.connect(HttpClient.java:633) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.succeeded(HttpClient.java:610) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.succeeded(HttpClient.java:606) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.SocketAddressResolver$Async.lambda$resolve$1(SocketAddressResolver.java:180) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289) [solr-solrj-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> Caused by: java.net.ConnectException: Connection refused 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:774) ~[?:?] 2> at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:120) ~[?:?] 2> at org.eclipse.jetty.io.ClientConnector.connect(ClientConnector.java:427) ~[jetty-io-10.0.13.jar:10.0.13] 2> ... 11 more 2> 3632371 ERROR (DistributedCollectionApiExecutorService-6871-thread-1-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr] o.a.s.c.a.c.CollectionHandlingUtils Error from shard 127.0.0.1:46857_solr: {STATUS=failed} 2> 3632373 INFO (zkCallback-6849-thread-2) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (2) 2> 3632473 INFO (zkCallback-6843-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (2) 2> 3632455 WARN (DistributedCollectionApiCommandExecutor-6872-thread-2-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr] o.a.s.c.a.c.DeleteNodeCmd Error deleting 2> => java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$338/0x00000001005b7040@34c123c7 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@7771b2ba[Shutting down, pool size = 2, active threads = 2, queued tasks = 0, completed tasks = 0] 2> at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2055) 2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$338/0x00000001005b7040@34c123c7 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@7771b2ba[Shutting down, pool size = 2, active threads = 2, queued tasks = 0, completed tasks = 0] 2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2055) ~[?:?] 2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:825) ~[?:?] 2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1355) ~[?:?] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:270) ~[solr-solrj-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140) ~[?:?] 2> at org.apache.solr.cloud.api.collections.DeleteReplicaCmd.deleteCore(DeleteReplicaCmd.java:359) ~[main/:?] 2> at org.apache.solr.cloud.api.collections.DeleteReplicaCmd.deleteReplica(DeleteReplicaCmd.java:107) ~[main/:?] 2> at org.apache.solr.cloud.api.collections.DeleteNodeCmd.cleanupReplicas(DeleteNodeCmd.java:136) [main/:?] 2> at org.apache.solr.cloud.api.collections.ReplaceNodeCmd.call(ReplaceNodeCmd.java:293) [main/:?] 2> at org.apache.solr.cloud.api.collections.DistributedCollectionConfigSetCommandRunner$CollectionCommandRunner.call(DistributedCollectionConfigSetCommandRunner.java:431) [main/:?] 2> at org.apache.solr.cloud.api.collections.DistributedCollectionConfigSetCommandRunner$CollectionCommandRunner.call(DistributedCollectionConfigSetCommandRunner.java:364) [main/:?] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289) [solr-solrj-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> 3632493 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [2]) 2> 3632494 INFO (zkCallback-6838-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [2]) 2> 3632494 INFO (zkCallback-6838-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [2]) 2> 3632494 INFO (zkCallback-6843-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [2]) 2> 3632494 INFO (zkCallback-6843-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [2]) 2> 3632494 INFO (zkCallback-6843-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [2]) 2> 3632495 INFO (zkCallback-6843-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [2]) 2> 3632495 INFO (zkCallback-6849-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [2]) 2> 3632493 INFO (zkCallback-6838-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [2]) 2> 3632371 ERROR (DistributedCollectionApiExecutorService-6871-thread-2-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr] o.a.s.c.a.c.CollectionHandlingUtils Error from shard: http://127.0.0.1:46857/solr 2> => org.apache.solr.client.solrj.SolrServerException: Connection refused 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) 2> org.apache.solr.client.solrj.SolrServerException: Connection refused 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) ~[solr-solrj-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:192) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:184) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpExchange.notifyFailureComplete(HttpExchange.java:284) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpExchange.abort(HttpExchange.java:256) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpConversation.abort(HttpConversation.java:159) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpRequest.abort(HttpRequest.java:865) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpDestination.abort(HttpDestination.java:530) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpDestination.failed(HttpDestination.java:294) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.AbstractConnectionPool$FutureConnection.failed(AbstractConnectionPool.java:574) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.Promise$Wrapper.failed(Promise.java:169) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1$1.failed(HttpClient.java:628) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HTTPSessionListenerPromise.failConnectionPromise(HTTPSessionListenerPromise.java:136) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HTTPSessionListenerPromise.failed(HTTPSessionListenerPromise.java:51) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.Promise$1.failed(Promise.java:94) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.ClientConnector.connectFailed(ClientConnector.java:539) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.ClientConnector.connect(ClientConnector.java:455) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.HTTP2Client.connect(HTTP2Client.java:406) ~[http2-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2.connect(HttpClientTransportOverHTTP2.java:146) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2.connect(HttpClientTransportOverHTTP2.java:135) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.connect(HttpClient.java:633) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.succeeded(HttpClient.java:610) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.succeeded(HttpClient.java:606) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.SocketAddressResolver$Async.lambda$resolve$1(SocketAddressResolver.java:180) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289) [solr-solrj-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> Caused by: java.net.ConnectException: Connection refused 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:774) ~[?:?] 2> at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:120) ~[?:?] 2> at org.eclipse.jetty.io.ClientConnector.connect(ClientConnector.java:427) ~[jetty-io-10.0.13.jar:10.0.13] 2> ... 11 more 2> 3632564 INFO (zkCallback-6838-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [2]) 2> 3632564 ERROR (DistributedCollectionApiExecutorService-6871-thread-2-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr] o.a.s.c.a.c.CollectionHandlingUtils Error from shard 127.0.0.1:46857_solr: {STATUS=failed} 2> 3632550 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [2]) 2> 3632556 INFO (zkCallback-6838-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [2]) 2> 3632554 INFO (zkCallback-6838-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [2]) 2> 3632552 INFO (zkCallback-6843-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [2]) 2> 3632552 INFO (zkCallback-6843-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [2]) 2> 3632552 INFO (zkCallback-6843-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [2]) 2> 3632552 INFO (zkCallback-6843-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [2]) 2> 3632550 INFO (zkCallback-6849-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [2]) 2> 3632484 ERROR (DistributedCollectionApiCommandExecutor-6872-thread-2-processing-127.0.0.1:34951_solr) [n:127.0.0.1:34951_solr] o.a.s.c.a.c.DistributedCollectionConfigSetCommandRunner Operation REPLACENODE failed 2> => java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$338/0x00000001005b7040@34c123c7 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@7771b2ba[Shutting down, pool size = 2, active threads = 2, queued tasks = 0, completed tasks = 0] 2> at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2055) 2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$338/0x00000001005b7040@34c123c7 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@7771b2ba[Shutting down, pool size = 2, active threads = 2, queued tasks = 0, completed tasks = 0] 2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2055) ~[?:?] 2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:825) ~[?:?] 2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1355) ~[?:?] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:270) ~[solr-solrj-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140) ~[?:?] 2> at org.apache.solr.cloud.api.collections.DeleteReplicaCmd.deleteCore(DeleteReplicaCmd.java:359) ~[main/:?] 2> at org.apache.solr.cloud.api.collections.DeleteReplicaCmd.deleteReplica(DeleteReplicaCmd.java:107) ~[main/:?] 2> at org.apache.solr.cloud.api.collections.DeleteNodeCmd.cleanupReplicas(DeleteNodeCmd.java:136) ~[main/:?] 2> at org.apache.solr.cloud.api.collections.ReplaceNodeCmd.call(ReplaceNodeCmd.java:293) ~[main/:?] 2> at org.apache.solr.cloud.api.collections.DistributedCollectionConfigSetCommandRunner$CollectionCommandRunner.call(DistributedCollectionConfigSetCommandRunner.java:431) [main/:?] 2> at org.apache.solr.cloud.api.collections.DistributedCollectionConfigSetCommandRunner$CollectionCommandRunner.call(DistributedCollectionConfigSetCommandRunner.java:364) [main/:?] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289) [solr-solrj-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> 3632717 INFO (coreCloseExecutor-6951-thread-1) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@ad3e568 replacenodetest_coll_shard3_replica_n2 2> 3632739 INFO (coreCloseExecutor-6951-thread-2) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@42141f3 replacenodetest_coll_shard2_replica_n6 2> 3632787 INFO (coreCloseExecutor-6952-thread-1) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@4680806a replacenodetest_coll_shard1_replica_n3 2> 3632828 INFO (coreCloseExecutor-6952-thread-2) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@54e18d6f replacenodetest_coll_shard3_replica_n1 2> 3632818 INFO (coreCloseExecutor-6952-thread-1) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replacenodetest_coll.shard1.replica_n3 tag=SolrCore@4680806a 2> 3632922 INFO (coreCloseExecutor-6952-thread-3) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard2 r:core_node10 x:replacenodetest_coll_shard2_replica_n4] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@6f296dec replacenodetest_coll_shard2_replica_n4 2> 3633577 INFO (jetty-closer-6945-thread-1) [] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2034559573 2> 3633587 INFO (jetty-closer-6945-thread-1) [] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:35735_solr 2> 3633615 INFO (jetty-closer-6945-thread-1) [] o.a.s.c.ZkController Publish this node as DOWN... 2> 3633615 INFO (jetty-closer-6945-thread-1) [] o.a.s.c.ZkController Publish node=127.0.0.1:35735_solr as DOWN 2> 3633639 INFO (zkCallback-6838-thread-3) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1) 2> 3633639 INFO (zkCallback-6849-thread-2) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1) 2> 3633640 INFO (zkCallback-6843-thread-4) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1) 2> 3633645 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1) 2> 3633759 INFO (zkCallback-6843-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3633759 INFO (zkCallback-6843-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3633759 INFO (zkCallback-6838-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3633759 INFO (zkCallback-6843-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3633760 INFO (zkCallback-6838-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3633761 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3633769 INFO (zkCallback-6849-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3633769 INFO (zkCallback-6838-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3633770 INFO (zkCallback-6843-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3633796 INFO (parallelCoreAdminAPIBaseExecutor-6874-thread-1-processing-127.0.0.1:46857_solr replacenodetest_coll_shard1_replica_n5 000356784677406883 unload) [n:127.0.0.1:46857_solr x:replacenodetest_coll_shard1_replica_n5] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@7ab3974c replacenodetest_coll_shard1_replica_n5 2> 3633797 INFO (parallelCoreAdminAPIBaseExecutor-6874-thread-1-processing-127.0.0.1:46857_solr replacenodetest_coll_shard1_replica_n5 000356784677406883 unload) [n:127.0.0.1:46857_solr x:replacenodetest_coll_shard1_replica_n5] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replacenodetest_coll.shard1.replica_n5 tag=SolrCore@7ab3974c 2> 3633798 INFO (parallelCoreAdminAPIBaseExecutor-6874-thread-1-processing-127.0.0.1:46857_solr replacenodetest_coll_shard1_replica_n5 000356784677406883 unload) [n:127.0.0.1:46857_solr x:replacenodetest_coll_shard1_replica_n5] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replacenodetest_coll.shard1.leader tag=SolrCore@7ab3974c 2> 3633876 INFO (parallelCoreAdminAPIBaseExecutor-6874-thread-1-processing-127.0.0.1:46857_solr replacenodetest_coll_shard1_replica_n5 000356784677406883 unload) [n:127.0.0.1:46857_solr x:replacenodetest_coll_shard1_replica_n5] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 3633904 INFO (coreCloseExecutor-6951-thread-1) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replacenodetest_coll.shard3.replica_n2 tag=SolrCore@ad3e568 2> 3633904 INFO (coreCloseExecutor-6955-thread-1) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@71ad9534 replacenodetest_coll_shard2_replica_n15 2> 3633906 INFO (coreCloseExecutor-6955-thread-1) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replacenodetest_coll.shard2.replica_n15 tag=SolrCore@71ad9534 2> 3634101 INFO (parallelCoreAdminAPIBaseExecutor-6874-thread-1-processing-127.0.0.1:46857_solr replacenodetest_coll_shard1_replica_n5 000356784677406883 unload) [n:127.0.0.1:46857_solr] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replacenodetest_coll/terms/shard1 to Terms{values={core_node9=0, core_node14=0}, version=3} for removeTerm 2> 3634261 INFO (zkCallback-6838-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3634263 INFO (zkCallback-6846-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3634268 INFO (zkCallback-6838-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3634268 INFO (zkCallback-6838-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3634270 INFO (zkCallback-6843-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3634270 INFO (zkCallback-6843-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3634270 INFO (zkCallback-6849-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3634270 INFO (zkCallback-6843-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3634369 INFO (zkCallback-6843-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3635838 INFO (coreCloseExecutor-6952-thread-1) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replacenodetest_coll.shard1.leader tag=SolrCore@4680806a 2> 3635840 INFO (coreCloseExecutor-6952-thread-2) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replacenodetest_coll.shard3.replica_n1 tag=SolrCore@54e18d6f 2> 3635866 INFO (coreCloseExecutor-6952-thread-1) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard1 r:core_node9 x:replacenodetest_coll_shard1_replica_n3] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 3637167 INFO (coreCloseExecutor-6951-thread-1) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replacenodetest_coll.shard3.leader tag=SolrCore@ad3e568 2> 3637172 INFO (coreCloseExecutor-6951-thread-2) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replacenodetest_coll.shard2.replica_n6 tag=SolrCore@42141f3 2> 3637199 INFO (coreCloseExecutor-6951-thread-1) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard3 r:core_node8 x:replacenodetest_coll_shard3_replica_n2] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 3637455 INFO (coreCloseExecutor-6955-thread-1) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replacenodetest_coll.shard2.leader tag=SolrCore@71ad9534 2> 3637487 INFO (coreCloseExecutor-6955-thread-1) [n:127.0.0.1:35735_solr c:replacenodetest_coll s:shard2 r:core_node16 x:replacenodetest_coll_shard2_replica_n15] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 3637760 INFO (jetty-closer-6945-thread-1) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null 2> 3638494 INFO (coreCloseExecutor-6952-thread-2) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replacenodetest_coll.shard3.leader tag=SolrCore@54e18d6f 2> 3638495 INFO (coreCloseExecutor-6952-thread-3) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard2 r:core_node10 x:replacenodetest_coll_shard2_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replacenodetest_coll.shard2.replica_n4 tag=SolrCore@6f296dec 2> 3638502 INFO (coreCloseExecutor-6952-thread-2) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard3 r:core_node7 x:replacenodetest_coll_shard3_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 3638670 INFO (jetty-closer-6945-thread-1) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null 2> 3639076 INFO (coreCloseExecutor-6951-thread-2) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replacenodetest_coll.shard2.leader tag=SolrCore@42141f3 2> 3639104 INFO (coreCloseExecutor-6951-thread-2) [n:127.0.0.1:46857_solr c:replacenodetest_coll s:shard2 r:core_node12 x:replacenodetest_coll_shard2_replica_n6] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 3639184 INFO (jetty-closer-6945-thread-2) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null 2> 3639209 INFO (jetty-closer-6945-thread-1) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null 2> 3639209 INFO (jetty-closer-6945-thread-1) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null 2> 3639805 INFO (coreCloseExecutor-6952-thread-3) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard2 r:core_node10 x:replacenodetest_coll_shard2_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replacenodetest_coll.shard2.leader tag=SolrCore@6f296dec 2> 3639823 INFO (coreCloseExecutor-6952-thread-3) [n:127.0.0.1:36667_solr c:replacenodetest_coll s:shard2 r:core_node10 x:replacenodetest_coll_shard2_replica_n4] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 3639952 INFO (jetty-closer-6945-thread-3) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null 2> 3639964 INFO (jetty-closer-6945-thread-2) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null 2> 3640446 INFO (jetty-closer-6945-thread-3) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null 2> 3640483 INFO (jetty-closer-6945-thread-2) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null 2> 3640484 INFO (jetty-closer-6945-thread-2) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null 2> 3640530 INFO (closeThreadPool-6957-thread-2) [] o.a.s.c.Overseer Overseer (id=72080973716586505-127.0.0.1:46857_solr-n_0000000000) closing 2> 3640531 INFO (OverseerStateUpdate-72080973716586505-127.0.0.1:46857_solr-n_0000000000) [n:127.0.0.1:46857_solr] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:46857_solr 2> 3640614 INFO (zkCallback-6843-thread-3) [] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34951_solr 2> 3640644 INFO (closeThreadPool-6957-thread-1) [] o.a.s.c.Overseer Overseer (id=72080973716586505-127.0.0.1:46857_solr-n_0000000000) closing 2> 3640644 INFO (zkCallback-6843-thread-3) [n:127.0.0.1:34951_solr] o.a.s.c.Overseer Overseer (id=72080973716586504-127.0.0.1:34951_solr-n_0000000001) starting 2> 3640706 INFO (jetty-closer-6945-thread-2) [] o.a.s.c.Overseer Overseer (id=72080973716586505-127.0.0.1:46857_solr-n_0000000000) closing 2> 3640803 INFO (jetty-closer-6945-thread-3) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null 2> 3640803 INFO (jetty-closer-6945-thread-3) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null 2> 3640829 INFO (OverseerStateUpdate-72080973716586504-127.0.0.1:34951_solr-n_0000000001) [n:127.0.0.1:34951_solr] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:34951_solr 2> 3640967 INFO (zkCallback-6843-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3640968 INFO (zkCallback-6843-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3640980 INFO (zkCallback-6843-thread-5) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3640993 INFO (zkCallback-6843-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3641034 INFO (zkCallback-6843-thread-3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 3641069 INFO (zkCallback-6843-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3641069 INFO (zkCallback-6843-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3641070 INFO (zkCallback-6843-thread-5) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3641070 INFO (zkCallback-6843-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [1]) 2> 3641076 INFO (zkCallback-6843-thread-1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 3642214 INFO (jetty-closer-6945-thread-4) [] o.a.s.c.CoreContainer Shutting down CoreContainer instance=365480551 2> 3642214 INFO (jetty-closer-6945-thread-4) [] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:34951_solr 2> 3642222 INFO (jetty-closer-6945-thread-4) [] o.a.s.c.ZkController Publish this node as DOWN... 2> 3642223 INFO (jetty-closer-6945-thread-4) [] o.a.s.c.ZkController Publish node=127.0.0.1:34951_solr as DOWN 2> 3642264 INFO (zkCallback-6843-thread-2) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) 2> 3642297 INFO (zkCallback-6843-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [0]) 2> 3642297 INFO (zkCallback-6843-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [0]) 2> 3642298 INFO (zkCallback-6843-thread-5) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [0]) 2> 3642298 INFO (zkCallback-6843-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll/state.json] for collection [replacenodetest_coll] has occurred - updating... (live nodes size: [0]) 2> 3642405 INFO (jetty-closer-6945-thread-4) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null 2> 3643050 INFO (jetty-closer-6945-thread-4) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null 2> 3643483 INFO (jetty-closer-6945-thread-4) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null 2> 3643483 INFO (jetty-closer-6945-thread-4) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null 2> 3643535 INFO (zkCallback-6843-thread-3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34951/solr/replacenodetest_coll_shard1_replica_n13/ 2> 3643535 INFO (zkCallback-6843-thread-3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 3643535 INFO (zkCallback-6843-thread-3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.SyncStrategy http://127.0.0.1:34951/solr/replacenodetest_coll_shard1_replica_n13/ has no replicas 2> 3643536 INFO (zkCallback-6843-thread-3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@62c65314 replacenodetest_coll_shard1_replica_n13 2> 3643536 INFO (zkCallback-6843-thread-3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replacenodetest_coll.shard1.replica_n13 tag=SolrCore@62c65314 2> 3643553 INFO (closeThreadPool-6964-thread-2) [] o.a.s.c.Overseer Overseer (id=72080973716586504-127.0.0.1:34951_solr-n_0000000001) closing 2> 3643576 WARN (zkCallback-6843-thread-1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.SyncStrategy Closed, skipping sync up. 2> 3643579 INFO (zkCallback-6843-thread-1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway 2> 3643579 INFO (zkCallback-6843-thread-1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@4e12bf24 replacenodetest_coll_shard3_replica_n17 2> 3643582 INFO (OverseerStateUpdate-72080973716586504-127.0.0.1:34951_solr-n_0000000001) [n:127.0.0.1:34951_solr] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:34951_solr 2> 3643621 INFO (closeThreadPool-6964-thread-1) [] o.a.s.c.Overseer Overseer (id=72080973716586504-127.0.0.1:34951_solr-n_0000000001) closing 2> 3644493 INFO (zkCallback-6843-thread-3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replacenodetest_coll.shard1.leader tag=SolrCore@62c65314 2> 3644496 INFO (zkCallback-6843-thread-1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replacenodetest_coll.shard3.replica_n17 tag=SolrCore@4e12bf24 2> 3644501 INFO (zkCallback-6843-thread-3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 3644600 INFO (zkCallback-6843-thread-3) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard1 r:core_node14 x:replacenodetest_coll_shard1_replica_n13] o.a.s.c.ShardLeaderElectionContextBase No version found for ephemeral leader parent node, won't remove previous leader registration. 2> 3645857 INFO (zkCallback-6843-thread-1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replacenodetest_coll.shard3.leader tag=SolrCore@4e12bf24 2> 3645869 INFO (zkCallback-6843-thread-1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 3645903 INFO (zkCallback-6843-thread-1) [n:127.0.0.1:34951_solr c:replacenodetest_coll s:shard3 r:core_node18 x:replacenodetest_coll_shard3_replica_n17] o.a.s.c.ShardLeaderElectionContextBase No version found for ephemeral leader parent node, won't remove previous leader registration. 2> 3646028 INFO (jetty-closer-6945-thread-4) [] o.a.s.c.Overseer Overseer (id=72080973716586504-127.0.0.1:34951_solr-n_0000000001) closing 2> 3646050 INFO (SUITE-ReplaceNodeTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.ZkTestServer Shutting down ZkTestServer. 2> 3646286 WARN (ZkTestServer Run Thread) [] o.a.s.c.ZkTestServer Watch limit violations: 2> Maximum concurrent create/delete watches above limit: 2> 2> 10 /solr/collections/replacenodetest_coll/terms/shard1 2> 9 /solr/collections/replacenodetest_coll/terms/shard3 2> 9 /solr/collections/replacenodetest_coll/terms/shard2 2> 5 /solr/aliases.json 2> 5 /solr/clusterprops.json 2> 4 /solr/packages.json 2> 4 /solr/security.json 2> 4 /solr/configs/conf1 2> 4 /solr/collections/replacenodetest_coll/collectionprops.json 2> 2> Maximum concurrent data watches above limit: 2> 2> 172 /solr/collections/replacenodetest_coll/state.json 2> 2 /solr/overseer_elect/election/72080973716586504-127.0.0.1:34951_solr-n_0000000001 2> 2> Maximum concurrent children watches above limit: 2> 2> 32 /solr/collections/replacenodetest_coll/state.json 2> 26 /solr/live_nodes 2> 10 /solr/collections 2> 2 /solr/overseer/queue 2> 2 /solr/overseer/queue-work 2> 2 /solr/overseer/collection-queue-work 2> 2> 3646569 INFO (SUITE-ReplaceNodeTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.u.ErrorLogMuter Closing ErrorLogMuter-regex-145 after mutting 0 log messages 2> 3646570 INFO (SUITE-ReplaceNodeTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.u.ErrorLogMuter Creating ErrorLogMuter-regex-146 for ERROR logs matching regex: ignore_exception 2> NOTE: leaving temporary files on disk at: /build/solr/src/solr-9.2.1/solr/core/build/tmp/tests-tmp/solr.cloud.ReplaceNodeTest_9CAA27B60427D55F-001 2> NOTE: test params are: codec=Asserting(Lucene94): {}, docValues:{}, maxPointsInLeafNode=1854, maxMBSortInHeap=5.951943351798814, sim=Asserting(RandomSimilarity(queryNorm=false): {}), locale=en-GH, timezone=Iran 2> NOTE: Linux 5.10.113-g387b6863253c-dirty riscv64/Oracle Corporation 11.0.20 (64-bit)/cpus=1,threads=2,free=143429800,total=503316480 2> NOTE: All tests run in this JVM: [BasicFunctionalityTest, CursorPagingTest, DistributedIntervalFacetingTest, HelloWorldSolrCloudTestCase, OutputWriterTest, SolrInfoBeanTest, SolrTestCaseJ4Test, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, TestDocumentBuilder, TestHighlightDedupGrouping, TestRandomDVFaceting, TestSimpleTrackingShardHandler, TestTolerantSearch, CommonGramsPhraseQueryTest, ProtectedTermFilterFactoryTest, TestDeprecatedFilters, TestReversedWildcardFilterFactory, TokenizerChainTest, TestEmbeddedSolrServerAdminHandler, TestEmbeddedSolrServerSchemaAPI, ActionThrottleTest, AliasIntegrationTest, BasicDistributedZk2Test, ChaosMonkeyNothingIsSafeTest, ChaosMonkeySafeLeaderTest, CloudExitableDirectoryReaderTest, ClusterStateTest, CollectionPropsTest, CollectionsAPISolrJTest, ConfigSetApiLockingTest, CreateCollectionCleanupTest, DeleteInactiveReplicaTest, DeleteNodeTest, DeleteShardTest, DistribCursorPagingTest, DistribJoinFromCollectionTest, DistributedQueueTest, DocValuesNotIndexedTest, ForceLeaderWithTlogReplicasTest, HttpPartitionOnCommitTest, HttpPartitionWithTlogReplicasTest, LeaderElectionIntegrationTest, LeaderFailoverAfterPartitionTest, LeaderTragicEventTest, MigrateRouteKeyTest, MoveReplicaTest, MultiThreadedOCPTest, NodeMutatorTest, OutOfBoxZkACLAndCredentialsProvidersTest, OverseerCollectionConfigSetProcessorTest, OverseerRolesTest, OverseerStatusTest, OverseerTest, ParallelCommitExecutionTest, RecoveryAfterSoftCommitTest, ReindexCollectionTest, ReplaceNodeTest] WARNING: Test org.apache.solr.schema.TestCloudSchemaless wrote 11,061,504 bytes of output. WARNING: Test org.apache.solr.search.facet.TestCloudJSONFacetSKGEquiv wrote 16,387,756 bytes of output. WARNING: Test org.apache.solr.search.facet.TestCloudJSONFacetSKG wrote 26,264,317 bytes of output. > Task :solr:benchmark:test :solr:benchmark:test (SUCCESS): 11 test(s) > Task :solr:benchmark:wipeTaskTemp > Task :solr:prometheus-exporter:compileJava UP-TO-DATE > Task :solr:prometheus-exporter:processResources NO-SOURCE > Task :solr:prometheus-exporter:classes UP-TO-DATE > Task :solr:prometheus-exporter:compileTestJava Note: /build/solr/src/solr-9.2.1/solr/prometheus-exporter/src/test/org/apache/solr/prometheus/exporter/MetricsQueryTemplateTest.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. > Task :solr:prometheus-exporter:processTestResources > Task :solr:prometheus-exporter:testClasses > Task :solr:prometheus-exporter:test :solr:prometheus-exporter:test (SUCCESS): 21 test(s) > Task :solr:prometheus-exporter:wipeTaskTemp > Task :solr:server:compileJava SKIPPED > Task :solr:server:processResources NO-SOURCE > Task :solr:server:classes UP-TO-DATE > Task :solr:server:compileTestJava NO-SOURCE > Task :solr:server:processTestResources NO-SOURCE > Task :solr:server:testClasses UP-TO-DATE > Task :solr:server:test NO-SOURCE > Task :solr:server:wipeTaskTemp > Task :solr:solr-ref-guide:compileJava NO-SOURCE > Task :solr:solr-ref-guide:processResources NO-SOURCE > Task :solr:solr-ref-guide:classes UP-TO-DATE > Task :solr:solr-ref-guide:compileTestJava Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. > Task :solr:solr-ref-guide:copySolrjTestResources > Task :solr:solr-ref-guide:processTestResources NO-SOURCE > Task :solr:solr-ref-guide:testClasses > Task :solr:solr-ref-guide:test :solr:solr-ref-guide:test (SUCCESS): 32 test(s) > Task :solr:solr-ref-guide:wipeTaskTemp > Task :solr:solrj:compileTestJava Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. > Task :solr:solrj:processTestResources > Task :solr:solrj:testClasses > Task :solr:solrj:test org.apache.solr.client.solrj.impl.CloudSolrClientTest > queryWithLocalShardsPreferenceRulesTest FAILED java.lang.RuntimeException: Failed while waiting for active collection Timeout waiting to see state for collection=localShardsTestColl :DocCollection(localShardsTestColl//collections/localShardsTestColl/state.json/24 )={ "pullReplicas":"0", "configName":"conf", "replicationFactor":"3", "shards":{ "shard1":{ "range":"80000000-d554ffff", "state":"active", "replicas":{ "core_node13":{ "core":"localShardsTestColl_shard1_replica_n4", "node_name":"127.0.0.1:40391_solr", "type":"NRT", "state":"active", "force_set_state":"false", "base_url":"http://127.0.0.1:40391/solr"}, "core_node14":{ "core":"localShardsTestColl_shard1_replica_n5", "node_name":"127.0.0.1:44127_solr", "type":"NRT", "state":"active", "force_set_state":"false", "base_url":"http://127.0.0.1:44127/solr"}, "core_node16":{ "core":"localShardsTestColl_shard1_replica_n7", "node_name":"127.0.0.1:39159_solr", "type":"NRT", "state":"active", "leader":"true", "force_set_state":"false", "base_url":"http://127.0.0.1:39159/solr"}}}, "shard2":{ "range":"d5550000-2aa9ffff", "state":"active", "replicas":{ "core_node10":{ "core":"localShardsTestColl_shard2_replica_n1", "node_name":"127.0.0.1:44127_solr", "type":"NRT", "state":"active", "leader":"true", "force_set_state":"false", "base_url":"http://127.0.0.1:44127/solr"}, "core_node11":{ "core":"localShardsTestColl_shard2_replica_n2", "node_name":"127.0.0.1:39159_solr", "type":"NRT", "state":"down", "force_set_state":"false", "base_url":"http://127.0.0.1:39159/solr"}, "core_node17":{ "core":"localShardsTestColl_shard2_replica_n8", "node_name":"127.0.0.1:40391_solr", "type":"NRT", "state":"active", "force_set_state":"false", "base_url":"http://127.0.0.1:40391/solr"}}}, "shard3":{ "range":"2aaa0000-7fffffff", "state":"active", "replicas":{ "core_node12":{ "core":"localShardsTestColl_shard3_replica_n3", "node_name":"127.0.0.1:44127_solr", "type":"NRT", "state":"active", "force_set_state":"false", "base_url":"http://127.0.0.1:44127/solr"}, "core_node15":{ "core":"localShardsTestColl_shard3_replica_n6", "node_name":"127.0.0.1:39159_solr", "type":"NRT", "state":"active", "leader":"true", "force_set_state":"false", "base_url":"http://127.0.0.1:39159/solr"}, "core_node18":{ "core":"localShardsTestColl_shard3_replica_n9", "node_name":"127.0.0.1:40391_solr", "type":"NRT", "state":"active", "force_set_state":"false", "base_url":"http://127.0.0.1:40391/solr"}}}}, "router":{"name":"compositeId"}, "nrtReplicas":"3", "tlogReplicas":"0"} Live Nodes: [127.0.0.1:39159_solr, 127.0.0.1:40391_solr, 127.0.0.1:44127_solr] Last available state: DocCollection(localShardsTestColl//collections/localShardsTestColl/state.json/24 )={ "pullReplicas":"0", "configName":"conf", "replicationFactor":"3", "shards":{ "shard1":{ "range":"80000000-d554ffff", "state":"active", "replicas":{ "core_node13":{ "core":"localShardsTestColl_shard1_replica_n4", "node_name":"127.0.0.1:40391_solr", "type":"NRT", "state":"active", "force_set_state":"false", "base_url":"http://127.0.0.1:40391/solr"}, "core_node14":{ "core":"localShardsTestColl_shard1_replica_n5", "node_name":"127.0.0.1:44127_solr", "type":"NRT", "state":"active", "force_set_state":"false", "base_url":"http://127.0.0.1:44127/solr"}, "core_node16":{ "core":"localShardsTestColl_shard1_replica_n7", "node_name":"127.0.0.1:39159_solr", "type":"NRT", "state":"active", "leader":"true", "force_set_state":"false", "base_url":"http://127.0.0.1:39159/solr"}}}, "shard2":{ "range":"d5550000-2aa9ffff", "state":"active", "replicas":{ "core_node10":{ "core":"localShardsTestColl_shard2_replica_n1", "node_name":"127.0.0.1:44127_solr", "type":"NRT", "state":"active", "leader":"true", "force_set_state":"false", "base_url":"http://127.0.0.1:44127/solr"}, "core_node11":{ "core":"localShardsTestColl_shard2_replica_n2", "node_name":"127.0.0.1:39159_solr", "type":"NRT", "state":"down", "force_set_state":"false", "base_url":"http://127.0.0.1:39159/solr"}, "core_node17":{ "core":"localShardsTestColl_shard2_replica_n8", "node_name":"127.0.0.1:40391_solr", "type":"NRT", "state":"active", "force_set_state":"false", "base_url":"http://127.0.0.1:40391/solr"}}}, "shard3":{ "range":"2aaa0000-7fffffff", "state":"active", "replicas":{ "core_node12":{ "core":"localShardsTestColl_shard3_replica_n3", "node_name":"127.0.0.1:44127_solr", "type":"NRT", "state":"active", "force_set_state":"false", "base_url":"http://127.0.0.1:44127/solr"}, "core_node15":{ "core":"localShardsTestColl_shard3_replica_n6", "node_name":"127.0.0.1:39159_solr", "type":"NRT", "state":"active", "leader":"true", "force_set_state":"false", "base_url":"http://127.0.0.1:39159/solr"}, "core_node18":{ "core":"localShardsTestColl_shard3_replica_n9", "node_name":"127.0.0.1:40391_solr", "type":"NRT", "state":"active", "force_set_state":"false", "base_url":"http://127.0.0.1:40391/solr"}}}}, "router":{"name":"compositeId"}, "nrtReplicas":"3", "tlogReplicas":"0"} at __randomizedtesting.SeedInfo.seed([9CAA27B60427D55F:3A81F37DF334062F]:0) at org.apache.solr.cloud.MiniSolrCloudCluster.waitForActiveCollection(MiniSolrCloudCluster.java:895) at org.apache.solr.cloud.MiniSolrCloudCluster.waitForActiveCollection(MiniSolrCloudCluster.java:900) at org.apache.solr.client.solrj.impl.CloudSolrClientTest.queryWithLocalShardsPreferenceRulesTest(CloudSolrClientTest.java:445) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:44) at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) at java.base/java.lang.Thread.run(Thread.java:829) org.apache.solr.client.solrj.impl.CloudSolrClientTest > classMethod FAILED com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.client.solrj.impl.CloudSolrClientTest: 1) Thread[id=2926, name=parallelCoreAdminAPIBaseExecutor-1177-thread-12-processing-127.0.0.1:39159_solr localShardsTestColl_shard2_replica_n2 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 create, state=WAITING, group=TGRP-CloudSolrClientTest] at java.base@11.0.20/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.20/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.20/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) at java.base@11.0.20/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:917) at java.base@11.0.20/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1240) at java.base@11.0.20/java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:959) at java.management@11.0.20/com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:415) at java.management@11.0.20/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1855) at java.management@11.0.20/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:955) at java.management@11.0.20/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:890) at java.management@11.0.20/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:320) at java.management@11.0.20/com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) at app//org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.registerMBean(JmxMetricsReporter.java:546) at app//org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.onCounterAdded(JmxMetricsReporter.java:615) at app//com.codahale.metrics.MetricRegistry.notifyListenerOfAddedMetric(MetricRegistry.java:554) at app//com.codahale.metrics.MetricRegistry.onMetricAdded(MetricRegistry.java:546) at app//com.codahale.metrics.MetricRegistry.register(MetricRegistry.java:166) at app//com.codahale.metrics.MetricRegistry.getOrAdd(MetricRegistry.java:521) at app//com.codahale.metrics.MetricRegistry.counter(MetricRegistry.java:204) at app//org.apache.solr.metrics.SolrMetricManager.counter(SolrMetricManager.java:686) at app//org.apache.solr.metrics.SolrMetricsContext.counter(SolrMetricsContext.java:143) at app//org.apache.solr.handler.RequestHandlerBase$HandlerMetrics.(RequestHandlerBase.java:192) at app//org.apache.solr.handler.RequestHandlerBase.initializeMetrics(RequestHandlerBase.java:162) at app//org.apache.solr.metrics.SolrCoreMetricManager.registerMetricProducer(SolrCoreMetricManager.java:139) at app//org.apache.solr.core.SolrCore.registerInfoBean(SolrCore.java:3454) at app//org.apache.solr.core.PluginBag.registerMBean(PluginBag.java:367) at app//org.apache.solr.core.PluginBag.put(PluginBag.java:284) at app//org.apache.solr.core.PluginBag.init(PluginBag.java:333) at app//org.apache.solr.core.RequestHandlers.initHandlersFromConfig(RequestHandlers.java:119) at app//org.apache.solr.core.SolrCore.(SolrCore.java:1138) at app//org.apache.solr.core.SolrCore.(SolrCore.java:1051) at app//org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1666) at app//org.apache.solr.core.CoreContainer.create(CoreContainer.java:1532) at app//org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:111) at app//org.apache.solr.handler.admin.CoreAdminOperation$$Lambda$373/0x000000010056f840.execute(Unknown Source) at app//org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:398) at app//org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:354) at app//org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:225) at app//org.apache.solr.handler.admin.CoreAdminHandler$$Lambda$1248/0x0000000100c3ec40.call(Unknown Source) at app//org.apache.solr.handler.admin.CoreAdminHandler$CoreAdminAsyncTracker.lambda$submitAsyncTask$0(CoreAdminHandler.java:445) at app//org.apache.solr.handler.admin.CoreAdminHandler$CoreAdminAsyncTracker$$Lambda$1249/0x0000000100c3f040.run(Unknown Source) at app//com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:212) at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289) at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$354/0x000000010052f040.run(Unknown Source) at java.base@11.0.20/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base@11.0.20/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.20/java.lang.Thread.run(Thread.java:829) at __randomizedtesting.SeedInfo.seed([9CAA27B60427D55F]:0) org.apache.solr.client.solrj.impl.CloudSolrClientTest > classMethod FAILED com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated: 1) Thread[id=2926, name=parallelCoreAdminAPIBaseExecutor-1177-thread-12-processing-127.0.0.1:39159_solr localShardsTestColl_shard2_replica_n2 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 create, state=WAITING, group=TGRP-CloudSolrClientTest] at java.base@11.0.20/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.20/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.20/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) at java.base@11.0.20/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:917) at java.base@11.0.20/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1240) at java.base@11.0.20/java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:959) at java.management@11.0.20/com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:415) at java.management@11.0.20/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1855) at java.management@11.0.20/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:955) at java.management@11.0.20/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:890) at java.management@11.0.20/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:320) at java.management@11.0.20/com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) at app//org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.registerMBean(JmxMetricsReporter.java:546) at app//org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.onCounterAdded(JmxMetricsReporter.java:615) at app//com.codahale.metrics.MetricRegistry.notifyListenerOfAddedMetric(MetricRegistry.java:554) at app//com.codahale.metrics.MetricRegistry.onMetricAdded(MetricRegistry.java:546) at app//com.codahale.metrics.MetricRegistry.register(MetricRegistry.java:166) at app//com.codahale.metrics.MetricRegistry.getOrAdd(MetricRegistry.java:521) at app//com.codahale.metrics.MetricRegistry.counter(MetricRegistry.java:204) at app//org.apache.solr.metrics.SolrMetricManager.counter(SolrMetricManager.java:686) at app//org.apache.solr.metrics.SolrMetricsContext.counter(SolrMetricsContext.java:143) at app//org.apache.solr.handler.RequestHandlerBase$HandlerMetrics.(RequestHandlerBase.java:192) at app//org.apache.solr.handler.RequestHandlerBase.initializeMetrics(RequestHandlerBase.java:162) at app//org.apache.solr.metrics.SolrCoreMetricManager.registerMetricProducer(SolrCoreMetricManager.java:139) at app//org.apache.solr.core.SolrCore.registerInfoBean(SolrCore.java:3454) at app//org.apache.solr.core.PluginBag.registerMBean(PluginBag.java:367) at app//org.apache.solr.core.PluginBag.put(PluginBag.java:284) at app//org.apache.solr.core.PluginBag.init(PluginBag.java:333) at app//org.apache.solr.core.RequestHandlers.initHandlersFromConfig(RequestHandlers.java:119) at app//org.apache.solr.core.SolrCore.(SolrCore.java:1138) at app//org.apache.solr.core.SolrCore.(SolrCore.java:1051) at app//org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1666) at app//org.apache.solr.core.CoreContainer.create(CoreContainer.java:1532) at app//org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:111) at app//org.apache.solr.handler.admin.CoreAdminOperation$$Lambda$373/0x000000010056f840.execute(Unknown Source) at app//org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:398) at app//org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:354) at app//org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:225) at app//org.apache.solr.handler.admin.CoreAdminHandler$$Lambda$1248/0x0000000100c3ec40.call(Unknown Source) at app//org.apache.solr.handler.admin.CoreAdminHandler$CoreAdminAsyncTracker.lambda$submitAsyncTask$0(CoreAdminHandler.java:445) at app//org.apache.solr.handler.admin.CoreAdminHandler$CoreAdminAsyncTracker$$Lambda$1249/0x0000000100c3f040.run(Unknown Source) at app//com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:212) at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289) at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$354/0x000000010052f040.run(Unknown Source) at java.base@11.0.20/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base@11.0.20/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.20/java.lang.Thread.run(Thread.java:829) at __randomizedtesting.SeedInfo.seed([9CAA27B60427D55F]:0) org.apache.solr.client.solrj.impl.CloudSolrClientTest > test suite's output saved to /build/solr/src/solr-9.2.1/solr/solrj/build/test-results/test/outputs/OUTPUT-org.apache.solr.client.solrj.impl.CloudSolrClientTest.txt, copied below: 2> 576509 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/build/solr/src/solr-9.2.1/solr/server/solr/configsets/_default/conf' 2> 576510 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom 2> 576516 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.u.ErrorLogMuter Closing ErrorLogMuter-regex-32 after mutting 0 log messages 2> 576517 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.u.ErrorLogMuter Creating ErrorLogMuter-regex-33 for ERROR logs matching regex: ignore_exception 2> 576521 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.SolrTestCaseJ4 Created dataDir: /build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/data-dir-12-001 2> 576522 WARN (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=79 numCloses=79 2> 576523 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true 2> 576527 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, clientAuth=0.0/0.0) 2> 576530 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in /build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001 2> 576531 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER 2> 576532 INFO (ZkTestServer Run Thread) [] o.a.s.c.ZkTestServer client port: 0.0.0.0/0.0.0.0:0 2> 576533 INFO (ZkTestServer Run Thread) [] o.a.s.c.ZkTestServer Starting server 2> 576542 WARN (ZkTestServer Run Thread) [] o.a.z.s.ServerCnxnFactory maxCnxns is not configured, using default value 0. 2> 576632 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.ZkTestServer start zk server on port: 40255 2> 576638 WARN (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 576644 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 576664 INFO (zkConnectionManagerCallback-1126-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 576665 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 576665 WARN (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 576673 WARN (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 576680 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 576694 INFO (zkConnectionManagerCallback-1128-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 576694 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 576694 WARN (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 576696 WARN (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 576707 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 576722 INFO (zkConnectionManagerCallback-1130-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 576723 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 576723 WARN (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 576850 INFO (jetty-launcher-1131-thread-2) [] o.a.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0) 2> 576851 INFO (jetty-launcher-1131-thread-2) [] o.a.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ... 2> 576852 INFO (jetty-launcher-1131-thread-2) [] o.e.j.s.Server jetty-10.0.13; built: 2022-12-07T20:13:20.134Z; git: 1c2636ea05c0ca8de1ffd6ca7f3a98ac084c766d; jvm 11.0.20+8 2> 576854 INFO (jetty-launcher-1131-thread-1) [] o.a.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0) 2> 576855 INFO (jetty-launcher-1131-thread-1) [] o.a.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ... 2> 576856 INFO (jetty-launcher-1131-thread-1) [] o.e.j.s.Server jetty-10.0.13; built: 2022-12-07T20:13:20.134Z; git: 1c2636ea05c0ca8de1ffd6ca7f3a98ac084c766d; jvm 11.0.20+8 2> 576861 INFO (jetty-launcher-1131-thread-3) [] o.a.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0) 2> 576861 INFO (jetty-launcher-1131-thread-3) [] o.a.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ... 2> 576862 INFO (jetty-launcher-1131-thread-3) [] o.e.j.s.Server jetty-10.0.13; built: 2022-12-07T20:13:20.134Z; git: 1c2636ea05c0ca8de1ffd6ca7f3a98ac084c766d; jvm 11.0.20+8 2> 576894 INFO (jetty-launcher-1131-thread-3) [] o.e.j.s.s.DefaultSessionIdManager Session workerName=node0 2> 576904 INFO (jetty-launcher-1131-thread-1) [] o.e.j.s.s.DefaultSessionIdManager Session workerName=node0 2> 576907 INFO (jetty-launcher-1131-thread-1) [] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5c2ab22c{/solr,file:///build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-cwd/,AVAILABLE} 2> 576909 INFO (jetty-launcher-1131-thread-3) [] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@f7c6320{/solr,file:///build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-cwd/,AVAILABLE} 2> 576913 INFO (jetty-launcher-1131-thread-1) [] o.e.j.s.AbstractConnector Started ServerConnector@4afdf971{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:40391} 2> 576913 INFO (jetty-launcher-1131-thread-3) [] o.e.j.s.AbstractConnector Started ServerConnector@3c7a151{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:44127} 2> 576914 INFO (jetty-launcher-1131-thread-3) [] o.e.j.s.Server Started Server@7372ec3a{STARTING}[10.0.13,sto=0] @577137ms 2> 576914 INFO (jetty-launcher-1131-thread-1) [] o.e.j.s.Server Started Server@3a01df69{STARTING}[10.0.13,sto=0] @577137ms 2> 576916 ERROR (jetty-launcher-1131-thread-3) [] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. 2> 576916 INFO (jetty-launcher-1131-thread-3) [] o.a.s.s.CoreContainerProvider Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory 2> 576916 INFO (jetty-launcher-1131-thread-3) [] o.a.s.s.CoreContainerProvider ___ _ Welcome to Apache Solr™ version 9.2.1 2> 576917 ERROR (jetty-launcher-1131-thread-1) [] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. 2> 576917 INFO (jetty-launcher-1131-thread-1) [] o.a.s.s.CoreContainerProvider Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory 2> 576917 INFO (jetty-launcher-1131-thread-3) [] o.a.s.s.CoreContainerProvider / __| ___| |_ _ Starting in cloud mode on port null 2> 576917 INFO (jetty-launcher-1131-thread-1) [] o.a.s.s.CoreContainerProvider ___ _ Welcome to Apache Solr™ version 9.2.1 2> 576917 INFO (jetty-launcher-1131-thread-3) [] o.a.s.s.CoreContainerProvider \__ \/ _ \ | '_| Install dir: /build/solr/src/solr-9.2.1/solr 2> 576918 INFO (jetty-launcher-1131-thread-3) [] o.a.s.s.CoreContainerProvider |___/\___/_|_| Start time: 2023-10-03T12:19:04.371337Z 2> 576919 INFO (jetty-launcher-1131-thread-2) [] o.e.j.s.s.DefaultSessionIdManager Session workerName=node0 2> 576919 INFO (jetty-launcher-1131-thread-3) [] o.a.s.s.CoreContainerProvider Solr Home: /build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node3 (source: servlet config: solr.solr.home) 2> 576918 INFO (jetty-launcher-1131-thread-1) [] o.a.s.s.CoreContainerProvider / __| ___| |_ _ Starting in cloud mode on port null 2> 576921 WARN (jetty-launcher-1131-thread-3) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 576921 INFO (jetty-launcher-1131-thread-1) [] o.a.s.s.CoreContainerProvider \__ \/ _ \ | '_| Install dir: /build/solr/src/solr-9.2.1/solr 2> 576921 INFO (jetty-launcher-1131-thread-1) [] o.a.s.s.CoreContainerProvider |___/\___/_|_| Start time: 2023-10-03T12:19:04.374868Z 2> 576923 INFO (jetty-launcher-1131-thread-1) [] o.a.s.s.CoreContainerProvider Solr Home: /build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1 (source: servlet config: solr.solr.home) 2> 576925 WARN (jetty-launcher-1131-thread-1) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 576929 INFO (jetty-launcher-1131-thread-2) [] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@185700aa{/solr,file:///build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-cwd/,AVAILABLE} 2> 576933 INFO (jetty-launcher-1131-thread-2) [] o.e.j.s.AbstractConnector Started ServerConnector@23ad07f{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:39159} 2> 576934 INFO (jetty-launcher-1131-thread-2) [] o.e.j.s.Server Started Server@720ef990{STARTING}[10.0.13,sto=0] @577158ms 2> 576936 ERROR (jetty-launcher-1131-thread-2) [] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. 2> 576936 INFO (jetty-launcher-1131-thread-2) [] o.a.s.s.CoreContainerProvider Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory 2> 576936 INFO (jetty-launcher-1131-thread-2) [] o.a.s.s.CoreContainerProvider ___ _ Welcome to Apache Solr™ version 9.2.1 2> 576936 INFO (jetty-launcher-1131-thread-3) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 576937 INFO (jetty-launcher-1131-thread-2) [] o.a.s.s.CoreContainerProvider / __| ___| |_ _ Starting in cloud mode on port null 2> 576937 INFO (jetty-launcher-1131-thread-2) [] o.a.s.s.CoreContainerProvider \__ \/ _ \ | '_| Install dir: /build/solr/src/solr-9.2.1/solr 2> 576938 INFO (jetty-launcher-1131-thread-2) [] o.a.s.s.CoreContainerProvider |___/\___/_|_| Start time: 2023-10-03T12:19:04.391200Z 2> 576941 INFO (jetty-launcher-1131-thread-2) [] o.a.s.s.CoreContainerProvider Solr Home: /build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2 (source: servlet config: solr.solr.home) 2> 576947 WARN (jetty-launcher-1131-thread-2) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 576954 INFO (zkConnectionManagerCallback-1133-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 576954 INFO (jetty-launcher-1131-thread-3) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 576955 WARN (jetty-launcher-1131-thread-3) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 576959 INFO (jetty-launcher-1131-thread-3) [] o.a.s.c.NodeConfig solr.xml found in ZooKeeper. Loading... 2> 576968 INFO (jetty-launcher-1131-thread-1) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 576971 INFO (jetty-launcher-1131-thread-2) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 576982 INFO (zkConnectionManagerCallback-1135-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 576983 INFO (jetty-launcher-1131-thread-1) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 576984 WARN (jetty-launcher-1131-thread-1) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 576986 INFO (zkConnectionManagerCallback-1137-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 576987 INFO (jetty-launcher-1131-thread-2) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 576987 WARN (jetty-launcher-1131-thread-2) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 576991 INFO (jetty-launcher-1131-thread-1) [] o.a.s.c.NodeConfig solr.xml found in ZooKeeper. Loading... 2> 576993 INFO (jetty-launcher-1131-thread-2) [] o.a.s.c.NodeConfig solr.xml found in ZooKeeper. Loading... 2> 577194 INFO (jetty-launcher-1131-thread-2) [] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40255/solr 2> 577195 INFO (jetty-launcher-1131-thread-2) [] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=true. Solr will be using distributed cluster state updates. 2> 577207 INFO (jetty-launcher-1131-thread-1) [] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40255/solr 2> 577207 INFO (jetty-launcher-1131-thread-2) [n:127.0.0.1:39159_solr] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 577212 INFO (jetty-launcher-1131-thread-3) [] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40255/solr 2> 577220 INFO (zkConnectionManagerCallback-1161-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 577220 INFO (jetty-launcher-1131-thread-2) [n:127.0.0.1:39159_solr] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 577220 INFO (jetty-launcher-1131-thread-1) [] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=true. Solr will be using distributed cluster state updates. 2> 577234 INFO (jetty-launcher-1131-thread-3) [] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=true. Solr will be using distributed cluster state updates. 2> 577249 INFO (jetty-launcher-1131-thread-1) [n:127.0.0.1:40391_solr] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 577272 INFO (jetty-launcher-1131-thread-3) [n:127.0.0.1:44127_solr] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 577276 INFO (zkConnectionManagerCallback-1165-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 577277 INFO (jetty-launcher-1131-thread-1) [n:127.0.0.1:40391_solr] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 577289 INFO (zkConnectionManagerCallback-1167-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 577289 INFO (jetty-launcher-1131-thread-3) [n:127.0.0.1:44127_solr] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 577521 WARN (jetty-launcher-1131-thread-1) [n:127.0.0.1:40391_solr] o.a.s.c.ZkController Contents of zookeeper /security.json are world-readable; consider setting up ACLs as described in https://solr.apache.org/guide/solr/latest/deployment-guide/zookeeper-access-control.html 2> 577522 WARN (jetty-launcher-1131-thread-3) [n:127.0.0.1:44127_solr] o.a.s.c.ZkController Contents of zookeeper /security.json are world-readable; consider setting up ACLs as described in https://solr.apache.org/guide/solr/latest/deployment-guide/zookeeper-access-control.html 2> 577526 WARN (jetty-launcher-1131-thread-2) [n:127.0.0.1:39159_solr] o.a.s.c.ZkController Contents of zookeeper /security.json are world-readable; consider setting up ACLs as described in https://solr.apache.org/guide/solr/latest/deployment-guide/zookeeper-access-control.html 2> 577555 INFO (jetty-launcher-1131-thread-1) [n:127.0.0.1:40391_solr] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=true. Solr will be using distributed cluster state updates. 2> 577575 INFO (jetty-launcher-1131-thread-3) [n:127.0.0.1:44127_solr] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=true. Solr will be using distributed cluster state updates. 2> 577580 INFO (jetty-launcher-1131-thread-2) [n:127.0.0.1:39159_solr] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=true. Solr will be using distributed cluster state updates. 2> 577593 INFO (jetty-launcher-1131-thread-1) [n:127.0.0.1:40391_solr] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40391_solr 2> 577604 INFO (jetty-launcher-1131-thread-1) [n:127.0.0.1:40391_solr] o.a.s.c.Overseer Overseer (id=72081801657778183-127.0.0.1:40391_solr-n_0000000000) starting 2> 577614 INFO (jetty-launcher-1131-thread-3) [n:127.0.0.1:44127_solr] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44127_solr 2> 577621 INFO (jetty-launcher-1131-thread-2) [n:127.0.0.1:39159_solr] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39159_solr 2> 577642 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2) 2> 577644 INFO (zkCallback-1166-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2) 2> 577645 INFO (zkCallback-1160-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2) 2> 577672 WARN (jetty-launcher-1131-thread-3) [n:127.0.0.1:44127_solr] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info 2> 577678 WARN (jetty-launcher-1131-thread-2) [n:127.0.0.1:39159_solr] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info 2> 577695 INFO (jetty-launcher-1131-thread-1) [n:127.0.0.1:40391_solr] o.a.s.c.ZkController Publish node=127.0.0.1:40391_solr as DOWN 2> 577699 INFO (OverseerStateUpdate-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [n:127.0.0.1:40391_solr] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:40391_solr 2> 577703 INFO (jetty-launcher-1131-thread-1) [n:127.0.0.1:40391_solr] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40391_solr 2> 577724 INFO (zkCallback-1160-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) 2> 577725 INFO (zkCallback-1166-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) 2> 577744 INFO (OverseerStateUpdate-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [n:127.0.0.1:40391_solr] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3) 2> 577766 WARN (jetty-launcher-1131-thread-1) [n:127.0.0.1:40391_solr] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info 2> 579085 INFO (jetty-launcher-1131-thread-2) [n:127.0.0.1:39159_solr] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2 2> 579472 INFO (jetty-launcher-1131-thread-2) [n:127.0.0.1:39159_solr] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 579543 INFO (jetty-launcher-1131-thread-3) [n:127.0.0.1:44127_solr] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node3 2> 579648 INFO (jetty-launcher-1131-thread-1) [n:127.0.0.1:40391_solr] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1 2> 579654 INFO (jetty-launcher-1131-thread-2) [n:127.0.0.1:39159_solr] o.a.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39159, zkHost=127.0.0.1:40255/solr} 2> 579889 INFO (jetty-launcher-1131-thread-3) [n:127.0.0.1:44127_solr] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 579923 INFO (jetty-launcher-1131-thread-1) [n:127.0.0.1:40391_solr] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 579963 INFO (jetty-launcher-1131-thread-3) [n:127.0.0.1:44127_solr] o.a.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44127, zkHost=127.0.0.1:40255/solr} 2> 579982 INFO (jetty-launcher-1131-thread-1) [n:127.0.0.1:40391_solr] o.a.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40391, zkHost=127.0.0.1:40255/solr} 2> 579989 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.MiniSolrCloudCluster waitForAllNodes: numServers=3 2> 579990 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.MiniSolrCloudCluster waitForNode: 127.0.0.1:39159_solr 2> 579991 WARN (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 580003 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.c.ConnectionManager Waiting up to 15000ms for client to connect to ZooKeeper 2> 580019 INFO (zkConnectionManagerCallback-1187-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 580019 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 580020 WARN (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 580027 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3) 2> 580034 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40255/solr ready 2> 580035 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.MiniSolrCloudCluster waitForNode: 127.0.0.1:44127_solr 2> 580035 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.MiniSolrCloudCluster waitForNode: 127.0.0.1:40391_solr 2> 580036 WARN (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 580046 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.c.ConnectionManager Waiting up to 45000ms for client to connect to ZooKeeper 2> 580055 INFO (zkConnectionManagerCallback-1189-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 580055 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 580056 WARN (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 580203 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=5 2> 580330 INFO (TEST-CloudSolrClientTest.testRouting-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Starting testRouting 2> 580362 INFO (OverseerThreadFactory-1176-thread-1) [] o.a.s.c.a.c.CreateCollectionCmd Create collection routing_collection 2> 580525 INFO (OverseerThreadFactory-1176-thread-1) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"routing_collection_shard1_replica_n1", 2> "node_name":"127.0.0.1:40391_solr", 2> "base_url":"http://127.0.0.1:40391/solr", 2> "collection":"routing_collection", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 580535 INFO (OverseerThreadFactory-1176-thread-1) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"routing_collection_shard2_replica_n2", 2> "node_name":"127.0.0.1:39159_solr", 2> "base_url":"http://127.0.0.1:39159/solr", 2> "collection":"routing_collection", 2> "shard":"shard2", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 580596 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr x:routing_collection_shard2_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=routing_collection_shard2_replica_n2&action=CREATE&numShards=2&collection=routing_collection&shard=shard2&wt=javabin&version=2&replicaType=NRT 2> 580603 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr x:routing_collection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=routing_collection_shard1_replica_n1&action=CREATE&numShards=2&collection=routing_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 580712 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 580742 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 580746 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.s.IndexSchema Schema name=test 2> 580781 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.s.IndexSchema Schema name=test 2> 580784 WARN (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 580785 WARN (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 580785 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 580787 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'routing_collection_shard2_replica_n2' using configuration from configset conf, trusted=true 2> 580792 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/routing_collection_shard2_replica_n2], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/routing_collection_shard2_replica_n2/data/] 2> 580820 WARN (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 580821 WARN (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 580822 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 580824 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'routing_collection_shard1_replica_n1' using configuration from configset conf, trusted=true 2> 580828 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/routing_collection_shard1_replica_n1], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/routing_collection_shard1_replica_n1/data/] 2> 581233 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 581248 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 581249 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 581268 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 581269 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 581272 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 581285 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 581285 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 581292 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 581295 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 581300 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 581302 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736744371322880 2> 581307 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 581307 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 581338 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 581343 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 581352 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 581354 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736744425848832 2> 581354 INFO (searcherExecutor-1193-thread-1-processing-127.0.0.1:39159_solr routing_collection_shard2_replica_n2 routing_collection shard2 core_node4) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 581388 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/routing_collection/terms/shard2 to Terms{values={core_node4=0}, version=0} for registerTerm 2> 581391 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/routing_collection/leaders/shard2 2> 581408 INFO (searcherExecutor-1195-thread-1-processing-127.0.0.1:40391_solr routing_collection_shard1_replica_n1 routing_collection shard1 core_node3) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 581438 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/routing_collection/terms/shard1 to Terms{values={core_node3=0}, version=0} for registerTerm 2> 581440 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/routing_collection/leaders/shard1 2> 581442 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 581443 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 581443 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39159/solr/routing_collection_shard2_replica_n2/ 2> 581448 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 581453 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:39159/solr/routing_collection_shard2_replica_n2/ has no replicas 2> 581453 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/routing_collection/leaders/shard2/leader after winning as /collections/routing_collection/leader_elect/shard2/election/72081801657778182-core_node4-n_0000000000 2> 581484 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39159/solr/routing_collection_shard2_replica_n2/ shard2 2> 581499 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 581500 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 581500 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40391/solr/routing_collection_shard1_replica_n1/ 2> 581505 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 581510 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:40391/solr/routing_collection_shard1_replica_n1/ has no replicas 2> 581510 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/routing_collection/leaders/shard1/leader after winning as /collections/routing_collection/leader_elect/shard1/election/72081801657778183-core_node3-n_0000000000 2> 581519 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 581538 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40391/solr/routing_collection_shard1_replica_n1/ shard1 2> 581561 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=routing_collection_shard2_replica_n2&action=CREATE&numShards=2&collection=routing_collection&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=968 2> 581585 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 581595 INFO (zkCallback-1160-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routing_collection/state.json] for collection [routing_collection] has occurred - updating... (live nodes size: [3]) 2> 581614 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=routing_collection_shard1_replica_n1&action=CREATE&numShards=2&collection=routing_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1013 2> 581628 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas 2> 581628 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf&name=routing_collection&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=1292 2> 581631 INFO (TEST-CloudSolrClientTest.testRouting-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForActiveCollection: routing_collection 2> 581656 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:routing_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=routing_collection&wt=javabin&version=2} status=0 QTime=5 2> 581665 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:routing_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=routing_collection&wt=javabin&version=2} status=0 QTime=5 2> 581674 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LISTALIASES&wt=javabin&version=2} status=0 QTime=4 2> 581683 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:routing_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=routing_collection&wt=javabin&version=2} status=0 QTime=5 2> 581729 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/routing_collection/terms/shard1 to Terms{values={core_node3=1}, version=1} for ensureHighestTermsAreNotZero 2> 581732 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=routing_collection:7&wt=javabin&version=2} status=0 QTime=40 2> 581731 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/routing_collection/terms/shard2 to Terms{values={core_node4=1}, version=1} for ensureHighestTermsAreNotZero 2> 581737 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=routing_collection:7&wt=javabin&version=2} status=0 QTime=43 2> 581889 INFO (searcherExecutor-1195-thread-1-processing-127.0.0.1:40391_solr routing_collection_shard1_replica_n1 routing_collection shard1 core_node3) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 581922 INFO (searcherExecutor-1193-thread-1-processing-127.0.0.1:39159_solr routing_collection_shard2_replica_n2 routing_collection shard2 core_node4) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 581925 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:40391/solr/routing_collection_shard1_replica_n1/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false} status=0 QTime=138 2> 581927 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=routing_collection:7&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=184 2> 581951 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={q=id:2&distrib=false&wt=javabin&version=2} hits=1 status=0 QTime=12 2> 581964 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={q=id:0&distrib=false&wt=javabin&version=2} hits=1 status=0 QTime=2 2> 581982 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=routing_collection:7&wt=javabin&version=2} status=0 QTime=6 2> 581985 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=routing_collection:7&wt=javabin&version=2} status=0 QTime=6 2> 582047 INFO (searcherExecutor-1193-thread-1-processing-127.0.0.1:39159_solr routing_collection_shard2_replica_n2 routing_collection shard2 core_node4) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 582065 INFO (searcherExecutor-1195-thread-1-processing-127.0.0.1:40391_solr routing_collection_shard1_replica_n1 routing_collection shard1 core_node3) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 582068 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:39159/solr/routing_collection_shard2_replica_n2/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false} status=0 QTime=60 2> 582071 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=routing_collection:7&commit=true&wt=javabin&version=2} status=0 QTime=80 2> 582091 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=routing_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:39159/solr/routing_collection_shard2_replica_n2/&rid=127.0.0.1-1339&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335549528&isShard=true&wt=javabin} hits=0 status=0 QTime=2 2> 582094 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=routing_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:40391/solr/routing_collection_shard1_replica_n1/&rid=127.0.0.1-1339&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335549528&isShard=true&wt=javabin} hits=0 status=0 QTime=1 2> 582097 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=routing_collection:7&wt=javabin&version=2} rid=127.0.0.1-1339 hits=0 status=0 QTime=22 2> 582104 WARN (TEST-CloudSolrClientTest.testRouting-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 582119 INFO (TEST-CloudSolrClientTest.testRouting-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 15000ms for client to connect to ZooKeeper 2> 582127 INFO (zkConnectionManagerCallback-1208-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 582128 INFO (TEST-CloudSolrClientTest.testRouting-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 582128 WARN (TEST-CloudSolrClientTest.testRouting-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 582136 INFO (TEST-CloudSolrClientTest.testRouting-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3) 2> 582144 INFO (TEST-CloudSolrClientTest.testRouting-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40255/solr ready 2> 582173 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=routing_collection:7&wt=javabin&version=2} status=0 QTime=16 2> 582177 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=routing_collection:7&wt=javabin&version=2} status=0 QTime=24 2> 582273 INFO (searcherExecutor-1195-thread-1-processing-127.0.0.1:40391_solr routing_collection_shard1_replica_n1 routing_collection shard1 core_node3) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 582275 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:39159/solr/routing_collection_shard2_replica_n2/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false} status=0 QTime=83 2> 582279 INFO (searcherExecutor-1193-thread-1-processing-127.0.0.1:39159_solr routing_collection_shard2_replica_n2 routing_collection shard2 core_node4) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 582282 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=routing_collection:7&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=98 2> 582294 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={q=id:2&distrib=false&wt=javabin&version=2} hits=1 status=0 QTime=2 2> 582308 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={q=id:0&distrib=false&wt=javabin&version=2} hits=1 status=0 QTime=2 2> 582373 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 582454 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/admin/mbeans params={qt=/admin/mbeans&stats=true&cat=QUERY&wt=javabin&version=2&key=/select} status=0 QTime=2 2> 582470 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/admin/mbeans params={qt=/admin/mbeans&stats=true&cat=QUERY&wt=javabin&version=2&key=/select} status=0 QTime=2 2> 582475 INFO (TEST-CloudSolrClientTest.testRouting-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Expected Slices [shard1:{ 2> "range":"80000000-ffffffff", 2> "state":"active", 2> "replicas":{"core_node3":{ 2> "core":"routing_collection_shard1_replica_n1", 2> "node_name":"127.0.0.1:40391_solr", 2> "type":"NRT", 2> "state":"active", 2> "leader":"true", 2> "force_set_state":"false", 2> "base_url":"http://127.0.0.1:40391/solr"}}}] 2> 582481 INFO (TEST-CloudSolrClientTest.testRouting-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Expected Slices [shard2:{ 2> "range":"0-7fffffff", 2> "state":"active", 2> "replicas":{"core_node4":{ 2> "core":"routing_collection_shard2_replica_n2", 2> "node_name":"127.0.0.1:39159_solr", 2> "type":"NRT", 2> "state":"active", 2> "leader":"true", 2> "force_set_state":"false", 2> "base_url":"http://127.0.0.1:39159/solr"}}}] 2> 582487 INFO (TEST-CloudSolrClientTest.testRouting-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Expected Slices [shard2:{ 2> "range":"0-7fffffff", 2> "state":"active", 2> "replicas":{"core_node4":{ 2> "core":"routing_collection_shard2_replica_n2", 2> "node_name":"127.0.0.1:39159_solr", 2> "type":"NRT", 2> "state":"active", 2> "leader":"true", 2> "force_set_state":"false", 2> "base_url":"http://127.0.0.1:39159/solr"}}}] 2> 582492 INFO (TEST-CloudSolrClientTest.testRouting-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Expected Slices [shard1:{ 2> "range":"80000000-ffffffff", 2> "state":"active", 2> "replicas":{"core_node3":{ 2> "core":"routing_collection_shard1_replica_n1", 2> "node_name":"127.0.0.1:40391_solr", 2> "type":"NRT", 2> "state":"active", 2> "leader":"true", 2> "force_set_state":"false", 2> "base_url":"http://127.0.0.1:40391/solr"}}}] 2> 582498 INFO (TEST-CloudSolrClientTest.testRouting-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Expected Slices [shard2:{ 2> "range":"0-7fffffff", 2> "state":"active", 2> "replicas":{"core_node4":{ 2> "core":"routing_collection_shard2_replica_n2", 2> "node_name":"127.0.0.1:39159_solr", 2> "type":"NRT", 2> "state":"active", 2> "leader":"true", 2> "force_set_state":"false", 2> "base_url":"http://127.0.0.1:39159/solr"}}}] 2> 582509 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=routing_collection:7&wt=javabin&version=2&_route_=1} rid=127.0.0.1-1342 hits=1 status=0 QTime=2 2> 582513 INFO (TEST-CloudSolrClientTest.testRouting-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest output: {responseHeader={zkConnected=true, status=0, QTime=2}, response={numFound=1,numFoundExact=true,start=0,docs=[SolrDocument{id=0, a_t=hello1, _version_=1778736745269952512, multiDefault=[muLti-Default], intDefault=42, timestamp=Tue Oct 03 13:19:09 WAT 2023}]}} 2> 582521 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:routing_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=routing_collection&wt=javabin&version=2} status=0 QTime=5 2> 582530 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=routing_collection&wt=javabin&version=2} status=0 QTime=5 2> 582540 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=5 2> 582551 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:routing_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=routing_collection&wt=javabin&version=2} status=0 QTime=6 2> 582560 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:routing_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=routing_collection&wt=javabin&version=2} status=0 QTime=5 2> 582566 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=routing_collection:7&wt=javabin&version=2&_route_=1} rid=127.0.0.1-1343 hits=1 status=0 QTime=2 2> 582568 INFO (TEST-CloudSolrClientTest.testRouting-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest output: {responseHeader={zkConnected=true, status=0, QTime=2}, response={numFound=1,numFoundExact=true,start=0,docs=[SolrDocument{id=0, a_t=hello1, _version_=1778736745269952512, multiDefault=[muLti-Default], intDefault=42, timestamp=Tue Oct 03 13:19:09 WAT 2023}]}} 2> 582575 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=routing_collection:7&wt=javabin&version=2&_route_=4} rid=127.0.0.1-1344 hits=1 status=0 QTime=2 2> 582579 INFO (TEST-CloudSolrClientTest.testRouting-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest output: {responseHeader={zkConnected=true, status=0, QTime=2}, response={numFound=1,numFoundExact=true,start=0,docs=[SolrDocument{id=0, a_t=hello1, _version_=1778736745269952512, multiDefault=[muLti-Default], intDefault=42, timestamp=Tue Oct 03 13:19:09 WAT 2023}]}} 2> 582590 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:routing_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=routing_collection&wt=javabin&version=2} status=0 QTime=6 2> 582600 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:routing_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=routing_collection&wt=javabin&version=2} status=0 QTime=5 2> 582609 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=6 2> 582618 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:routing_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=routing_collection&wt=javabin&version=2} status=0 QTime=5 2> 582627 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:routing_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=routing_collection&wt=javabin&version=2} status=0 QTime=5 2> 582633 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=routing_collection:7&wt=javabin&version=2&_route_=0} rid=127.0.0.1-1345 hits=1 status=0 QTime=2 2> 582635 INFO (TEST-CloudSolrClientTest.testRouting-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest output: {responseHeader={zkConnected=true, status=0, QTime=2}, response={numFound=1,numFoundExact=true,start=0,docs=[SolrDocument{id=0, a_t=hello1, _version_=1778736745269952512, multiDefault=[muLti-Default], intDefault=42, timestamp=Tue Oct 03 13:19:09 WAT 2023}]}} 2> 582643 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:routing_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=routing_collection&wt=javabin&version=2} status=0 QTime=5 2> 582652 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:routing_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=routing_collection&wt=javabin&version=2} status=0 QTime=5 2> 582660 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=5 2> 582669 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=routing_collection&wt=javabin&version=2} status=0 QTime=5 2> 582679 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:routing_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=routing_collection&wt=javabin&version=2} status=0 QTime=5 2> 582685 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=routing_collection:7&wt=javabin&version=2&_route_=1} rid=127.0.0.1-1346 hits=1 status=0 QTime=2 2> 582688 INFO (TEST-CloudSolrClientTest.testRouting-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest output: {responseHeader={zkConnected=true, status=0, QTime=2}, response={numFound=1,numFoundExact=true,start=0,docs=[SolrDocument{id=0, a_t=hello1, _version_=1778736745269952512, multiDefault=[muLti-Default], intDefault=42, timestamp=Tue Oct 03 13:19:09 WAT 2023}]}} 2> 582696 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:routing_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=routing_collection&wt=javabin&version=2} status=0 QTime=5 2> 582705 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:routing_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=routing_collection&wt=javabin&version=2} status=0 QTime=5 2> 582713 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=5 2> 582722 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:routing_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=routing_collection&wt=javabin&version=2} status=0 QTime=5 2> 582731 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:routing_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=routing_collection&wt=javabin&version=2} status=0 QTime=5 2> 582737 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=routing_collection:7&wt=javabin&version=2&_route_=4} rid=127.0.0.1-1347 hits=1 status=0 QTime=2 2> 582741 INFO (TEST-CloudSolrClientTest.testRouting-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest output: {responseHeader={zkConnected=true, status=0, QTime=2}, response={numFound=1,numFoundExact=true,start=0,docs=[SolrDocument{id=0, a_t=hello1, _version_=1778736745269952512, multiDefault=[muLti-Default], intDefault=42, timestamp=Tue Oct 03 13:19:09 WAT 2023}]}} 2> 582752 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/admin/mbeans params={qt=/admin/mbeans&stats=true&cat=QUERY&wt=javabin&version=2&key=/select} status=0 QTime=2 2> 582764 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/admin/mbeans params={qt=/admin/mbeans&stats=true&cat=QUERY&wt=javabin&version=2&key=/select} status=0 QTime=2 2> 582770 INFO (TEST-CloudSolrClientTest.testRouting-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Ending testRouting 2> 582951 INFO (TEST-CloudSolrClientTest.testOverwriteOption-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Starting testOverwriteOption 2> 582974 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=0a03e15a-d894-4881-8acd-90396ec5d74f&collection.configName=conf&name=overwrite&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=18 2> 582992 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=0a03e15a-d894-4881-8acd-90396ec5d74f&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=12 2> 582997 INFO (OverseerThreadFactory-1176-thread-2) [] o.a.s.c.a.c.CreateCollectionCmd Create collection overwrite 2> 583150 INFO (OverseerThreadFactory-1176-thread-2) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"overwrite_shard1_replica_n1", 2> "node_name":"127.0.0.1:39159_solr", 2> "base_url":"http://127.0.0.1:39159/solr", 2> "collection":"overwrite", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 583187 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr x:overwrite_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=overwrite&version=2&replicaType=NRT&async=0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062&coreNodeName=core_node2&name=overwrite_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&wt=javabin} status=0 QTime=3 2> 583187 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr x:overwrite_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=overwrite&version=2&replicaType=NRT&async=0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062&coreNodeName=core_node2&name=overwrite_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&wt=javabin 2> 583197 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 2> 583198 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=3 2> 583256 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 583261 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.s.IndexSchema Schema name=test 2> 583288 WARN (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 583289 WARN (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 583289 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 583290 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'overwrite_shard1_replica_n1' using configuration from configset conf, trusted=true 2> 583293 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/overwrite_shard1_replica_n1], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/overwrite_shard1_replica_n1/data/] 2> 583459 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 583459 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 583472 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 583473 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 583489 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 583493 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 583496 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 583497 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736746672947200 2> 583525 INFO (searcherExecutor-1210-thread-1-processing-overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 core_node2 create 127.0.0.1:39159_solr overwrite shard1) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 583549 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/overwrite/terms/shard1 to Terms{values={core_node2=0}, version=0} for registerTerm 2> 583551 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/overwrite/leaders/shard1 2> 583591 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 583591 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 583592 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39159/solr/overwrite_shard1_replica_n1/ 2> 583596 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 583598 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:39159/solr/overwrite_shard1_replica_n1/ has no replicas 2> 583599 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/overwrite/leaders/shard1/leader after winning as /collections/overwrite/leader_elect/shard1/election/72081801657778182-core_node2-n_0000000000 2> 583614 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39159/solr/overwrite_shard1_replica_n1/ shard1 2> 583633 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 create) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 584006 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=0a03e15a-d894-4881-8acd-90396ec5d74f&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=6 2> 584205 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062 2> 584205 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=0a03e15a-d894-4881-8acd-90396ec5d74f369385561448062&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 584224 INFO (OverseerThreadFactory-1176-thread-2) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 585002 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 585014 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=0a03e15a-d894-4881-8acd-90396ec5d74f&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=4 2> 585024 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=0a03e15a-d894-4881-8acd-90396ec5d74f&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=6 2> 585026 INFO (TEST-CloudSolrClientTest.testOverwriteOption-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForActiveCollection: overwrite 2> 585333 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/overwrite/terms/shard1 to Terms{values={core_node2=1}, version=1} for ensureHighestTermsAreNotZero 2> 585335 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=overwrite:4&wt=javabin&version=2} status=0 QTime=289 2> 585513 INFO (searcherExecutor-1210-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 overwrite shard1 core_node2) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 585515 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=overwrite:4&commit=true&wt=javabin&version=2} status=0 QTime=174 2> 585521 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=overwrite:4&wt=javabin&version=2} rid=127.0.0.1-1348 hits=1 status=0 QTime=2 2> 585537 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=overwrite:4&wt=javabin&version=2} status=0 QTime=10 2> 585616 INFO (searcherExecutor-1210-thread-1-processing-127.0.0.1:39159_solr overwrite_shard1_replica_n1 overwrite shard1 core_node2) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 585617 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=overwrite:4&commit=true&wt=javabin&version=2} status=0 QTime=76 2> 585625 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:overwrite] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=overwrite&wt=javabin&version=2} status=0 QTime=5 2> 585634 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:overwrite] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=overwrite&wt=javabin&version=2} status=0 QTime=4 2> 585643 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=5 2> 585652 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:overwrite] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=overwrite&wt=javabin&version=2} status=0 QTime=4 2> 585660 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:overwrite] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=overwrite&wt=javabin&version=2} status=0 QTime=4 2> 585669 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=overwrite:4&wt=javabin&version=2} rid=127.0.0.1-1349 hits=3 status=0 QTime=2 2> 585675 INFO (TEST-CloudSolrClientTest.testOverwriteOption-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Ending testOverwriteOption 2> 585838 INFO (TEST-CloudSolrClientTest.testVersionsAreReturned-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Starting testVersionsAreReturned 2> 585906 INFO (OverseerThreadFactory-1176-thread-3) [] o.a.s.c.a.c.CreateCollectionCmd Create collection testVersionsAreReturned 2> 586074 INFO (OverseerThreadFactory-1176-thread-3) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"testVersionsAreReturned_shard2_replica_n1", 2> "node_name":"127.0.0.1:39159_solr", 2> "base_url":"http://127.0.0.1:39159/solr", 2> "collection":"testVersionsAreReturned", 2> "shard":"shard2", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 586084 INFO (OverseerThreadFactory-1176-thread-3) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"testVersionsAreReturned_shard1_replica_n2", 2> "node_name":"127.0.0.1:40391_solr", 2> "base_url":"http://127.0.0.1:40391/solr", 2> "collection":"testVersionsAreReturned", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 586115 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr x:testVersionsAreReturned_shard2_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=testVersionsAreReturned_shard2_replica_n1&action=CREATE&numShards=2&collection=testVersionsAreReturned&shard=shard2&wt=javabin&version=2&replicaType=NRT 2> 586120 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr x:testVersionsAreReturned_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=testVersionsAreReturned_shard1_replica_n2&action=CREATE&numShards=2&collection=testVersionsAreReturned&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 586214 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 586227 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 586231 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.s.IndexSchema Schema name=test 2> 586236 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.s.IndexSchema Schema name=test 2> 586269 WARN (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 586271 WARN (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 586272 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 586272 WARN (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 586274 WARN (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 586274 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 586276 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testVersionsAreReturned_shard2_replica_n1' using configuration from configset conf, trusted=true 2> 586278 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'testVersionsAreReturned_shard1_replica_n2' using configuration from configset conf, trusted=true 2> 586279 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/testVersionsAreReturned_shard2_replica_n1], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/testVersionsAreReturned_shard2_replica_n1/data/] 2> 586283 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/testVersionsAreReturned_shard1_replica_n2], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/testVersionsAreReturned_shard1_replica_n2/data/] 2> 586521 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 586521 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 586529 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 586530 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 586538 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 586538 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 586558 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 586559 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 586586 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 586587 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 586592 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 586595 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 586599 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 586600 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736749926678528 2> 586600 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 586602 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736749928775680 2> 586630 INFO (searcherExecutor-1216-thread-1-processing-127.0.0.1:39159_solr testVersionsAreReturned_shard2_replica_n1 testVersionsAreReturned shard2 core_node3) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 586666 INFO (searcherExecutor-1218-thread-1-processing-127.0.0.1:40391_solr testVersionsAreReturned_shard1_replica_n2 testVersionsAreReturned shard1 core_node4) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 586689 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testVersionsAreReturned/terms/shard1 to Terms{values={core_node4=0}, version=0} for registerTerm 2> 586697 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/testVersionsAreReturned/leaders/shard1 2> 586701 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testVersionsAreReturned/terms/shard2 to Terms{values={core_node3=0}, version=0} for registerTerm 2> 586702 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/testVersionsAreReturned/leaders/shard2 2> 586744 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 586745 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 586745 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40391/solr/testVersionsAreReturned_shard1_replica_n2/ 2> 586749 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 586752 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 586752 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 586752 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39159/solr/testVersionsAreReturned_shard2_replica_n1/ 2> 586753 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:40391/solr/testVersionsAreReturned_shard1_replica_n2/ has no replicas 2> 586754 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/testVersionsAreReturned/leaders/shard1/leader after winning as /collections/testVersionsAreReturned/leader_elect/shard1/election/72081801657778183-core_node4-n_0000000000 2> 586758 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 586762 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:39159/solr/testVersionsAreReturned_shard2_replica_n1/ has no replicas 2> 586762 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/testVersionsAreReturned/leaders/shard2/leader after winning as /collections/testVersionsAreReturned/leader_elect/shard2/election/72081801657778182-core_node3-n_0000000000 2> 586780 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40391/solr/testVersionsAreReturned_shard1_replica_n2/ shard1 2> 586787 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39159/solr/testVersionsAreReturned_shard2_replica_n1/ shard2 2> 586815 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 586837 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 586874 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=testVersionsAreReturned_shard1_replica_n2&action=CREATE&numShards=2&collection=testVersionsAreReturned&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=755 2> 586880 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testVersionsAreReturned/state.json] for collection [testVersionsAreReturned] has occurred - updating... (live nodes size: [3]) 2> 586890 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=testVersionsAreReturned_shard2_replica_n1&action=CREATE&numShards=2&collection=testVersionsAreReturned&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=777 2> 586916 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas 2> 586916 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf&name=testVersionsAreReturned&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=1075 2> 586918 INFO (TEST-CloudSolrClientTest.testVersionsAreReturned-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForActiveCollection: testVersionsAreReturned 2> 586943 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:testVersionsAreReturned] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testVersionsAreReturned&wt=javabin&version=2} status=0 QTime=5 2> 586952 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testVersionsAreReturned] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testVersionsAreReturned&wt=javabin&version=2} status=0 QTime=5 2> 586962 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:testVersionsAreReturned] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testVersionsAreReturned&wt=javabin&version=2} status=0 QTime=5 2> 587001 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testVersionsAreReturned/terms/shard2 to Terms{values={core_node3=1}, version=1} for ensureHighestTermsAreNotZero 2> 587002 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=testVersionsAreReturned:7&versions=true&wt=javabin&version=2} status=0 QTime=35 2> 587019 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testVersionsAreReturned/terms/shard1 to Terms{values={core_node4=1}, version=1} for ensureHighestTermsAreNotZero 2> 587020 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=testVersionsAreReturned:7&versions=true&wt=javabin&version=2} status=0 QTime=52 2> 587123 INFO (searcherExecutor-1216-thread-1-processing-127.0.0.1:39159_solr testVersionsAreReturned_shard2_replica_n1 testVersionsAreReturned shard2 core_node3) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 587133 INFO (searcherExecutor-1218-thread-1-processing-127.0.0.1:40391_solr testVersionsAreReturned_shard1_replica_n2 testVersionsAreReturned shard1 core_node4) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 587135 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:39159/solr/testVersionsAreReturned_shard2_replica_n1/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false} status=0 QTime=97 2> 587137 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=testVersionsAreReturned:7&versions=true&commit=true&wt=javabin&version=2} status=0 QTime=112 2> 587151 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testVersionsAreReturned:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:39159/solr/testVersionsAreReturned_shard2_replica_n1/&rid=127.0.0.1-1350&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335554597&isShard=true&wt=javabin} hits=1 status=0 QTime=2 2> 587156 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testVersionsAreReturned:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:40391/solr/testVersionsAreReturned_shard1_replica_n2/&rid=127.0.0.1-1350&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335554597&isShard=true&wt=javabin} hits=1 status=0 QTime=2 2> 587163 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testVersionsAreReturned:7&shards.purpose=64&shard.url=http://127.0.0.1:40391/solr/testVersionsAreReturned_shard1_replica_n2/&rid=127.0.0.1-1350&version=2&q=*:*&omitHeader=false&NOW=1696335554597&ids=1&isShard=true&wt=javabin} status=0 QTime=2 2> 587168 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testVersionsAreReturned:7&shards.purpose=64&shard.url=http://127.0.0.1:39159/solr/testVersionsAreReturned_shard2_replica_n1/&rid=127.0.0.1-1350&version=2&q=*:*&omitHeader=false&NOW=1696335554597&ids=2&isShard=true&wt=javabin} status=0 QTime=2 2> 587171 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=testVersionsAreReturned:7&wt=javabin&version=2} rid=127.0.0.1-1350 hits=2 status=0 QTime=27 2> 587180 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:testVersionsAreReturned] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testVersionsAreReturned&wt=javabin&version=2} status=0 QTime=5 2> 587189 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testVersionsAreReturned] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testVersionsAreReturned&wt=javabin&version=2} status=0 QTime=5 2> 587198 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testVersionsAreReturned] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testVersionsAreReturned&wt=javabin&version=2} status=0 QTime=5 2> 587208 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=testVersionsAreReturned:7&versions=true&wt=javabin&version=2} status=0 QTime=5 2> 587230 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:40391/solr/testVersionsAreReturned_shard1_replica_n2/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false} status=0 QTime=9 2> 587259 INFO (searcherExecutor-1218-thread-1-processing-127.0.0.1:40391_solr testVersionsAreReturned_shard1_replica_n2 testVersionsAreReturned shard1 core_node4) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 587264 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=testVersionsAreReturned:7&versions=true&commit=true&wt=javabin&version=2} status=0 QTime=52 2> 587268 INFO (TEST-CloudSolrClientTest.testVersionsAreReturned-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Ending testVersionsAreReturned 2> 587436 INFO (TEST-CloudSolrClientTest.testWrongZkChrootTest-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Starting testWrongZkChrootTest 2> 587444 WARN (TEST-CloudSolrClientTest.testWrongZkChrootTest-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 587456 INFO (TEST-CloudSolrClientTest.testWrongZkChrootTest-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 60000ms for client to connect to ZooKeeper 2> 587464 INFO (zkConnectionManagerCallback-1231-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 587464 INFO (TEST-CloudSolrClientTest.testWrongZkChrootTest-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 587465 WARN (TEST-CloudSolrClientTest.testWrongZkChrootTest-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 587476 WARN (TEST-CloudSolrClientTest.testWrongZkChrootTest-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ZkStateReader Error fetching collection names: 2> => org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:118) 2> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:118) ~[zookeeper-3.8.1.jar:3.8.1] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[zookeeper-3.8.1.jar:3.8.1] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:2366) ~[zookeeper-3.8.1.jar:3.8.1] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$getChildren$4(SolrZkClient.java:383) ~[solr-solrj-zookeeper-9.2.1.jar:?] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:72) ~[solr-solrj-zookeeper-9.2.1.jar:?] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:383) ~[solr-solrj-zookeeper-9.2.1.jar:?] 2> at org.apache.solr.common.cloud.ZkStateReader.refreshCollectionList(ZkStateReader.java:722) [solr-solrj-zookeeper-9.2.1.jar:?] 2> at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:571) [solr-solrj-zookeeper-9.2.1.jar:?] 2> at org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.getZkStateReader(ZkClientClusterStateProvider.java:214) [solr-solrj-zookeeper-9.2.1.jar:?] 2> at org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:198) [solr-solrj-zookeeper-9.2.1.jar:?] 2> at org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:364) [main/:?] 2> at org.apache.lucene.tests.util.LuceneTestCase._expectThrows(LuceneTestCase.java:3012) [lucene-test-framework-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39] 2> at org.apache.lucene.tests.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2849) [lucene-test-framework-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39] 2> at org.apache.lucene.tests.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2840) [lucene-test-framework-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39] 2> at org.apache.solr.client.solrj.impl.CloudSolrClientTest.testWrongZkChrootTest(CloudSolrClientTest.java:843) [test/:?] 2> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] 2> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?] 2> at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] 2> at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) [randomizedtesting-runner-2.8.1.jar:?] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) [randomizedtesting-runner-2.8.1.jar:?] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) [randomizedtesting-runner-2.8.1.jar:?] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) [randomizedtesting-runner-2.8.1.jar:?] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) [randomizedtesting-runner-2.8.1.jar:?] 2> at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2] 2> at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:44) [lucene-test-framework-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39] 2> at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39] 2> at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) [lucene-test-framework-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39] 2> at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) [lucene-test-framework-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39] 2> at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) [lucene-test-framework-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39] 2> at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) [randomizedtesting-runner-2.8.1.jar:?] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) [randomizedtesting-runner-2.8.1.jar:?] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) [randomizedtesting-runner-2.8.1.jar:?] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) [randomizedtesting-runner-2.8.1.jar:?] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) [randomizedtesting-runner-2.8.1.jar:?] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) [randomizedtesting-runner-2.8.1.jar:?] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) [randomizedtesting-runner-2.8.1.jar:?] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?] 2> at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39] 2> at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) [randomizedtesting-runner-2.8.1.jar:?] 2> at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2] 2> at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?] 2> at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) [lucene-test-framework-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) [randomizedtesting-runner-2.8.1.jar:?] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) [randomizedtesting-runner-2.8.1.jar:?] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?] 2> at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) [lucene-test-framework-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39] 2> at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39] 2> at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) [lucene-test-framework-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39] 2> at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) [lucene-test-framework-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39] 2> at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) [lucene-test-framework-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39] 2> at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) [randomizedtesting-runner-2.8.1.jar:?] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) [randomizedtesting-runner-2.8.1.jar:?] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> 587623 INFO (TEST-CloudSolrClientTest.testWrongZkChrootTest-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Ending testWrongZkChrootTest 2> 587790 INFO (TEST-CloudSolrClientTest.checkCollectionParameters-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Starting checkCollectionParameters 2> 587797 WARN (TEST-CloudSolrClientTest.checkCollectionParameters-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 587805 INFO (TEST-CloudSolrClientTest.checkCollectionParameters-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 15000ms for client to connect to ZooKeeper 2> 587816 INFO (zkConnectionManagerCallback-1236-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 587816 INFO (TEST-CloudSolrClientTest.checkCollectionParameters-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 587817 WARN (TEST-CloudSolrClientTest.checkCollectionParameters-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 587824 INFO (TEST-CloudSolrClientTest.checkCollectionParameters-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3) 2> 587831 INFO (TEST-CloudSolrClientTest.checkCollectionParameters-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40255/solr ready 2> 587883 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=6999340a-c3e5-4a48-a8bf-ac8483f715e0&collection.configName=conf&name=multicollection1&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=47 2> 587924 INFO (OverseerThreadFactory-1176-thread-4) [] o.a.s.c.a.c.CreateCollectionCmd Create collection multicollection1 2> 587926 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=9f09dac1-3f38-4903-8bd4-0ee3b5e533b6&collection.configName=conf&name=multicollection2&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=26 2> 587930 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000003 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 587943 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=6999340a-c3e5-4a48-a8bf-ac8483f715e0&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=10 2> 587977 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CreateCollectionCmd Create collection multicollection2 2> 588131 INFO (OverseerThreadFactory-1176-thread-4) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"multicollection1_shard2_replica_n1", 2> "node_name":"127.0.0.1:39159_solr", 2> "base_url":"http://127.0.0.1:39159/solr", 2> "collection":"multicollection1", 2> "shard":"shard2", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 588152 INFO (OverseerThreadFactory-1176-thread-4) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"multicollection1_shard1_replica_n2", 2> "node_name":"127.0.0.1:40391_solr", 2> "base_url":"http://127.0.0.1:40391/solr", 2> "collection":"multicollection1", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 588186 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"multicollection2_shard2_replica_n1", 2> "node_name":"127.0.0.1:39159_solr", 2> "base_url":"http://127.0.0.1:39159/solr", 2> "collection":"multicollection2", 2> "shard":"shard2", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 588204 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr x:multicollection1_shard2_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=multicollection1&version=2&replicaType=NRT&async=6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773&coreNodeName=core_node3&name=multicollection1_shard2_replica_n1&action=CREATE&numShards=2&shard=shard2&wt=javabin} status=0 QTime=6 2> 588205 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr x:multicollection1_shard2_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=multicollection1&version=2&replicaType=NRT&async=6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773&coreNodeName=core_node3&name=multicollection1_shard2_replica_n1&action=CREATE&numShards=2&shard=shard2&wt=javabin 2> 588208 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr x:multicollection1_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=multicollection1&version=2&replicaType=NRT&async=6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542&coreNodeName=core_node4&name=multicollection1_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&wt=javabin} status=0 QTime=6 2> 588208 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr x:multicollection1_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=multicollection1&version=2&replicaType=NRT&async=6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542&coreNodeName=core_node4&name=multicollection1_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&wt=javabin 2> 588216 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 2> 588216 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 588219 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"multicollection2_shard1_replica_n2", 2> "node_name":"127.0.0.1:40391_solr", 2> "base_url":"http://127.0.0.1:40391/solr", 2> "collection":"multicollection2", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 588282 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr x:multicollection2_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=multicollection2&version=2&replicaType=NRT&async=9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598&coreNodeName=core_node4&name=multicollection2_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&wt=javabin} status=0 QTime=9 2> 588284 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr x:multicollection2_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=multicollection2&version=2&replicaType=NRT&async=9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598&coreNodeName=core_node4&name=multicollection2_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&wt=javabin 2> 588286 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr x:multicollection2_shard2_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=multicollection2&version=2&replicaType=NRT&async=9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487&coreNodeName=core_node3&name=multicollection2_shard2_replica_n1&action=CREATE&numShards=2&shard=shard2&wt=javabin} status=0 QTime=13 2> 588290 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr x:multicollection2_shard2_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=multicollection2&version=2&replicaType=NRT&async=9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487&coreNodeName=core_node3&name=multicollection2_shard2_replica_n1&action=CREATE&numShards=2&shard=shard2&wt=javabin 2> 588296 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 2> 588296 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 588352 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 588360 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.s.IndexSchema Schema name=test 2> 588401 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 588401 WARN (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 588401 WARN (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 588402 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 588404 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'multicollection1_shard2_replica_n1' using configuration from configset conf, trusted=true 2> 588408 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/multicollection1_shard2_replica_n1], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/multicollection1_shard2_replica_n1/data/] 2> 588413 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.s.IndexSchema Schema name=test 2> 588485 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 588485 WARN (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 588487 WARN (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 588489 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 588491 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'multicollection1_shard1_replica_n2' using configuration from configset conf, trusted=true 2> 588496 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.s.IndexSchema Schema name=test 2> 588498 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/multicollection1_shard1_replica_n2], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/multicollection1_shard1_replica_n2/data/] 2> 588518 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 588528 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.s.IndexSchema Schema name=test 2> 588557 WARN (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 588557 WARN (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 588558 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 588561 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'multicollection2_shard1_replica_n2' using configuration from configset conf, trusted=true 2> 588565 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/multicollection2_shard1_replica_n2], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/multicollection2_shard1_replica_n2/data/] 2> 588598 WARN (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 588607 WARN (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 588608 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 588612 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'multicollection2_shard2_replica_n1' using configuration from configset conf, trusted=true 2> 588616 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/multicollection2_shard2_replica_n1], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/multicollection2_shard2_replica_n1/data/] 2> 588804 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 588805 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 588848 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 588848 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 588911 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 588915 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 588922 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 588924 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736752363569152 2> 588961 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=6999340a-c3e5-4a48-a8bf-ac8483f715e0&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=12 2> 588971 INFO (searcherExecutor-1238-thread-1-processing-multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 core_node3 create 127.0.0.1:39159_solr multicollection1 shard2) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 589018 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/multicollection1/terms/shard2 to Terms{values={core_node3=0}, version=0} for registerTerm 2> 589020 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/multicollection1/leaders/shard2 2> 589029 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 589029 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 589064 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 589064 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 589069 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 589069 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 589070 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39159/solr/multicollection1_shard2_replica_n1/ 2> 589074 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 589078 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:39159/solr/multicollection1_shard2_replica_n1/ has no replicas 2> 589078 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/multicollection1/leaders/shard2/leader after winning as /collections/multicollection1/leader_elect/shard2/election/72081801657778182-core_node3-n_0000000000 2> 589103 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39159/solr/multicollection1_shard2_replica_n1/ shard2 2> 589123 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 589127 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 589133 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 589133 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 589140 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 589142 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736752592158720 2> 589146 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-2-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 create) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 589158 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 589159 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 589162 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 589162 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 589203 INFO (searcherExecutor-1240-thread-1-processing-multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 core_node4 create 127.0.0.1:40391_solr multicollection1 shard1) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 589206 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 589208 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 589216 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 589221 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 589223 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773 2> 589224 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=6999340a-c3e5-4a48-a8bf-ac8483f715e0369390523829773&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 589230 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 589230 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 2> 589231 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 589231 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736752685481984 2> 589256 INFO (searcherExecutor-1245-thread-1-processing-multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 core_node3 create 127.0.0.1:39159_solr multicollection2 shard2) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 589268 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 589273 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 589276 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/multicollection1/terms/shard1 to Terms{values={core_node4=0}, version=0} for registerTerm 2> 589277 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/multicollection1/leaders/shard1 2> 589287 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 589289 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736752746299392 2> 589303 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 2> 589304 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 589339 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/multicollection2/terms/shard2 to Terms{values={core_node3=0}, version=0} for registerTerm 2> 589340 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/multicollection2/leaders/shard2 2> 589341 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 589342 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 589342 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40391/solr/multicollection1_shard1_replica_n2/ 2> 589347 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 589352 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:40391/solr/multicollection1_shard1_replica_n2/ has no replicas 2> 589353 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/multicollection1/leaders/shard1/leader after winning as /collections/multicollection1/leader_elect/shard1/election/72081801657778183-core_node4-n_0000000000 2> 589363 INFO (searcherExecutor-1243-thread-1-processing-multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 core_node4 create 127.0.0.1:40391_solr multicollection2 shard1) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 589389 INFO (zkCallback-1160-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/multicollection1/state.json] for collection [multicollection1] has occurred - updating... (live nodes size: [3]) 2> 589390 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40391/solr/multicollection1_shard1_replica_n2/ shard1 2> 589400 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/multicollection2/terms/shard1 to Terms{values={core_node4=0}, version=0} for registerTerm 2> 589403 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/multicollection2/leaders/shard1 2> 589409 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 589409 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 589409 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39159/solr/multicollection2_shard2_replica_n1/ 2> 589415 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 589420 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:39159/solr/multicollection2_shard2_replica_n1/ has no replicas 2> 589421 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/multicollection2/leaders/shard2/leader after winning as /collections/multicollection2/leader_elect/shard2/election/72081801657778182-core_node3-n_0000000000 2> 589440 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 create) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 589465 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39159/solr/multicollection2_shard2_replica_n1/ shard2 2> 589473 INFO (zkCallback-1160-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/multicollection1/state.json] for collection [multicollection1] has occurred - updating... (live nodes size: [3]) 2> 589490 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 589491 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 589491 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40391/solr/multicollection2_shard1_replica_n2/ 2> 589497 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 589507 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:40391/solr/multicollection2_shard1_replica_n2/ has no replicas 2> 589507 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/multicollection2/leaders/shard1/leader after winning as /collections/multicollection2/leader_elect/shard1/election/72081801657778183-core_node4-n_0000000000 2> 589514 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-3-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 create) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 589539 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40391/solr/multicollection2_shard1_replica_n2/ shard1 2> 589568 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-2-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 create) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 589581 INFO (zkCallback-1160-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/multicollection2/state.json] for collection [multicollection2] has occurred - updating... (live nodes size: [3]) 2> 589971 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=6999340a-c3e5-4a48-a8bf-ac8483f715e0&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=6 2> 590236 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542 2> 590237 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=6999340a-c3e5-4a48-a8bf-ac8483f715e0369390540560542&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 590255 INFO (OverseerThreadFactory-1176-thread-4) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000005 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 590313 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487 2> 590313 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390576809487&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 590321 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598 2> 590321 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=9f09dac1-3f38-4903-8bd4-0ee3b5e533b6369390593825598&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 590343 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 590980 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=6999340a-c3e5-4a48-a8bf-ac8483f715e0&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=5 2> 590991 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=6999340a-c3e5-4a48-a8bf-ac8483f715e0&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=7 2> 590999 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=9f09dac1-3f38-4903-8bd4-0ee3b5e533b6&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=5 2> 591010 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=9f09dac1-3f38-4903-8bd4-0ee3b5e533b6&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=6 2> 591011 INFO (TEST-CloudSolrClientTest.checkCollectionParameters-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForActiveCollection: multicollection1 2> 591027 INFO (TEST-CloudSolrClientTest.checkCollectionParameters-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForActiveCollection: multicollection2 2> 591086 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/multicollection1/terms/shard2 to Terms{values={core_node3=1}, version=1} for ensureHighestTermsAreNotZero 2> 591089 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={wt=javabin&version=2} status=0 QTime=37 2> 591101 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/multicollection1/terms/shard1 to Terms{values={core_node4=1}, version=1} for ensureHighestTermsAreNotZero 2> 591103 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/update params={wt=javabin&version=2} status=0 QTime=46 2> 591230 INFO (searcherExecutor-1240-thread-1-processing-127.0.0.1:40391_solr multicollection1_shard1_replica_n2 multicollection1 shard1 core_node4) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 591232 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:39159/solr/multicollection1_shard2_replica_n1/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false} status=0 QTime=112 2> 591242 INFO (searcherExecutor-1238-thread-1-processing-127.0.0.1:39159_solr multicollection1_shard2_replica_n1 multicollection1 shard2 core_node3) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 591245 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=multicollection1:7&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=136 2> 591260 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=multicollection1:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:39159/solr/multicollection1_shard2_replica_n1/&rid=127.0.0.1-1351&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335558706&isShard=true&wt=javabin} hits=1 status=0 QTime=1 2> 591266 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=multicollection1:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:40391/solr/multicollection1_shard1_replica_n2/&rid=127.0.0.1-1351&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335558706&isShard=true&wt=javabin} hits=2 status=0 QTime=1 2> 591274 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=multicollection1:7&shards.purpose=64&shard.url=http://127.0.0.1:40391/solr/multicollection1_shard1_replica_n2/&rid=127.0.0.1-1351&version=2&q=*:*&omitHeader=false&NOW=1696335558706&ids=0,1&isShard=true&wt=javabin} status=0 QTime=2 2> 591280 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=multicollection1:7&shards.purpose=64&shard.url=http://127.0.0.1:39159/solr/multicollection1_shard2_replica_n1/&rid=127.0.0.1-1351&version=2&q=*:*&omitHeader=false&NOW=1696335558706&ids=2&isShard=true&wt=javabin} status=0 QTime=2 2> 591282 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=multicollection1:7&wt=javabin&version=2} rid=127.0.0.1-1351 hits=3 status=0 QTime=30 2> 591297 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=multicollection2:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:39159/solr/multicollection2_shard2_replica_n1/&rid=127.0.0.1-1352&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335558743&isShard=true&wt=javabin} hits=0 status=0 QTime=1 2> 591301 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=multicollection2:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:40391/solr/multicollection2_shard1_replica_n2/&rid=127.0.0.1-1352&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335558743&isShard=true&wt=javabin} hits=0 status=0 QTime=1 2> 591304 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=multicollection2:7&wt=javabin&version=2} rid=127.0.0.1-1352 hits=0 status=0 QTime=13 2> 591314 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=multicollection2:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&collection=multicollection2&shard.url=http://127.0.0.1:39159/solr/multicollection2_shard2_replica_n1/&rid=127.0.0.1-1353&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335558761&isShard=true&wt=javabin} hits=0 status=0 QTime=1 2> 591321 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=multicollection2:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&collection=multicollection2&shard.url=http://127.0.0.1:40391/solr/multicollection2_shard1_replica_n2/&rid=127.0.0.1-1353&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335558761&isShard=true&wt=javabin} hits=0 status=0 QTime=1 2> 591328 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=multicollection2:7&collection=multicollection2&wt=javabin&version=2} rid=127.0.0.1-1353 hits=0 status=0 QTime=20 2> 591362 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/multicollection2/terms/shard2 to Terms{values={core_node3=1}, version=1} for ensureHighestTermsAreNotZero 2> 591366 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={wt=javabin&version=2} status=0 QTime=29 2> 591371 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/multicollection2/terms/shard1 to Terms{values={core_node4=1}, version=1} for ensureHighestTermsAreNotZero 2> 591372 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/update params={wt=javabin&version=2} status=0 QTime=37 2> 591496 INFO (searcherExecutor-1243-thread-1-processing-127.0.0.1:40391_solr multicollection2_shard1_replica_n2 multicollection2 shard1 core_node4) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 591505 INFO (searcherExecutor-1245-thread-1-processing-127.0.0.1:39159_solr multicollection2_shard2_replica_n1 multicollection2 shard2 core_node3) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 591507 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:40391/solr/multicollection2_shard1_replica_n2/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false} status=0 QTime=120 2> 591509 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=multicollection2:7&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=131 2> 591521 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=multicollection2:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:39159/solr/multicollection2_shard2_replica_n1/&rid=127.0.0.1-1354&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335558967&isShard=true&wt=javabin} hits=1 status=0 QTime=2 2> 591524 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=multicollection2:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:40391/solr/multicollection2_shard1_replica_n2/&rid=127.0.0.1-1354&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335558967&isShard=true&wt=javabin} hits=2 status=0 QTime=3 2> 591535 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=multicollection2:7&shards.purpose=64&shard.url=http://127.0.0.1:40391/solr/multicollection2_shard1_replica_n2/&rid=127.0.0.1-1354&version=2&q=*:*&omitHeader=false&NOW=1696335558967&ids=0,1&isShard=true&wt=javabin} status=0 QTime=2 2> 591540 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=multicollection2:7&shards.purpose=64&shard.url=http://127.0.0.1:39159/solr/multicollection2_shard2_replica_n1/&rid=127.0.0.1-1354&version=2&q=*:*&omitHeader=false&NOW=1696335558967&ids=2&isShard=true&wt=javabin} status=0 QTime=9 2> 591543 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=multicollection2:7&wt=javabin&version=2} rid=127.0.0.1-1354 hits=3 status=0 QTime=29 2> 591657 INFO (TEST-CloudSolrClientTest.checkCollectionParameters-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Ending checkCollectionParameters 2> 591854 INFO (TEST-CloudSolrClientTest.stateVersionParamTest-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Starting stateVersionParamTest 2> 591914 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CreateCollectionCmd Create collection stateVersionParamTest 2> 591918 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000005 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 591926 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 592092 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"stateVersionParamTest_shard1_replica_n1", 2> "node_name":"127.0.0.1:40391_solr", 2> "base_url":"http://127.0.0.1:40391/solr", 2> "collection":"stateVersionParamTest", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 592103 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"stateVersionParamTest_shard2_replica_n2", 2> "node_name":"127.0.0.1:39159_solr", 2> "base_url":"http://127.0.0.1:39159/solr", 2> "collection":"stateVersionParamTest", 2> "shard":"shard2", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 595551 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr x:stateVersionParamTest_shard2_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=stateVersionParamTest_shard2_replica_n2&action=CREATE&numShards=2&collection=stateVersionParamTest&shard=shard2&wt=javabin&version=2&replicaType=NRT 2> 595555 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr x:stateVersionParamTest_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=stateVersionParamTest_shard1_replica_n1&action=CREATE&numShards=2&collection=stateVersionParamTest&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 595690 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 595695 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 595700 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.s.IndexSchema Schema name=test 2> 595705 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.s.IndexSchema Schema name=test 2> 595748 WARN (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 595748 WARN (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 595749 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 595749 WARN (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 595750 WARN (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 595750 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'stateVersionParamTest_shard2_replica_n2' using configuration from configset conf, trusted=true 2> 595751 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 595753 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'stateVersionParamTest_shard1_replica_n1' using configuration from configset conf, trusted=true 2> 595755 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/stateVersionParamTest_shard2_replica_n2], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/stateVersionParamTest_shard2_replica_n2/data/] 2> 595757 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/stateVersionParamTest_shard1_replica_n1], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/stateVersionParamTest_shard1_replica_n1/data/] 2> 596085 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 596085 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 596095 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 596096 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 596105 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 596105 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 596115 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 596116 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 596134 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 596138 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 596146 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 596146 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 596148 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736759938482176 2> 596151 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 596157 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 596159 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736759950016512 2> 596218 INFO (searcherExecutor-1264-thread-1-processing-127.0.0.1:40391_solr stateVersionParamTest_shard1_replica_n1 stateVersionParamTest shard1 core_node3) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 596225 INFO (searcherExecutor-1262-thread-1-processing-127.0.0.1:39159_solr stateVersionParamTest_shard2_replica_n2 stateVersionParamTest shard2 core_node4) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 596244 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/stateVersionParamTest/terms/shard2 to Terms{values={core_node4=0}, version=0} for registerTerm 2> 596245 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/stateVersionParamTest/leaders/shard2 2> 596258 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/stateVersionParamTest/terms/shard1 to Terms{values={core_node3=0}, version=0} for registerTerm 2> 596260 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/stateVersionParamTest/leaders/shard1 2> 596298 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 596298 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 596298 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39159/solr/stateVersionParamTest_shard2_replica_n2/ 2> 596302 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 596305 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:39159/solr/stateVersionParamTest_shard2_replica_n2/ has no replicas 2> 596306 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/stateVersionParamTest/leaders/shard2/leader after winning as /collections/stateVersionParamTest/leader_elect/shard2/election/72081801657778182-core_node4-n_0000000000 2> 596319 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 596320 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 596320 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40391/solr/stateVersionParamTest_shard1_replica_n1/ 2> 596323 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 596326 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:40391/solr/stateVersionParamTest_shard1_replica_n1/ has no replicas 2> 596327 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/stateVersionParamTest/leaders/shard1/leader after winning as /collections/stateVersionParamTest/leader_elect/shard1/election/72081801657778183-core_node3-n_0000000000 2> 596339 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39159/solr/stateVersionParamTest_shard2_replica_n2/ shard2 2> 596374 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40391/solr/stateVersionParamTest_shard1_replica_n1/ shard1 2> 596420 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 596423 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 596473 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=stateVersionParamTest_shard1_replica_n1&action=CREATE&numShards=2&collection=stateVersionParamTest&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=926 2> 596498 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=stateVersionParamTest_shard2_replica_n2&action=CREATE&numShards=2&collection=stateVersionParamTest&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=950 2> 596520 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas 2> 596521 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf&name=stateVersionParamTest&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=4657 2> 596524 INFO (TEST-CloudSolrClientTest.stateVersionParamTest-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForActiveCollection: stateVersionParamTest 2> 596587 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:39159/solr/stateVersionParamTest_shard2_replica_n2/&rid=127.0.0.1-1355&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335564024&isShard=true&wt=javabin} hits=0 status=0 QTime=5 2> 596596 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:40391/solr/stateVersionParamTest_shard1_replica_n1/&rid=127.0.0.1-1355&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335564024&isShard=true&wt=javabin} hits=0 status=0 QTime=2 2> 596604 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} rid=127.0.0.1-1355 hits=0 status=0 QTime=30 2> 596607 INFO (TEST-CloudSolrClientTest.stateVersionParamTest-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest should work query, result {responseHeader={zkConnected=true, status=0, QTime=30}, response={numFound=0,numFoundExact=true,start=0,maxScore=0.0,docs=[]}} 2> 596622 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=stateVersionParamTest:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:39159/solr/stateVersionParamTest_shard2_replica_n2/&rid=127.0.0.1-1356&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335564068&isShard=true&wt=javabin} hits=0 status=0 QTime=2 2> 596624 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=stateVersionParamTest:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:40391/solr/stateVersionParamTest_shard1_replica_n1/&rid=127.0.0.1-1356&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335564068&isShard=true&wt=javabin} hits=0 status=0 QTime=2 2> 596630 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=stateVersionParamTest:7&wt=javabin&version=2} rid=127.0.0.1-1356 hits=0 status=0 QTime=15 2> 596632 INFO (TEST-CloudSolrClientTest.stateVersionParamTest-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest 2nd query , result {responseHeader={zkConnected=true, status=0, QTime=15}, response={numFound=0,numFoundExact=true,start=0,maxScore=0.0,docs=[]}} 2> 596645 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=stateVersionParamTest:6&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:39159/solr/stateVersionParamTest_shard2_replica_n2/&rid=127.0.0.1-1357&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335564090&isShard=true&wt=javabin} hits=0 status=0 QTime=2 2> 596649 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=stateVersionParamTest:6&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:40391/solr/stateVersionParamTest_shard1_replica_n1/&rid=127.0.0.1-1357&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335564090&isShard=true&wt=javabin} hits=0 status=0 QTime=2 2> 596655 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=stateVersionParamTest:6&wt=javabin&version=2} rid=127.0.0.1-1357 hits=0 status=0 QTime=18 2> 596659 INFO (TEST-CloudSolrClientTest.stateVersionParamTest-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest the node which does not serve this collectionhttp://127.0.0.1:44127/solr 2> 596690 INFO (TEST-CloudSolrClientTest.stateVersionParamTest-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Ending stateVersionParamTest 2> 597028 INFO (TEST-CloudSolrClientTest.customHttpClientTest-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Starting customHttpClientTest 2> 597039 INFO (TEST-CloudSolrClientTest.customHttpClientTest-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Ending customHttpClientTest 2> 597351 INFO (TEST-CloudSolrClientTest.testCollectionDoesntExist-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Starting testCollectionDoesntExist 2> 597358 INFO (TEST-CloudSolrClientTest.testCollectionDoesntExist-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Ending testCollectionDoesntExist 2> 597525 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000007 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 597652 INFO (TEST-CloudSolrClientTest.singleShardedPreferenceRules-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Starting singleShardedPreferenceRules 2> 597676 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=1&async=34dc4d86-0fe2-4619-9ea3-256f089e85e2&collection.configName=conf&name=singleShardPreferenceTestColl&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=1&wt=javabin&version=2} status=0 QTime=19 2> 597696 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=34dc4d86-0fe2-4619-9ea3-256f089e85e2&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=12 2> 597702 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CreateCollectionCmd Create collection singleShardPreferenceTestColl 2> 597943 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"singleShardPreferenceTestColl_shard1_replica_p1", 2> "node_name":"127.0.0.1:44127_solr", 2> "base_url":"http://127.0.0.1:44127/solr", 2> "collection":"singleShardPreferenceTestColl", 2> "shard":"shard1", 2> "state":"down", 2> "type":"PULL", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 597958 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"singleShardPreferenceTestColl_shard1_replica_t2", 2> "node_name":"127.0.0.1:40391_solr", 2> "base_url":"http://127.0.0.1:40391/solr", 2> "collection":"singleShardPreferenceTestColl", 2> "shard":"shard1", 2> "state":"down", 2> "type":"TLOG", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 597976 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"singleShardPreferenceTestColl_shard1_replica_n3", 2> "node_name":"127.0.0.1:39159_solr", 2> "base_url":"http://127.0.0.1:39159/solr", 2> "collection":"singleShardPreferenceTestColl", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 598022 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=singleShardPreferenceTestColl&version=2&replicaType=TLOG&async=34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187&coreNodeName=core_node5&name=singleShardPreferenceTestColl_shard1_replica_t2&action=CREATE&numShards=1&shard=shard1&wt=javabin} status=0 QTime=4 2> 598022 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=singleShardPreferenceTestColl&version=2&replicaType=TLOG&async=34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187&coreNodeName=core_node5&name=singleShardPreferenceTestColl_shard1_replica_t2&action=CREATE&numShards=1&shard=shard1&wt=javabin 2> 598025 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=singleShardPreferenceTestColl&version=2&replicaType=NRT&async=34dc4d86-0fe2-4619-9ea3-256f089e85e2369400352996368&coreNodeName=core_node6&name=singleShardPreferenceTestColl_shard1_replica_n3&action=CREATE&numShards=1&shard=shard1&wt=javabin} status=0 QTime=4 2> 598025 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-4-processing-127.0.0.1:39159_solr singleShardPreferenceTestColl_shard1_replica_n3 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400352996368 create) [n:127.0.0.1:39159_solr x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=singleShardPreferenceTestColl&version=2&replicaType=NRT&async=34dc4d86-0fe2-4619-9ea3-256f089e85e2369400352996368&coreNodeName=core_node6&name=singleShardPreferenceTestColl_shard1_replica_n3&action=CREATE&numShards=1&shard=shard1&wt=javabin 2> 598040 INFO (qtp218128001-2523) [n:127.0.0.1:44127_solr x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=singleShardPreferenceTestColl&version=2&replicaType=PULL&async=34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161&coreNodeName=core_node4&name=singleShardPreferenceTestColl_shard1_replica_p1&action=CREATE&numShards=1&shard=shard1&wt=javabin} status=0 QTime=5 2> 598041 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-1-processing-127.0.0.1:44127_solr singleShardPreferenceTestColl_shard1_replica_p1 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161 create) [n:127.0.0.1:44127_solr x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=singleShardPreferenceTestColl&version=2&replicaType=PULL&async=34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161&coreNodeName=core_node4&name=singleShardPreferenceTestColl_shard1_replica_p1&action=CREATE&numShards=1&shard=shard1&wt=javabin 2> 598049 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161 2> 598053 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=6 2> 598176 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 598184 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.s.IndexSchema Schema name=test 2> 598197 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-4-processing-127.0.0.1:39159_solr singleShardPreferenceTestColl_shard1_replica_n3 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400352996368 create) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 598207 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-4-processing-127.0.0.1:39159_solr singleShardPreferenceTestColl_shard1_replica_n3 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400352996368 create) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.s.IndexSchema Schema name=test 2> 598237 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-1-processing-127.0.0.1:44127_solr singleShardPreferenceTestColl_shard1_replica_p1 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161 create) [n:127.0.0.1:44127_solr c:singleShardPreferenceTestColl s:shard1 r:core_node4 x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 598239 WARN (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 598239 WARN (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 598240 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 598242 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.CoreContainer Creating SolrCore 'singleShardPreferenceTestColl_shard1_replica_t2' using configuration from configset conf, trusted=true 2> 598246 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/singleShardPreferenceTestColl_shard1_replica_t2], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/singleShardPreferenceTestColl_shard1_replica_t2/data/] 2> 598279 WARN (parallelCoreAdminAPIBaseExecutor-1177-thread-4-processing-127.0.0.1:39159_solr singleShardPreferenceTestColl_shard1_replica_n3 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400352996368 create) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 598280 WARN (parallelCoreAdminAPIBaseExecutor-1177-thread-4-processing-127.0.0.1:39159_solr singleShardPreferenceTestColl_shard1_replica_n3 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400352996368 create) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 598281 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-4-processing-127.0.0.1:39159_solr singleShardPreferenceTestColl_shard1_replica_n3 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400352996368 create) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 598283 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-4-processing-127.0.0.1:39159_solr singleShardPreferenceTestColl_shard1_replica_n3 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400352996368 create) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'singleShardPreferenceTestColl_shard1_replica_n3' using configuration from configset conf, trusted=true 2> 598299 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-4-processing-127.0.0.1:39159_solr singleShardPreferenceTestColl_shard1_replica_n3 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400352996368 create) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/singleShardPreferenceTestColl_shard1_replica_n3], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/singleShardPreferenceTestColl_shard1_replica_n3/data/] 2> 598317 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-1-processing-127.0.0.1:44127_solr singleShardPreferenceTestColl_shard1_replica_p1 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161 create) [n:127.0.0.1:44127_solr c:singleShardPreferenceTestColl s:shard1 r:core_node4 x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.s.IndexSchema Schema name=test 2> 598384 WARN (parallelCoreAdminAPIBaseExecutor-1178-thread-1-processing-127.0.0.1:44127_solr singleShardPreferenceTestColl_shard1_replica_p1 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161 create) [n:127.0.0.1:44127_solr c:singleShardPreferenceTestColl s:shard1 r:core_node4 x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 598392 WARN (parallelCoreAdminAPIBaseExecutor-1178-thread-1-processing-127.0.0.1:44127_solr singleShardPreferenceTestColl_shard1_replica_p1 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161 create) [n:127.0.0.1:44127_solr c:singleShardPreferenceTestColl s:shard1 r:core_node4 x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 598395 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-1-processing-127.0.0.1:44127_solr singleShardPreferenceTestColl_shard1_replica_p1 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161 create) [n:127.0.0.1:44127_solr c:singleShardPreferenceTestColl s:shard1 r:core_node4 x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 598399 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-1-processing-127.0.0.1:44127_solr singleShardPreferenceTestColl_shard1_replica_p1 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161 create) [n:127.0.0.1:44127_solr c:singleShardPreferenceTestColl s:shard1 r:core_node4 x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.c.CoreContainer Creating SolrCore 'singleShardPreferenceTestColl_shard1_replica_p1' using configuration from configset conf, trusted=true 2> 598404 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-1-processing-127.0.0.1:44127_solr singleShardPreferenceTestColl_shard1_replica_p1 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161 create) [n:127.0.0.1:44127_solr c:singleShardPreferenceTestColl s:shard1 r:core_node4 x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node3/singleShardPreferenceTestColl_shard1_replica_p1], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node3/singleShardPreferenceTestColl_shard1_replica_p1/data/] 2> 598628 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 598628 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 598653 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 598653 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 598673 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-4-processing-127.0.0.1:39159_solr singleShardPreferenceTestColl_shard1_replica_n3 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400352996368 create) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 598673 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-4-processing-127.0.0.1:39159_solr singleShardPreferenceTestColl_shard1_replica_n3 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400352996368 create) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 598682 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 598685 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 598692 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 598693 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736762607108096 2> 598702 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-4-processing-127.0.0.1:39159_solr singleShardPreferenceTestColl_shard1_replica_n3 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400352996368 create) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 598704 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-4-processing-127.0.0.1:39159_solr singleShardPreferenceTestColl_shard1_replica_n3 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400352996368 create) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 598731 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=34dc4d86-0fe2-4619-9ea3-256f089e85e2&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=30 2> 598741 INFO (searcherExecutor-1276-thread-1-processing-singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 core_node5 create 127.0.0.1:40391_solr singleShardPreferenceTestColl shard1) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 598766 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/singleShardPreferenceTestColl/terms/shard1 to Terms{values={core_node5=0}, version=0} for registerTerm 2> 598766 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/singleShardPreferenceTestColl/leaders/shard1 2> 598780 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-4-processing-127.0.0.1:39159_solr singleShardPreferenceTestColl_shard1_replica_n3 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400352996368 create) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 598784 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-4-processing-127.0.0.1:39159_solr singleShardPreferenceTestColl_shard1_replica_n3 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400352996368 create) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 598793 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-4-processing-127.0.0.1:39159_solr singleShardPreferenceTestColl_shard1_replica_n3 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400352996368 create) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 598795 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-4-processing-127.0.0.1:39159_solr singleShardPreferenceTestColl_shard1_replica_n3 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400352996368 create) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736762714062848 2> 598828 INFO (searcherExecutor-1278-thread-1-processing-singleShardPreferenceTestColl_shard1_replica_n3 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400352996368 core_node6 create 127.0.0.1:39159_solr singleShardPreferenceTestColl shard1) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 598886 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoute in=9960ms 2> 598953 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-4-processing-127.0.0.1:39159_solr singleShardPreferenceTestColl_shard1_replica_n3 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400352996368 create) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.c.ZkShardTerms Successful update of terms at /collections/singleShardPreferenceTestColl/terms/shard1 to Terms{values={core_node6=0, core_node5=0}, version=1} for registerTerm 2> 598954 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-4-processing-127.0.0.1:39159_solr singleShardPreferenceTestColl_shard1_replica_n3 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400352996368 create) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/singleShardPreferenceTestColl/leaders/shard1 2> 599061 INFO (qtp218128001-2525) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161 2> 599061 INFO (qtp218128001-2525) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 599096 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-1-processing-127.0.0.1:44127_solr singleShardPreferenceTestColl_shard1_replica_p1 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161 create) [n:127.0.0.1:44127_solr c:singleShardPreferenceTestColl s:shard1 r:core_node4 x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 599111 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-1-processing-127.0.0.1:44127_solr singleShardPreferenceTestColl_shard1_replica_p1 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161 create) [n:127.0.0.1:44127_solr c:singleShardPreferenceTestColl s:shard1 r:core_node4 x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 599111 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-1-processing-127.0.0.1:44127_solr singleShardPreferenceTestColl_shard1_replica_p1 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161 create) [n:127.0.0.1:44127_solr c:singleShardPreferenceTestColl s:shard1 r:core_node4 x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 599136 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-1-processing-127.0.0.1:44127_solr singleShardPreferenceTestColl_shard1_replica_p1 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161 create) [n:127.0.0.1:44127_solr c:singleShardPreferenceTestColl s:shard1 r:core_node4 x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 599139 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-1-processing-127.0.0.1:44127_solr singleShardPreferenceTestColl_shard1_replica_p1 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161 create) [n:127.0.0.1:44127_solr c:singleShardPreferenceTestColl s:shard1 r:core_node4 x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 599145 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-1-processing-127.0.0.1:44127_solr singleShardPreferenceTestColl_shard1_replica_p1 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161 create) [n:127.0.0.1:44127_solr c:singleShardPreferenceTestColl s:shard1 r:core_node4 x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 599183 INFO (searcherExecutor-1280-thread-1-processing-singleShardPreferenceTestColl_shard1_replica_p1 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161 core_node4 create 127.0.0.1:44127_solr singleShardPreferenceTestColl shard1) [n:127.0.0.1:44127_solr c:singleShardPreferenceTestColl s:shard1 r:core_node4 x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 599193 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-1-processing-127.0.0.1:44127_solr singleShardPreferenceTestColl_shard1_replica_p1 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161 create) [n:127.0.0.1:44127_solr c:singleShardPreferenceTestColl s:shard1 r:core_node4 x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.c.ZkController singleShardPreferenceTestColl_shard1_replica_p1 starting background replication from leader 2> 599194 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-1-processing-127.0.0.1:44127_solr singleShardPreferenceTestColl_shard1_replica_p1 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161 create) [n:127.0.0.1:44127_solr c:singleShardPreferenceTestColl s:shard1 r:core_node4 x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:01 2> 599203 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-1-processing-127.0.0.1:44127_solr singleShardPreferenceTestColl_shard1_replica_p1 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161 create) [n:127.0.0.1:44127_solr c:singleShardPreferenceTestColl s:shard1 r:core_node4 x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms 2> 599203 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-1-processing-127.0.0.1:44127_solr singleShardPreferenceTestColl_shard1_replica_p1 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161 create) [n:127.0.0.1:44127_solr c:singleShardPreferenceTestColl s:shard1 r:core_node4 x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 599390 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 599390 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 599391 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40391/solr/singleShardPreferenceTestColl_shard1_replica_t2/ 2> 599395 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.u.PeerSync PeerSync: core=singleShardPreferenceTestColl_shard1_replica_t2 url=http://127.0.0.1:40391/solr START replicas=[http://127.0.0.1:39159/solr/singleShardPreferenceTestColl_shard1_replica_n3/] nUpdates=100 2> 599398 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.u.PeerSync PeerSync: core=singleShardPreferenceTestColl_shard1_replica_t2 url=http://127.0.0.1:40391/solr DONE. We have no versions. sync failed. 2> 599401 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.c.S.Request webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=2 2> 599404 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate 2> 599404 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway 2> 599405 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.ZkController singleShardPreferenceTestColl_shard1_replica_t2 stopping background replication from leader 2> 599405 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/singleShardPreferenceTestColl/leaders/shard1/leader after winning as /collections/singleShardPreferenceTestColl/leader_elect/shard1/election/72081801657778183-core_node5-n_0000000000 2> 599421 INFO (zkCallback-1166-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/singleShardPreferenceTestColl/state.json] for collection [singleShardPreferenceTestColl] has occurred - updating... (live nodes size: [3]) 2> 599423 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40391/solr/singleShardPreferenceTestColl_shard1_replica_t2/ shard1 2> 599426 INFO (indexFetcher-1293-thread-1) [] o.a.s.h.IndexFetcher Replica core_node5 is leader but it's state is down, skipping replication 2> 599446 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-3-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 create) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 599741 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=34dc4d86-0fe2-4619-9ea3-256f089e85e2&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=6 2> 600006 INFO (zkCallback-1164-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/singleShardPreferenceTestColl/state.json] for collection [singleShardPreferenceTestColl] has occurred - updating... (live nodes size: [3]) 2> 600067 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161 2> 600067 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 600231 INFO (zkCallback-1160-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/singleShardPreferenceTestColl/state.json] for collection [singleShardPreferenceTestColl] has occurred - updating... (live nodes size: [3]) 2> 600231 INFO (zkCallback-1164-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/singleShardPreferenceTestColl/state.json] for collection [singleShardPreferenceTestColl] has occurred - updating... (live nodes size: [3]) 2> 600433 INFO (indexFetcher-1293-thread-1) [] o.a.s.h.IndexFetcher Last replication failed, so I'll force replication 2> 600433 INFO (indexFetcher-1293-thread-1) [] o.a.s.h.IndexFetcher Updated leaderUrl to http://127.0.0.1:40391/solr/singleShardPreferenceTestColl_shard1_replica_t2/ 2> 600440 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 600441 INFO (indexFetcher-1293-thread-1) [] o.a.s.h.IndexFetcher Leader's generation: 1 2> 600442 INFO (indexFetcher-1293-thread-1) [] o.a.s.h.IndexFetcher Leader's version: 0 2> 600442 INFO (indexFetcher-1293-thread-1) [] o.a.s.h.IndexFetcher Follower's generation: 1 2> 600443 INFO (indexFetcher-1293-thread-1) [] o.a.s.h.IndexFetcher Follower's version: 0 2> 600752 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=34dc4d86-0fe2-4619-9ea3-256f089e85e2&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=7 2> 601072 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161 2> 601073 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=34dc4d86-0fe2-4619-9ea3-256f089e85e2369400306705161&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 601078 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187 2> 601079 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=34dc4d86-0fe2-4619-9ea3-256f089e85e2369400320469187&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 601084 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 34dc4d86-0fe2-4619-9ea3-256f089e85e2369400352996368 2> 601085 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=34dc4d86-0fe2-4619-9ea3-256f089e85e2369400352996368&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 601106 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000009 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 601447 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 601449 INFO (indexFetcher-1293-thread-1) [] o.a.s.h.IndexFetcher Leader's generation: 1 2> 601449 INFO (indexFetcher-1293-thread-1) [] o.a.s.h.IndexFetcher Leader's version: 0 2> 601449 INFO (indexFetcher-1293-thread-1) [] o.a.s.h.IndexFetcher Follower's generation: 1 2> 601449 INFO (indexFetcher-1293-thread-1) [] o.a.s.h.IndexFetcher Follower's version: 0 2> 601711 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000009 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 601761 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=34dc4d86-0fe2-4619-9ea3-256f089e85e2&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=5 2> 601773 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=34dc4d86-0fe2-4619-9ea3-256f089e85e2&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=7 2> 601774 INFO (TEST-CloudSolrClientTest.singleShardedPreferenceRules-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForActiveCollection: singleShardPreferenceTestColl 2> 601806 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=singleShardPreferenceTestColl&wt=javabin&version=2} status=0 QTime=5 2> 601818 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=singleShardPreferenceTestColl&wt=javabin&version=2} status=0 QTime=6 2> 601829 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LISTALIASES&wt=javabin&version=2} status=0 QTime=6 2> 601840 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=singleShardPreferenceTestColl&wt=javabin&version=2} status=0 QTime=5 2> 601951 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/singleShardPreferenceTestColl/terms/shard1 to Terms{values={core_node6=1, core_node5=1}, version=2} for ensureHighestTermsAreNotZero 2> 601971 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.c.S.Request webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:40391/solr/singleShardPreferenceTestColl_shard1_replica_t2/&wt=javabin&version=2} status=0 QTime=39 2> 601975 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=singleShardPreferenceTestColl:9&wt=javabin&version=2} status=0 QTime=124 2> 602193 INFO (searcherExecutor-1276-thread-1-processing-127.0.0.1:40391_solr singleShardPreferenceTestColl_shard1_replica_t2 singleShardPreferenceTestColl shard1 core_node5) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 602233 INFO (searcherExecutor-1278-thread-1-processing-127.0.0.1:39159_solr singleShardPreferenceTestColl_shard1_replica_n3 singleShardPreferenceTestColl shard1 core_node6) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 602235 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.c.S.Request webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:40391/solr/singleShardPreferenceTestColl_shard1_replica_t2/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false} status=0 QTime=243 2> 602239 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=singleShardPreferenceTestColl:9&commit=true&wt=javabin&version=2} status=0 QTime=258 2> 602252 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=singleShardPreferenceTestColl&wt=javabin&version=2} status=0 QTime=6 2> 602265 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=singleShardPreferenceTestColl&wt=javabin&version=2} status=0 QTime=8 2> 602285 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=11 2> 602301 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=singleShardPreferenceTestColl&wt=javabin&version=2} status=0 QTime=9 2> 602318 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=singleShardPreferenceTestColl&wt=javabin&version=2} status=0 QTime=10 2> 602338 INFO (qtp218128001-2523) [n:127.0.0.1:44127_solr c:singleShardPreferenceTestColl s:shard1 r:core_node4 x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=singleShardPreferenceTestColl:9&shards.info=true&shards.preference=replica.type:PULL&wt=javabin&version=2} rid=127.0.0.1-1359 hits=0 status=0 QTime=11 2> 602363 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=singleShardPreferenceTestColl&wt=javabin&version=2} status=0 QTime=10 2> 602375 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=singleShardPreferenceTestColl&wt=javabin&version=2} status=0 QTime=6 2> 602393 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=12 2> 602417 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=singleShardPreferenceTestColl&wt=javabin&version=2} status=0 QTime=9 2> 602440 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=singleShardPreferenceTestColl&wt=javabin&version=2} status=0 QTime=9 2> 602455 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=2 2> 602457 INFO (indexFetcher-1293-thread-1) [] o.a.s.h.IndexFetcher Leader's generation: 2 2> 602458 INFO (indexFetcher-1293-thread-1) [] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 602458 INFO (indexFetcher-1293-thread-1) [] o.a.s.h.IndexFetcher Follower's generation: 1 2> 602458 INFO (indexFetcher-1293-thread-1) [] o.a.s.h.IndexFetcher Follower's version: 0 2> 602458 INFO (indexFetcher-1293-thread-1) [] o.a.s.h.IndexFetcher Starting replication process 2> 602460 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=singleShardPreferenceTestColl:9&shards.info=true&shards.preference=replica.type:TLOG&wt=javabin&version=2} rid=127.0.0.1-1360 hits=3 status=0 QTime=4 2> 602482 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&wt=javabin&version=2&command=filelist} status=0 QTime=21 2> 602484 INFO (indexFetcher-1293-thread-1) [] o.a.s.h.IndexFetcher Number of files in latest index in leader: 34 2> 602484 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=singleShardPreferenceTestColl:9&shards.info=true&shards.preference=replica.type:NRT&wt=javabin&version=2} rid=127.0.0.1-1361 hits=3 status=0 QTime=4 2> 602492 INFO (TEST-CloudSolrClientTest.singleShardedPreferenceRules-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Ending singleShardedPreferenceRules 2> 602508 INFO (indexFetcher-1293-thread-1) [] o.a.s.h.IndexFetcher Starting download (fullCopy=false) to MockDirectoryWrapper(ByteBuffersDirectory@362b9bda lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@249769c9) 2> 602510 INFO (indexFetcher-1293-thread-1) [] o.a.s.h.IndexFetcher tmpIndexDir_type : class org.apache.lucene.tests.store.MockDirectoryWrapper , ByteBuffersDirectory@362b9bda lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@249769c9 2> 602526 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 602540 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_1.dvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=1 2> 602553 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 602565 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=3 2> 602576 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_BlockTreeOrds_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 602593 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_BlockTreeOrds_0.tipo&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 602613 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=1 2> 602624 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.dvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 602674 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=13 2> 602710 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=18 2> 602747 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=9 2> 602810 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.tmd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=27 2> 602833 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.kdm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=3 2> 602919 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_BlockTreeOrds_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=21 2> 602958 INFO (TEST-CloudSolrClientTest.testParallelUpdateQTime-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Starting testParallelUpdateQTime 2> 602963 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 602988 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 603001 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CreateCollectionCmd Create collection testParallelUpdateQTime 2> 603005 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.kdi&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 603019 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=1 2> 603033 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 603047 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 603055 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdx&checksum=true&wt=filestream&command=filecontent} status=0 QTime=1 2> 603064 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_BlockTreeOrds_0.tio&checksum=true&wt=filestream&command=filecontent} status=0 QTime=1 2> 603076 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdt&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 603087 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.tmd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=3 2> 603100 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.dvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=1 2> 603109 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=1 2> 603116 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.tmd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=1 2> 603361 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.kdd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=1 2> 603370 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=1 2> 603381 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_1.dvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 603393 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 603404 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 603414 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fnm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=1 2> 603423 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=segments_2&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 603428 INFO (indexFetcher-1293-thread-1) [] o.a.s.h.IndexFetcher Bytes downloaded: 6042, Bytes skipped downloading: 0 2> 603428 INFO (indexFetcher-1293-thread-1) [] o.a.s.h.IndexFetcher Total time taken for download (fullCopy=false,bytesDownloaded=6042) : 0 secs (null bytes/sec) to MockDirectoryWrapper(ByteBuffersDirectory@362b9bda lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@249769c9) 2> 603449 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"testParallelUpdateQTime_shard2_replica_n1", 2> "node_name":"127.0.0.1:39159_solr", 2> "base_url":"http://127.0.0.1:39159/solr", 2> "collection":"testParallelUpdateQTime", 2> "shard":"shard2", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 603464 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"testParallelUpdateQTime_shard1_replica_n2", 2> "node_name":"127.0.0.1:40391_solr", 2> "base_url":"http://127.0.0.1:40391/solr", 2> "collection":"testParallelUpdateQTime", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 603506 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=testParallelUpdateQTime_shard2_replica_n1&action=CREATE&numShards=2&collection=testParallelUpdateQTime&shard=shard2&wt=javabin&version=2&replicaType=NRT 2> 603510 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=testParallelUpdateQTime_shard1_replica_n2&action=CREATE&numShards=2&collection=testParallelUpdateQTime&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 603544 INFO (indexFetcher-1293-thread-1) [] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used. 2> 603652 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 603661 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.s.IndexSchema Schema name=test 2> 603673 INFO (searcherExecutor-1280-thread-1-processing-127.0.0.1:44127_solr singleShardPreferenceTestColl_shard1_replica_p1 singleShardPreferenceTestColl shard1 core_node4) [n:127.0.0.1:44127_solr c:singleShardPreferenceTestColl s:shard1 r:core_node4 x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 603679 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 603688 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.s.IndexSchema Schema name=test 2> 603715 WARN (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 603716 WARN (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 603717 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 603718 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testParallelUpdateQTime_shard2_replica_n1' using configuration from configset conf, trusted=true 2> 603722 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/testParallelUpdateQTime_shard2_replica_n1], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/testParallelUpdateQTime_shard2_replica_n1/data/] 2> 603741 WARN (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 603743 WARN (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 603745 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 603748 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'testParallelUpdateQTime_shard1_replica_n2' using configuration from configset conf, trusted=true 2> 603752 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/testParallelUpdateQTime_shard1_replica_n2], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/testParallelUpdateQTime_shard1_replica_n2/data/] 2> 604044 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 604044 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 604062 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 604062 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 604068 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 604070 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 604085 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 604088 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 604095 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 604094 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 604096 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736768272564224 2> 604097 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 604126 INFO (searcherExecutor-1296-thread-1-processing-127.0.0.1:39159_solr testParallelUpdateQTime_shard2_replica_n1 testParallelUpdateQTime shard2 core_node3) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 604129 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 604134 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 604153 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 604159 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736768338624512 2> 604165 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testParallelUpdateQTime/terms/shard2 to Terms{values={core_node3=0}, version=0} for registerTerm 2> 604166 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/testParallelUpdateQTime/leaders/shard2 2> 604210 INFO (searcherExecutor-1298-thread-1-processing-127.0.0.1:40391_solr testParallelUpdateQTime_shard1_replica_n2 testParallelUpdateQTime shard1 core_node4) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 604219 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 604220 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 604220 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39159/solr/testParallelUpdateQTime_shard2_replica_n1/ 2> 604224 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 604229 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:39159/solr/testParallelUpdateQTime_shard2_replica_n1/ has no replicas 2> 604230 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/testParallelUpdateQTime/leaders/shard2/leader after winning as /collections/testParallelUpdateQTime/leader_elect/shard2/election/72081801657778182-core_node3-n_0000000000 2> 604256 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39159/solr/testParallelUpdateQTime_shard2_replica_n1/ shard2 2> 604267 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testParallelUpdateQTime/terms/shard1 to Terms{values={core_node4=0}, version=0} for registerTerm 2> 604270 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/testParallelUpdateQTime/leaders/shard1 2> 604297 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 604325 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 604325 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 604325 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40391/solr/testParallelUpdateQTime_shard1_replica_n2/ 2> 604329 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 604333 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:40391/solr/testParallelUpdateQTime_shard1_replica_n2/ has no replicas 2> 604333 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/testParallelUpdateQTime/leaders/shard1/leader after winning as /collections/testParallelUpdateQTime/leader_elect/shard1/election/72081801657778183-core_node4-n_0000000000 2> 604344 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=testParallelUpdateQTime_shard2_replica_n1&action=CREATE&numShards=2&collection=testParallelUpdateQTime&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=840 2> 604359 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40391/solr/testParallelUpdateQTime_shard1_replica_n2/ shard1 2> 604359 INFO (zkCallback-1160-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testParallelUpdateQTime/state.json] for collection [testParallelUpdateQTime] has occurred - updating... (live nodes size: [3]) 2> 604384 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 604396 INFO (zkCallback-1160-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testParallelUpdateQTime/state.json] for collection [testParallelUpdateQTime] has occurred - updating... (live nodes size: [3]) 2> 604415 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=testParallelUpdateQTime_shard1_replica_n2&action=CREATE&numShards=2&collection=testParallelUpdateQTime&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=907 2> 604427 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas 2> 604428 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf&name=testParallelUpdateQTime&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=1466 2> 604429 INFO (TEST-CloudSolrClientTest.testParallelUpdateQTime-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForActiveCollection: testParallelUpdateQTime 2> 604452 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testParallelUpdateQTime&wt=javabin&version=2} status=0 QTime=4 2> 604460 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testParallelUpdateQTime&wt=javabin&version=2} status=0 QTime=4 2> 604469 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testParallelUpdateQTime&wt=javabin&version=2} status=0 QTime=4 2> 604510 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testParallelUpdateQTime/terms/shard1 to Terms{values={core_node4=1}, version=1} for ensureHighestTermsAreNotZero 2> 604517 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/update params={wt=javabin&version=2} status=0 QTime=43 2> 604519 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testParallelUpdateQTime/terms/shard2 to Terms{values={core_node3=1}, version=1} for ensureHighestTermsAreNotZero 2> 604520 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={wt=javabin&version=2} status=0 QTime=41 2> 604526 INFO (TEST-CloudSolrClientTest.testParallelUpdateQTime-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Ending testParallelUpdateQTime 2> 604691 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=9 2> 604702 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 604702 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 604702 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 604702 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 604702 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 604742 INFO (TEST-CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Starting testRetryUpdatesWhenClusterStateIsStale 2> 604772 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CreateCollectionCmd Create collection stale_state_test_col 2> 604775 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000010 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 604931 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"stale_state_test_col_shard1_replica_n1", 2> "node_name":"127.0.0.1:39159_solr", 2> "base_url":"http://127.0.0.1:39159/solr", 2> "collection":"stale_state_test_col", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 604962 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr x:stale_state_test_col_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=stale_state_test_col_shard1_replica_n1&action=CREATE&numShards=1&collection=stale_state_test_col&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 605032 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 605037 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.s.IndexSchema Schema name=test 2> 605064 WARN (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 605065 WARN (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 605065 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 605067 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'stale_state_test_col_shard1_replica_n1' using configuration from configset conf, trusted=true 2> 605069 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/stale_state_test_col_shard1_replica_n1], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/stale_state_test_col_shard1_replica_n1/data/] 2> 605263 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 605263 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 605277 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 605277 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 605295 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 605298 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 605302 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 605303 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736769538195456 2> 605351 INFO (searcherExecutor-1308-thread-1-processing-127.0.0.1:39159_solr stale_state_test_col_shard1_replica_n1 stale_state_test_col shard1 core_node2) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 605355 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/stale_state_test_col/terms/shard1 to Terms{values={core_node2=0}, version=0} for registerTerm 2> 605356 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/stale_state_test_col/leaders/shard1 2> 605389 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 605389 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 605389 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39159/solr/stale_state_test_col_shard1_replica_n1/ 2> 605393 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 605395 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:39159/solr/stale_state_test_col_shard1_replica_n1/ has no replicas 2> 605395 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/stale_state_test_col/leaders/shard1/leader after winning as /collections/stale_state_test_col/leader_elect/shard1/election/72081801657778182-core_node2-n_0000000000 2> 605412 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39159/solr/stale_state_test_col_shard1_replica_n1/ shard1 2> 605432 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 605457 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=stale_state_test_col_shard1_replica_n1&action=CREATE&numShards=1&collection=stale_state_test_col&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=496 2> 605471 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas 2> 605471 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf&name=stale_state_test_col&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:39159_solr&wt=javabin&version=2} status=0 QTime=726 2> 605473 INFO (TEST-CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForActiveCollection: stale_state_test_col 2> 605498 WARN (TEST-CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 605507 INFO (TEST-CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 15000ms for client to connect to ZooKeeper 2> 605523 INFO (zkConnectionManagerCallback-1317-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 605526 INFO (TEST-CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 605529 WARN (TEST-CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 605540 INFO (TEST-CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3) 2> 605550 INFO (TEST-CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40255/solr ready 2> 605560 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=stale_state_test_col:4&wt=javabin&version=2} rid=127.0.0.1-1362 hits=0 status=0 QTime=2 2> 605593 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000012 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 605608 INFO (OverseerThreadFactory-1176-thread-5) [ c:stale_state_test_col s:shard1] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:44127_solr for creating new replica of shard shard1 for collection stale_state_test_col 2> 605619 INFO (OverseerThreadFactory-1176-thread-5) [ c:stale_state_test_col s:shard1] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command. 2> 605622 INFO (OverseerThreadFactory-1176-thread-5) [ c:stale_state_test_col s:shard1] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"stale_state_test_col_shard1_replica_n3", 2> "node_name":"127.0.0.1:44127_solr", 2> "base_url":"http://127.0.0.1:44127/solr", 2> "collection":"stale_state_test_col", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"addreplica"} 2> 605637 INFO (zkCallback-1160-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 605657 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr x:stale_state_test_col_shard1_replica_n3] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&name=stale_state_test_col_shard1_replica_n3&action=CREATE&collection=stale_state_test_col&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 605684 INFO (zkCallback-1160-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 605706 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 605709 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 605709 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 605709 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 605709 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 605709 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 605731 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 605739 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.s.IndexSchema Schema name=test 2> 605767 WARN (qtp218128001-2524) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 605767 WARN (qtp218128001-2524) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 605768 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 605769 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'stale_state_test_col_shard1_replica_n3' using configuration from configset conf, trusted=true 2> 605772 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node3/stale_state_test_col_shard1_replica_n3], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node3/stale_state_test_col_shard1_replica_n3/data/] 2> 606008 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 606008 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 606025 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 606025 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 606046 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 606049 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 606054 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 606055 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736770326724608 2> 606113 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.ZkShardTerms Successful update of terms at /collections/stale_state_test_col/terms/shard1 to Terms{values={core_node2=0, core_node4=0}, version=1} for registerTerm 2> 606113 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/stale_state_test_col/leaders/shard1 2> 606117 INFO (searcherExecutor-1319-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 606159 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.ZkController Core needs to recover:stale_state_test_col_shard1_replica_n3 2> 606161 INFO (updateExecutor-1154-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.u.DefaultSolrCoreState Running recovery 2> 606166 INFO (recoveryExecutor-1163-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true 2> 606167 INFO (recoveryExecutor-1163-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.RecoveryStrategy startupVersions is empty 2> 606183 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&name=stale_state_test_col_shard1_replica_n3&action=CREATE&collection=stale_state_test_col&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=527 2> 606192 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/admin/ping params={wt=javabin&version=2} hits=0 status=0 QTime=4 2> 606192 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/admin/ping params={wt=javabin&version=2} status=0 QTime=4 2> 606194 INFO (recoveryExecutor-1163-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[stale_state_test_col_shard1_replica_n3] 2> 606198 INFO (recoveryExecutor-1163-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null} 2> 606198 INFO (recoveryExecutor-1163-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.RecoveryStrategy Publishing state of core [stale_state_test_col_shard1_replica_n3] as recovering, leader is [http://127.0.0.1:39159/solr/stale_state_test_col_shard1_replica_n1/] and I am [http://127.0.0.1:44127/solr/stale_state_test_col_shard1_replica_n3/] 2> 606213 INFO (zkCallback-1166-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 606214 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 606214 INFO (zkCallback-1160-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 606217 INFO (zkCallback-1166-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 606222 INFO (recoveryExecutor-1163-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:39159/solr]; [WaitForState: action=PREPRECOVERY&core=stale_state_test_col_shard1_replica_n1&nodeName=127.0.0.1:44127_solr&coreNodeName=core_node4&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true] 2> 606232 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr x:stale_state_test_col_shard1_replica_n1] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true 2> 606232 INFO (zkCallback-1166-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 606238 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=stale_state_test_col, shard=shard1, thisCore=stale_state_test_col_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:44127_solr, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{ 2> "core":"stale_state_test_col_shard1_replica_n3", 2> "node_name":"127.0.0.1:44127_solr", 2> "base_url":"http://127.0.0.1:44127/solr", 2> "state":"recovering", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 606245 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=stale_state_test_col, shard=shard1, thisCore=stale_state_test_col_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:44127_solr, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{ 2> "core":"stale_state_test_col_shard1_replica_n3", 2> "node_name":"127.0.0.1:44127_solr", 2> "base_url":"http://127.0.0.1:44127/solr", 2> "state":"recovering", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 606247 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=stale_state_test_col, shard=shard1, thisCore=stale_state_test_col_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:44127_solr, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{ 2> "core":"stale_state_test_col_shard1_replica_n3", 2> "node_name":"127.0.0.1:44127_solr", 2> "base_url":"http://127.0.0.1:44127/solr", 2> "state":"recovering", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 606247 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:44127_solr&onlyIfLeaderActive=true&core=stale_state_test_col_shard1_replica_n1&coreNodeName=core_node4&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=16 2> 606250 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:stale_state_test_col] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:44127_solr&action=ADDREPLICA&collection=stale_state_test_col&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=684 2> 606251 INFO (TEST-CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale-seed#[9CAA27B60427D55F]) [] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection:stale_state_test_col failOnTimeout:true timeout:330SECONDS 1> replica:core_node2 rstate:active live:true 1> replica:core_node4 rstate:recovering live:true 1> replica:core_node2 rstate:active live:true 1> replica:core_node4 rstate:recovering live:true 1> replica:core_node2 rstate:active live:true 1> replica:core_node4 rstate:recovering live:true 1> replica:core_node2 rstate:active live:true 1> replica:core_node4 rstate:recovering live:true 1> replica:core_node2 rstate:active live:true 1> replica:core_node4 rstate:recovering live:true 2> 606714 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 606717 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 606717 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 606717 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 606717 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 606717 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 606751 INFO (recoveryExecutor-1163-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:39159/solr/stale_state_test_col_shard1_replica_n1/] - recoveringAfterStartup=[true] 2> 606754 WARN (recoveryExecutor-1163-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.u.PeerSyncWithLeader no frame of reference to tell if we've missed updates 2> 606755 INFO (recoveryExecutor-1163-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.RecoveryStrategy PeerSync Recovery was not successful - trying replication. 2> 606755 INFO (recoveryExecutor-1163-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.RecoveryStrategy Starting Replication Recovery. 2> 606755 INFO (recoveryExecutor-1163-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.RecoveryStrategy Attempting to replicate from [http://127.0.0.1:39159/solr/stale_state_test_col_shard1_replica_n1/]. 2> 606801 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.u.p.DistributedUpdateProcessor Ignoring commit while not ACTIVE - state: BUFFERING replay: false 2> 606801 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.S.Request webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:39159/solr/stale_state_test_col_shard1_replica_n1/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false} status=0 QTime=12 2> 606805 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=47 2> 606816 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:stale_state_test_col s:shard1 r:core_node2 x:stale_state_test_col_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 606817 INFO (recoveryExecutor-1163-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.h.IndexFetcher Leader's generation: 1 2> 606817 INFO (recoveryExecutor-1163-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.h.IndexFetcher Leader's version: 0 2> 606818 INFO (recoveryExecutor-1163-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.h.IndexFetcher Follower's generation: 1 2> 606818 INFO (recoveryExecutor-1163-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.h.IndexFetcher Follower's version: 0 2> 606818 INFO (recoveryExecutor-1163-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.RecoveryStrategy No replay needed. 2> 606824 INFO (recoveryExecutor-1163-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.RecoveryStrategy Replication Recovery was successful. 2> 606825 INFO (recoveryExecutor-1163-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.RecoveryStrategy Registering as Active after recovery. 2> 606838 INFO (zkCallback-1186-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 606839 INFO (zkCallback-1160-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 606839 INFO (zkCallback-1166-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 606839 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 606839 INFO (zkCallback-1166-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 606839 INFO (zkCallback-1166-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 606845 INFO (recoveryExecutor-1163-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.RecoveryStrategy Updating version bucket highest from index after successful recovery. 2> 606846 INFO (recoveryExecutor-1163-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736771155099648 1> replica:core_node2 rstate:active live:true 1> replica:core_node4 rstate:active live:true 1> no one is recoverying 2> 606848 INFO (TEST-CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale-seed#[9CAA27B60427D55F]) [] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection:stale_state_test_col 2> 606859 INFO (recoveryExecutor-1163-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.RecoveryStrategy Finished recovery process, successful=[true] msTimeTaken=691.0 2> 606860 INFO (recoveryExecutor-1163-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.RecoveryStrategy Finished recovery process. recoveringAfterStartup=true msTimeTaken=692.0 2> 606916 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000014 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 606939 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr x:stale_state_test_col_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.stale_state_test_col.shard1.replica_n1 tag=null 2> 607243 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr x:stale_state_test_col_shard1_replica_n1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@13ec3ab8 stale_state_test_col_shard1_replica_n1 2> 607244 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr x:stale_state_test_col_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.stale_state_test_col.shard1.replica_n1 tag=SolrCore@13ec3ab8 2> 607244 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr x:stale_state_test_col_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.stale_state_test_col.shard1.leader tag=SolrCore@13ec3ab8 2> 607246 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr x:stale_state_test_col_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 607263 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.c.ZkShardTerms Successful update of terms at /collections/stale_state_test_col/terms/shard1 to Terms{values={core_node4=0}, version=2} for removeTerm 2> 607303 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 607303 INFO (zkCallback-1166-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 607304 INFO (zkCallback-1166-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 607304 INFO (zkCallback-1160-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 607306 INFO (zkCallback-1166-thread-5) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 607309 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=stale_state_test_col_shard1_replica_n1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2&deleteIndex=true} status=0 QTime=372 2> 607327 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 607327 INFO (zkCallback-1166-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 607327 INFO (zkCallback-1166-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 607327 INFO (zkCallback-1166-thread-5) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 607341 INFO (zkCallback-1166-thread-1) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 607355 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:stale_state_test_col] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replica=core_node2&action=DELETEREPLICA&collection=stale_state_test_col&shard=shard1&wt=javabin&version=2} status=0 QTime=495 2> 607357 INFO (TEST-CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale-seed#[9CAA27B60427D55F]) [] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection:stale_state_test_col failOnTimeout:true timeout:330SECONDS 1> replica:core_node4 rstate:active live:true 1> no one is recoverying 1> replica:core_node4 rstate:active live:true 1> no one is recoverying 1> replica:core_node4 rstate:active live:true 1> no one is recoverying 1> replica:core_node4 rstate:active live:true 1> no one is recoverying 2> 607391 INFO (TEST-CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale-seed#[9CAA27B60427D55F]) [] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection:stale_state_test_col 2> 607405 INFO (TEST-CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClient request was not communication error it seems 2> 607405 INFO (TEST-CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClient Request to collection [stale_state_test_col] failed due to (510) org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39159/solr/stale_state_test_col: {"stale_state_test_col":10}, retry=0 maxRetries=5 commError=false errorCode=510 2> 607406 WARN (TEST-CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClient Re-trying request to collection(s) [stale_state_test_col] after stale state error from server. 2> 607721 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 607723 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 607723 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 607723 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 607724 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 607724 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 608728 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 608729 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 608730 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 608730 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 608730 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 608730 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 608940 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000016 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 609734 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 609735 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 609736 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 609736 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 609736 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 609736 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 609842 INFO (zkCallback-1166-thread-1) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:44127/solr/stale_state_test_col_shard1_replica_n3/ 2> 609842 INFO (zkCallback-1166-thread-1) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 609843 INFO (zkCallback-1166-thread-1) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.SyncStrategy http://127.0.0.1:44127/solr/stale_state_test_col_shard1_replica_n3/ has no replicas 2> 609843 INFO (zkCallback-1166-thread-1) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/stale_state_test_col/leaders/shard1/leader after winning as /collections/stale_state_test_col/leader_elect/shard1/election/72081801657778184-core_node4-n_0000000001 2> 609857 INFO (zkCallback-1166-thread-5) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 609857 INFO (zkCallback-1166-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 609858 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 609863 INFO (zkCallback-1166-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [3]) 2> 609864 INFO (zkCallback-1166-thread-1) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:44127/solr/stale_state_test_col_shard1_replica_n3/ shard1 2> 609955 INFO (searcherExecutor-1319-thread-1-processing-127.0.0.1:44127_solr stale_state_test_col_shard1_replica_n3 stale_state_test_col shard1 core_node4) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 609961 INFO (qtp218128001-2523) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.ZkShardTerms Successful update of terms at /collections/stale_state_test_col/terms/shard1 to Terms{values={core_node4=1}, version=3} for ensureHighestTermsAreNotZero 2> 609962 INFO (qtp218128001-2523) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=stale_state_test_col:10&commit=true&wt=javabin&version=2} status=0 QTime=2544 2> 609968 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=stale_state_test_col:10&wt=javabin&version=2} rid=127.0.0.1-1364 hits=1 status=0 QTime=2 2> 610088 INFO (TEST-CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Ending testRetryUpdatesWhenClusterStateIsStale 2> 610308 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Starting testNonRetryableRequests 2> 610314 WARN (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 610321 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 15000ms for client to connect to ZooKeeper 2> 610331 INFO (zkConnectionManagerCallback-1329-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 610331 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 610332 WARN (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 610339 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3) 2> 610347 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40255/solr ready 2> 610371 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=73c5f73b-3955-4124-9cbd-e69e2f6607cb&collection.configName=conf&name=testNonRetryableRequests&nrtReplicas=3&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=18 2> 610390 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=73c5f73b-3955-4124-9cbd-e69e2f6607cb&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=14 2> 610393 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CreateCollectionCmd Create collection testNonRetryableRequests 2> 610585 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"testNonRetryableRequests_shard1_replica_n1", 2> "node_name":"127.0.0.1:44127_solr", 2> "base_url":"http://127.0.0.1:44127/solr", 2> "collection":"testNonRetryableRequests", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 610597 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"testNonRetryableRequests_shard1_replica_n2", 2> "node_name":"127.0.0.1:39159_solr", 2> "base_url":"http://127.0.0.1:39159/solr", 2> "collection":"testNonRetryableRequests", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 610608 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"testNonRetryableRequests_shard1_replica_n3", 2> "node_name":"127.0.0.1:40391_solr", 2> "base_url":"http://127.0.0.1:40391/solr", 2> "collection":"testNonRetryableRequests", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 610640 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr x:testNonRetryableRequests_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=testNonRetryableRequests&version=2&replicaType=NRT&async=73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797&coreNodeName=core_node4&name=testNonRetryableRequests_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&wt=javabin} status=0 QTime=2 2> 610640 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-2-processing-127.0.0.1:44127_solr testNonRetryableRequests_shard1_replica_n1 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797 create) [n:127.0.0.1:44127_solr x:testNonRetryableRequests_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=testNonRetryableRequests&version=2&replicaType=NRT&async=73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797&coreNodeName=core_node4&name=testNonRetryableRequests_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&wt=javabin 2> 610648 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr x:testNonRetryableRequests_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=testNonRetryableRequests&version=2&replicaType=NRT&async=73c5f73b-3955-4124-9cbd-e69e2f6607cb369412973540072&coreNodeName=core_node5&name=testNonRetryableRequests_shard1_replica_n2&action=CREATE&numShards=1&shard=shard1&wt=javabin} status=0 QTime=4 2> 610650 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-5-processing-127.0.0.1:39159_solr testNonRetryableRequests_shard1_replica_n2 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412973540072 create) [n:127.0.0.1:39159_solr x:testNonRetryableRequests_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=testNonRetryableRequests&version=2&replicaType=NRT&async=73c5f73b-3955-4124-9cbd-e69e2f6607cb369412973540072&coreNodeName=core_node5&name=testNonRetryableRequests_shard1_replica_n2&action=CREATE&numShards=1&shard=shard1&wt=javabin 2> 610656 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr x:testNonRetryableRequests_shard1_replica_n3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=testNonRetryableRequests&version=2&replicaType=NRT&async=73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329&coreNodeName=core_node6&name=testNonRetryableRequests_shard1_replica_n3&action=CREATE&numShards=1&shard=shard1&wt=javabin} status=0 QTime=10 2> 610658 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr x:testNonRetryableRequests_shard1_replica_n3] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=testNonRetryableRequests&version=2&replicaType=NRT&async=73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329&coreNodeName=core_node6&name=testNonRetryableRequests_shard1_replica_n3&action=CREATE&numShards=1&shard=shard1&wt=javabin 2> 610666 INFO (qtp218128001-2528) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797 2> 610666 INFO (qtp218128001-2528) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 610741 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 610743 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 610743 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 610743 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 610744 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 610744 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 610767 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-2-processing-127.0.0.1:44127_solr testNonRetryableRequests_shard1_replica_n1 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797 create) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 610776 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-5-processing-127.0.0.1:39159_solr testNonRetryableRequests_shard1_replica_n2 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412973540072 create) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 610777 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-2-processing-127.0.0.1:44127_solr testNonRetryableRequests_shard1_replica_n1 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797 create) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.s.IndexSchema Schema name=test 2> 610786 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-5-processing-127.0.0.1:39159_solr testNonRetryableRequests_shard1_replica_n2 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412973540072 create) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.s.IndexSchema Schema name=test 2> 610792 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 610799 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.s.IndexSchema Schema name=test 2> 610825 WARN (parallelCoreAdminAPIBaseExecutor-1178-thread-2-processing-127.0.0.1:44127_solr testNonRetryableRequests_shard1_replica_n1 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797 create) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 610826 WARN (parallelCoreAdminAPIBaseExecutor-1178-thread-2-processing-127.0.0.1:44127_solr testNonRetryableRequests_shard1_replica_n1 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797 create) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 610826 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-2-processing-127.0.0.1:44127_solr testNonRetryableRequests_shard1_replica_n1 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797 create) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 610829 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-2-processing-127.0.0.1:44127_solr testNonRetryableRequests_shard1_replica_n1 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797 create) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testNonRetryableRequests_shard1_replica_n1' using configuration from configset conf, trusted=true 2> 610833 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-2-processing-127.0.0.1:44127_solr testNonRetryableRequests_shard1_replica_n1 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797 create) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node3/testNonRetryableRequests_shard1_replica_n1], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node3/testNonRetryableRequests_shard1_replica_n1/data/] 2> 610834 WARN (parallelCoreAdminAPIBaseExecutor-1177-thread-5-processing-127.0.0.1:39159_solr testNonRetryableRequests_shard1_replica_n2 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412973540072 create) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 610835 WARN (parallelCoreAdminAPIBaseExecutor-1177-thread-5-processing-127.0.0.1:39159_solr testNonRetryableRequests_shard1_replica_n2 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412973540072 create) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 610837 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-5-processing-127.0.0.1:39159_solr testNonRetryableRequests_shard1_replica_n2 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412973540072 create) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 610839 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-5-processing-127.0.0.1:39159_solr testNonRetryableRequests_shard1_replica_n2 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412973540072 create) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'testNonRetryableRequests_shard1_replica_n2' using configuration from configset conf, trusted=true 2> 610840 WARN (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 610840 WARN (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 610841 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 610843 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'testNonRetryableRequests_shard1_replica_n3' using configuration from configset conf, trusted=true 2> 610846 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/testNonRetryableRequests_shard1_replica_n3], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/testNonRetryableRequests_shard1_replica_n3/data/] 2> 610846 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-5-processing-127.0.0.1:39159_solr testNonRetryableRequests_shard1_replica_n2 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412973540072 create) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/testNonRetryableRequests_shard1_replica_n2], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/testNonRetryableRequests_shard1_replica_n2/data/] 2> 611160 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-2-processing-127.0.0.1:44127_solr testNonRetryableRequests_shard1_replica_n1 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797 create) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 611161 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-2-processing-127.0.0.1:44127_solr testNonRetryableRequests_shard1_replica_n1 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797 create) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 611166 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 611166 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 611177 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-5-processing-127.0.0.1:39159_solr testNonRetryableRequests_shard1_replica_n2 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412973540072 create) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 611178 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-5-processing-127.0.0.1:39159_solr testNonRetryableRequests_shard1_replica_n2 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412973540072 create) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 611181 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-2-processing-127.0.0.1:44127_solr testNonRetryableRequests_shard1_replica_n1 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797 create) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 611181 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-2-processing-127.0.0.1:44127_solr testNonRetryableRequests_shard1_replica_n1 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797 create) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 611187 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 611188 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 611199 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-5-processing-127.0.0.1:39159_solr testNonRetryableRequests_shard1_replica_n2 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412973540072 create) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 611200 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-5-processing-127.0.0.1:39159_solr testNonRetryableRequests_shard1_replica_n2 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412973540072 create) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 611209 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-2-processing-127.0.0.1:44127_solr testNonRetryableRequests_shard1_replica_n1 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797 create) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 611214 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 611217 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-2-processing-127.0.0.1:44127_solr testNonRetryableRequests_shard1_replica_n1 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797 create) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 611219 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 611224 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-2-processing-127.0.0.1:44127_solr testNonRetryableRequests_shard1_replica_n1 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797 create) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 611224 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 611226 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736775748911104 2> 611228 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-2-processing-127.0.0.1:44127_solr testNonRetryableRequests_shard1_replica_n1 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797 create) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736775751008256 2> 611232 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-5-processing-127.0.0.1:39159_solr testNonRetryableRequests_shard1_replica_n2 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412973540072 create) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 611235 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-5-processing-127.0.0.1:39159_solr testNonRetryableRequests_shard1_replica_n2 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412973540072 create) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 611242 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-5-processing-127.0.0.1:39159_solr testNonRetryableRequests_shard1_replica_n2 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412973540072 create) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 611244 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-5-processing-127.0.0.1:39159_solr testNonRetryableRequests_shard1_replica_n2 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412973540072 create) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736775767785472 2> 611263 INFO (searcherExecutor-1331-thread-1-processing-testNonRetryableRequests_shard1_replica_n1 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797 core_node4 create 127.0.0.1:44127_solr testNonRetryableRequests shard1) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 611282 INFO (searcherExecutor-1335-thread-1-processing-testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 core_node6 create 127.0.0.1:40391_solr testNonRetryableRequests shard1) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 611306 INFO (searcherExecutor-1333-thread-1-processing-testNonRetryableRequests_shard1_replica_n2 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412973540072 core_node5 create 127.0.0.1:39159_solr testNonRetryableRequests shard1) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 611326 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testNonRetryableRequests/terms/shard1 to Terms{values={core_node6=0}, version=0} for registerTerm 2> 611326 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/testNonRetryableRequests/leaders/shard1 2> 611346 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-2-processing-127.0.0.1:44127_solr testNonRetryableRequests_shard1_replica_n1 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797 create) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testNonRetryableRequests/terms/shard1 to Terms{values={core_node6=0, core_node4=0}, version=1} for registerTerm 2> 611355 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-2-processing-127.0.0.1:44127_solr testNonRetryableRequests_shard1_replica_n1 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797 create) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/testNonRetryableRequests/leaders/shard1 2> 611380 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-5-processing-127.0.0.1:39159_solr testNonRetryableRequests_shard1_replica_n2 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412973540072 create) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testNonRetryableRequests/terms/shard1 to Terms{values={core_node6=0, core_node4=0, core_node5=0}, version=2} for registerTerm 2> 611381 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-5-processing-127.0.0.1:39159_solr testNonRetryableRequests_shard1_replica_n2 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412973540072 create) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/testNonRetryableRequests/leaders/shard1 2> 611419 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=73c5f73b-3955-4124-9cbd-e69e2f6607cb&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=24 2> 611423 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 611423 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 611424 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40391/solr/testNonRetryableRequests_shard1_replica_n3/ 2> 611430 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.u.PeerSync PeerSync: core=testNonRetryableRequests_shard1_replica_n3 url=http://127.0.0.1:40391/solr START replicas=[http://127.0.0.1:44127/solr/testNonRetryableRequests_shard1_replica_n1/, http://127.0.0.1:39159/solr/testNonRetryableRequests_shard1_replica_n2/] nUpdates=100 2> 611436 INFO (qtp218128001-2523) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=2 2> 611436 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.u.PeerSync PeerSync: core=testNonRetryableRequests_shard1_replica_n3 url=http://127.0.0.1:40391/solr DONE. We have no versions. sync failed. 2> 611440 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=2 2> 611442 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate 2> 611442 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway 2> 611443 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/testNonRetryableRequests/leaders/shard1/leader after winning as /collections/testNonRetryableRequests/leader_elect/shard1/election/72081801657778183-core_node6-n_0000000000 2> 611458 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40391/solr/testNonRetryableRequests_shard1_replica_n3/ shard1 2> 611479 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-4-processing-127.0.0.1:40391_solr testNonRetryableRequests_shard1_replica_n3 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 create) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 611672 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797 2> 611673 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 611747 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 611749 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 611749 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 611749 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 611749 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 611750 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 612440 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=73c5f73b-3955-4124-9cbd-e69e2f6607cb&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=17 2> 612468 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNonRetryableRequests/state.json] for collection [testNonRetryableRequests] has occurred - updating... (live nodes size: [3]) 2> 612480 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNonRetryableRequests/state.json] for collection [testNonRetryableRequests] has occurred - updating... (live nodes size: [3]) 2> 612678 INFO (qtp218128001-2525) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797 2> 612679 INFO (qtp218128001-2525) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=73c5f73b-3955-4124-9cbd-e69e2f6607cb369412962997797&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 612684 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412973540072 2> 612685 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=73c5f73b-3955-4124-9cbd-e69e2f6607cb369412973540072&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 612691 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329 2> 612691 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=73c5f73b-3955-4124-9cbd-e69e2f6607cb369412996156329&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 612710 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000018 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 612753 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 612754 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 612755 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 612755 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 612755 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 612755 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 613451 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=73c5f73b-3955-4124-9cbd-e69e2f6607cb&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=6 2> 614003 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=73c5f73b-3955-4124-9cbd-e69e2f6607cb&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=547 2> 614005 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForActiveCollection: testNonRetryableRequests 2> 614005 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 614007 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 614007 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 614008 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 614008 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 614008 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 614051 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/admin/mbeans params={qt=/admin/mbeans&stats=true&cat=ADMIN&wt=javabin&version=2&key=org.apache.solr.handler.admin.ConfigSetsHandler} status=0 QTime=3 2> 614059 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Found 0 requests to /admin/configs on http://127.0.0.1:39159/solr 2> 614077 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/admin/mbeans params={qt=/admin/mbeans&stats=true&cat=ADMIN&wt=javabin&version=2&key=org.apache.solr.handler.admin.ConfigSetsHandler} status=0 QTime=4 2> 614080 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Found 0 requests to /admin/configs on http://127.0.0.1:44127/solr 2> 614098 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.c.S.Request webapp=/solr path=/admin/mbeans params={qt=/admin/mbeans&stats=true&cat=ADMIN&wt=javabin&version=2&key=org.apache.solr.handler.admin.ConfigSetsHandler} status=0 QTime=3 2> 614103 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Found 0 requests to /admin/configs on http://127.0.0.1:40391/solr 2> 614109 ERROR (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Unknown action: foobar 2> => org.apache.solr.common.SolrException: Unknown action: foobar 2> at org.apache.solr.handler.admin.ConfigSetsHandler.handleRequestBody(ConfigSetsHandler.java:94) 2> org.apache.solr.common.SolrException: Unknown action: foobar 2> at org.apache.solr.handler.admin.ConfigSetsHandler.handleRequestBody(ConfigSetsHandler.java:94) ~[solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:224) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:929) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:877) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:548) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.SolrDispatchFilter.dispatch(SolrDispatchFilter.java:252) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.SolrDispatchFilter.lambda$doFilter$0(SolrDispatchFilter.java:220) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.ServletUtils.traceHttpRequestExecution2(ServletUtils.java:257) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.ServletUtils.rateLimitRequest(ServletUtils.java:227) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:215) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.apache.solr.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:187) [solr-test-framework-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:527) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1570) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1383) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1543) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1305) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:301) [jetty-rewrite-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:822) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.Server.handle(Server.java:563) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.HttpChannel.lambda$handle$0(HttpChannel.java:505) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:762) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:497) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:282) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314) [jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100) [jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) [jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:934) [jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1078) [jetty-util-10.0.13.jar:10.0.13] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> 614128 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/configs params={qt=/admin/configs&action=foobar&wt=javabin&version=2} status=400 QTime=20 2> 614156 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/admin/mbeans params={qt=/admin/mbeans&stats=true&cat=ADMIN&wt=javabin&version=2&key=org.apache.solr.handler.admin.ConfigSetsHandler} status=0 QTime=2 2> 614160 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Found 1 requests to /admin/configs on http://127.0.0.1:39159/solr 2> 614176 INFO (qtp218128001-2525) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/admin/mbeans params={qt=/admin/mbeans&stats=true&cat=ADMIN&wt=javabin&version=2&key=org.apache.solr.handler.admin.ConfigSetsHandler} status=0 QTime=3 2> 614180 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Found 0 requests to /admin/configs on http://127.0.0.1:44127/solr 2> 614196 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.c.S.Request webapp=/solr path=/admin/mbeans params={qt=/admin/mbeans&stats=true&cat=ADMIN&wt=javabin&version=2&key=org.apache.solr.handler.admin.ConfigSetsHandler} status=0 QTime=3 2> 614200 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Found 0 requests to /admin/configs on http://127.0.0.1:40391/solr 2> 614216 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/admin/mbeans params={qt=/admin/mbeans&stats=true&cat=ADMIN&wt=javabin&version=2&key=org.apache.solr.handler.admin.CollectionsHandler} status=0 QTime=3 2> 614220 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Found 0 requests to /admin/collections on http://127.0.0.1:39159/solr 2> 614234 INFO (qtp218128001-2523) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/admin/mbeans params={qt=/admin/mbeans&stats=true&cat=ADMIN&wt=javabin&version=2&key=org.apache.solr.handler.admin.CollectionsHandler} status=0 QTime=2 2> 614237 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Found 0 requests to /admin/collections on http://127.0.0.1:44127/solr 2> 614252 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.c.S.Request webapp=/solr path=/admin/mbeans params={qt=/admin/mbeans&stats=true&cat=ADMIN&wt=javabin&version=2&key=org.apache.solr.handler.admin.CollectionsHandler} status=0 QTime=2 2> 614257 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Found 0 requests to /admin/collections on http://127.0.0.1:40391/solr 2> 614264 ERROR (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Unknown action: foobar 2> => org.apache.solr.common.SolrException: Unknown action: foobar 2> at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:320) 2> org.apache.solr.common.SolrException: Unknown action: foobar 2> at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:320) ~[solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:224) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:929) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:877) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:548) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.SolrDispatchFilter.dispatch(SolrDispatchFilter.java:252) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.SolrDispatchFilter.lambda$doFilter$0(SolrDispatchFilter.java:220) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.ServletUtils.traceHttpRequestExecution2(ServletUtils.java:257) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.ServletUtils.rateLimitRequest(ServletUtils.java:227) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:215) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.apache.solr.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:187) [solr-test-framework-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:527) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1570) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1383) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1543) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1305) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:301) [jetty-rewrite-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:822) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.Server.handle(Server.java:563) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.HttpChannel.lambda$handle$0(HttpChannel.java:505) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:762) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:497) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:282) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314) [jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100) [jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) [jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:934) [jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1078) [jetty-util-10.0.13.jar:10.0.13] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> 614284 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={qt=/admin/collections&action=foobar&wt=javabin&version=2} status=400 QTime=21 2> 614324 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/admin/mbeans params={qt=/admin/mbeans&stats=true&cat=ADMIN&wt=javabin&version=2&key=org.apache.solr.handler.admin.CollectionsHandler} status=0 QTime=3 2> 614330 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Found 1 requests to /admin/collections on http://127.0.0.1:39159/solr 2> 614346 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/admin/mbeans params={qt=/admin/mbeans&stats=true&cat=ADMIN&wt=javabin&version=2&key=org.apache.solr.handler.admin.CollectionsHandler} status=0 QTime=3 2> 614351 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Found 0 requests to /admin/collections on http://127.0.0.1:44127/solr 2> 614369 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.c.S.Request webapp=/solr path=/admin/mbeans params={qt=/admin/mbeans&stats=true&cat=ADMIN&wt=javabin&version=2&key=org.apache.solr.handler.admin.CollectionsHandler} status=0 QTime=3 2> 614376 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Found 0 requests to /admin/collections on http://127.0.0.1:40391/solr 2> 614391 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/admin/mbeans params={qt=/admin/mbeans&stats=true&cat=ADMIN&wt=javabin&version=2&key=org.apache.solr.handler.admin.CoreAdminHandler} status=0 QTime=2 2> 614395 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Found 0 requests to /admin/cores on http://127.0.0.1:39159/solr 2> 614406 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000018 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 614417 INFO (qtp218128001-2525) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/admin/mbeans params={qt=/admin/mbeans&stats=true&cat=ADMIN&wt=javabin&version=2&key=org.apache.solr.handler.admin.CoreAdminHandler} status=0 QTime=2 2> 614421 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Found 0 requests to /admin/cores on http://127.0.0.1:44127/solr 2> 614442 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.c.S.Request webapp=/solr path=/admin/mbeans params={qt=/admin/mbeans&stats=true&cat=ADMIN&wt=javabin&version=2&key=org.apache.solr.handler.admin.CoreAdminHandler} status=0 QTime=2 2> 614446 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Found 0 requests to /admin/cores on http://127.0.0.1:40391/solr 2> 614449 WARN (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminHandler action 'foobar' not found, calling custom action handler. If original intention was to target some custom behaviour use custom actions defined in 'solr.xml' instead 2> 614450 ERROR (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Unsupported operation: foobar 2> => org.apache.solr.common.SolrException: Unsupported operation: foobar 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleCustomAction(CoreAdminHandler.java:247) 2> org.apache.solr.common.SolrException: Unsupported operation: foobar 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleCustomAction(CoreAdminHandler.java:247) ~[solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:209) ~[solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:224) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:929) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:877) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:548) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.SolrDispatchFilter.dispatch(SolrDispatchFilter.java:252) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.SolrDispatchFilter.lambda$doFilter$0(SolrDispatchFilter.java:220) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.ServletUtils.traceHttpRequestExecution2(ServletUtils.java:257) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.ServletUtils.rateLimitRequest(ServletUtils.java:227) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:215) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.apache.solr.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:187) [solr-test-framework-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:527) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1570) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1383) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484) [jetty-servlet-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1543) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1305) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:301) [jetty-rewrite-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:822) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.Server.handle(Server.java:563) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.HttpChannel.lambda$handle$0(HttpChannel.java:505) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:762) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:497) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:282) [jetty-server-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314) [jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100) [jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) [jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:934) [jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1078) [jetty-util-10.0.13.jar:10.0.13] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> 614463 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&action=foobar&wt=javabin&version=2} status=400 QTime=14 2> 614479 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/admin/mbeans params={qt=/admin/mbeans&stats=true&cat=ADMIN&wt=javabin&version=2&key=org.apache.solr.handler.admin.CoreAdminHandler} status=0 QTime=1 2> 614481 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Found 1 requests to /admin/cores on http://127.0.0.1:39159/solr 2> 614498 INFO (qtp218128001-2523) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/admin/mbeans params={qt=/admin/mbeans&stats=true&cat=ADMIN&wt=javabin&version=2&key=org.apache.solr.handler.admin.CoreAdminHandler} status=0 QTime=2 2> 614502 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Found 0 requests to /admin/cores on http://127.0.0.1:44127/solr 2> 614517 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.c.S.Request webapp=/solr path=/admin/mbeans params={qt=/admin/mbeans&stats=true&cat=ADMIN&wt=javabin&version=2&key=org.apache.solr.handler.admin.CoreAdminHandler} status=0 QTime=3 2> 614520 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Found 0 requests to /admin/cores on http://127.0.0.1:40391/solr 2> 614633 INFO (TEST-CloudSolrClientTest.testNonRetryableRequests-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Ending testNonRetryableRequests 2> 614964 INFO (TEST-CloudSolrClientTest.testShutdown-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Starting testShutdown 2> 614972 WARN (TEST-CloudSolrClientTest.testShutdown-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 614982 INFO (TEST-CloudSolrClientTest.testShutdown-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 100ms for client to connect to ZooKeeper 2> 614989 WARN (TEST-CloudSolrClientTest.testShutdown-seed#[9CAA27B60427D55F]-SendThread(0:0:0:0:0:0:0:1:4)) [] o.a.z.ClientCnxn Session 0x0 for server localhost/0:0:0:0:0:0:0:1:4, Closing socket connection. Attempting reconnect except it is a SessionExpiredException. 2> => java.net.ConnectException: Connection refused 2> at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) 2> java.net.ConnectException: Connection refused 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777) ~[?:?] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:344) ~[zookeeper-3.8.1.jar:3.8.1] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1289) [zookeeper-3.8.1.jar:3.8.1] 2> 615012 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 615014 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 615014 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 615015 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 615015 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 615015 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 616020 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 616022 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 616022 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 616023 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 616023 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 616023 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 616218 INFO (TEST-CloudSolrClientTest.testShutdown-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Ending testShutdown 2> 616473 INFO (TEST-CloudSolrClientTest.testInitializationWithSolrUrls-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Starting testInitializationWithSolrUrls 2> 616504 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CreateCollectionCmd Create collection testInitializationWithSolrUrls 2> 616671 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"testInitializationWithSolrUrls_shard2_replica_n1", 2> "node_name":"127.0.0.1:39159_solr", 2> "base_url":"http://127.0.0.1:39159/solr", 2> "collection":"testInitializationWithSolrUrls", 2> "shard":"shard2", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 616682 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"testInitializationWithSolrUrls_shard1_replica_n2", 2> "node_name":"127.0.0.1:40391_solr", 2> "base_url":"http://127.0.0.1:40391/solr", 2> "collection":"testInitializationWithSolrUrls", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 616711 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=testInitializationWithSolrUrls_shard2_replica_n1&action=CREATE&numShards=2&collection=testInitializationWithSolrUrls&shard=shard2&wt=javabin&version=2&replicaType=NRT 2> 616713 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=testInitializationWithSolrUrls_shard1_replica_n2&action=CREATE&numShards=2&collection=testInitializationWithSolrUrls&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 616800 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 616805 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.s.IndexSchema Schema name=test 2> 616823 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 616832 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.s.IndexSchema Schema name=test 2> 616841 WARN (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 616841 WARN (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 616841 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 616845 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'testInitializationWithSolrUrls_shard1_replica_n2' using configuration from configset conf, trusted=true 2> 616848 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/testInitializationWithSolrUrls_shard1_replica_n2], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/testInitializationWithSolrUrls_shard1_replica_n2/data/] 2> 616881 WARN (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 616881 WARN (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 616883 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 616885 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testInitializationWithSolrUrls_shard2_replica_n1' using configuration from configset conf, trusted=true 2> 616889 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/testInitializationWithSolrUrls_shard2_replica_n1], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/testInitializationWithSolrUrls_shard2_replica_n1/data/] 2> 617027 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 617029 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 617030 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 617030 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 617030 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 617030 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 617090 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 617090 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 617106 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 617106 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 617127 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 617130 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 617135 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 617137 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736781947043840 2> 617144 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 617144 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 617182 INFO (searcherExecutor-1354-thread-1-processing-127.0.0.1:40391_solr testInitializationWithSolrUrls_shard1_replica_n2 testInitializationWithSolrUrls shard1 core_node4) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 617184 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 617184 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 617203 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testInitializationWithSolrUrls/terms/shard1 to Terms{values={core_node4=0}, version=0} for registerTerm 2> 617204 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/testInitializationWithSolrUrls/leaders/shard1 2> 617214 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 617220 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 617227 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 617229 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736782043512832 2> 617267 INFO (searcherExecutor-1356-thread-1-processing-127.0.0.1:39159_solr testInitializationWithSolrUrls_shard2_replica_n1 testInitializationWithSolrUrls shard2 core_node3) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 617270 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 617270 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 617270 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40391/solr/testInitializationWithSolrUrls_shard1_replica_n2/ 2> 617274 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 617278 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:40391/solr/testInitializationWithSolrUrls_shard1_replica_n2/ has no replicas 2> 617278 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/testInitializationWithSolrUrls/leaders/shard1/leader after winning as /collections/testInitializationWithSolrUrls/leader_elect/shard1/election/72081801657778183-core_node4-n_0000000000 2> 617304 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40391/solr/testInitializationWithSolrUrls_shard1_replica_n2/ shard1 2> 617317 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testInitializationWithSolrUrls/terms/shard2 to Terms{values={core_node3=0}, version=0} for registerTerm 2> 617319 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/testInitializationWithSolrUrls/leaders/shard2 2> 617336 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 617366 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 617372 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 617373 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39159/solr/testInitializationWithSolrUrls_shard2_replica_n1/ 2> 617377 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 617377 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=testInitializationWithSolrUrls_shard1_replica_n2&action=CREATE&numShards=2&collection=testInitializationWithSolrUrls&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=666 2> 617381 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:39159/solr/testInitializationWithSolrUrls_shard2_replica_n1/ has no replicas 2> 617381 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/testInitializationWithSolrUrls/leaders/shard2/leader after winning as /collections/testInitializationWithSolrUrls/leader_elect/shard2/election/72081801657778182-core_node3-n_0000000000 2> 617398 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testInitializationWithSolrUrls/state.json] for collection [testInitializationWithSolrUrls] has occurred - updating... (live nodes size: [3]) 2> 617398 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39159/solr/testInitializationWithSolrUrls_shard2_replica_n1/ shard2 2> 617421 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 617431 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testInitializationWithSolrUrls/state.json] for collection [testInitializationWithSolrUrls] has occurred - updating... (live nodes size: [3]) 2> 617456 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=testInitializationWithSolrUrls_shard2_replica_n1&action=CREATE&numShards=2&collection=testInitializationWithSolrUrls&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=747 2> 617471 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas 2> 617471 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf&name=testInitializationWithSolrUrls&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=993 2> 617473 INFO (TEST-CloudSolrClientTest.testInitializationWithSolrUrls-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForActiveCollection: testInitializationWithSolrUrls 2> 617497 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testInitializationWithSolrUrls&wt=javabin&version=2} status=0 QTime=4 2> 617506 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testInitializationWithSolrUrls&wt=javabin&version=2} status=0 QTime=4 2> 617513 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LISTALIASES&wt=javabin&version=2} status=0 QTime=4 2> 617522 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testInitializationWithSolrUrls&wt=javabin&version=2} status=0 QTime=5 2> 617554 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testInitializationWithSolrUrls/terms/shard1 to Terms{values={core_node4=1}, version=1} for ensureHighestTermsAreNotZero 2> 617554 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/update params={wt=javabin&version=2} status=0 QTime=26 2> 617574 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testInitializationWithSolrUrls&wt=javabin&version=2} status=0 QTime=15 2> 617583 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testInitializationWithSolrUrls&wt=javabin&version=2} status=0 QTime=5 2> 617592 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testInitializationWithSolrUrls&wt=javabin&version=2} status=0 QTime=6 2> 617609 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=12 2> 617620 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testInitializationWithSolrUrls&wt=javabin&version=2} status=0 QTime=4 2> 617629 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testInitializationWithSolrUrls&wt=javabin&version=2} status=0 QTime=4 2> 617708 INFO (searcherExecutor-1354-thread-1-processing-127.0.0.1:40391_solr testInitializationWithSolrUrls_shard1_replica_n2 testInitializationWithSolrUrls shard1 core_node4) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 617710 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:39159/solr/testInitializationWithSolrUrls_shard2_replica_n1/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false} status=0 QTime=63 2> 617712 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=testInitializationWithSolrUrls:7&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=77 2> 617720 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testInitializationWithSolrUrls&wt=javabin&version=2} status=0 QTime=5 2> 617729 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testInitializationWithSolrUrls&wt=javabin&version=2} status=0 QTime=5 2> 617742 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=9 2> 617753 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testInitializationWithSolrUrls&wt=javabin&version=2} status=0 QTime=5 2> 617761 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testInitializationWithSolrUrls&wt=javabin&version=2} status=0 QTime=5 2> 617773 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testInitializationWithSolrUrls:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:39159/solr/testInitializationWithSolrUrls_shard2_replica_n1/&rid=127.0.0.1-1367&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335585219&isShard=true&wt=javabin} hits=0 status=0 QTime=1 2> 617780 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testInitializationWithSolrUrls:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:40391/solr/testInitializationWithSolrUrls_shard1_replica_n2/&rid=127.0.0.1-1367&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335585219&isShard=true&wt=javabin} hits=1 status=0 QTime=2 2> 617787 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testInitializationWithSolrUrls:7&shards.purpose=64&shard.url=http://127.0.0.1:40391/solr/testInitializationWithSolrUrls_shard1_replica_n2/&rid=127.0.0.1-1367&version=2&q=*:*&omitHeader=false&NOW=1696335585219&ids=1&isShard=true&wt=javabin} status=0 QTime=2 2> 617791 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=testInitializationWithSolrUrls:7&wt=javabin&version=2} rid=127.0.0.1-1367 hits=1 status=0 QTime=26 2> 617795 INFO (TEST-CloudSolrClientTest.testInitializationWithSolrUrls-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Ending testInitializationWithSolrUrls 2> 618040 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 618043 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 618043 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 618045 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 618045 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 618045 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 618059 INFO (TEST-CloudSolrClientTest.preferReplicaTypesTest-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Starting preferReplicaTypesTest 2> 618076 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=1&async=5303af29-6bf9-481e-8638-bec815fe689a&collection.configName=conf&name=replicaTypesTestColl&nrtReplicas=1&action=CREATE&numShards=3&tlogReplicas=1&wt=javabin&version=2} status=0 QTime=14 2> 618093 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5303af29-6bf9-481e-8638-bec815fe689a&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=10 2> 618099 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CreateCollectionCmd Create collection replicaTypesTestColl 2> 618102 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000019 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 618348 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replicaTypesTestColl_shard3_replica_n1", 2> "node_name":"127.0.0.1:39159_solr", 2> "base_url":"http://127.0.0.1:39159/solr", 2> "collection":"replicaTypesTestColl", 2> "shard":"shard3", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 618359 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replicaTypesTestColl_shard1_replica_t2", 2> "node_name":"127.0.0.1:40391_solr", 2> "base_url":"http://127.0.0.1:40391/solr", 2> "collection":"replicaTypesTestColl", 2> "shard":"shard1", 2> "state":"down", 2> "type":"TLOG", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 618370 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replicaTypesTestColl_shard2_replica_t3", 2> "node_name":"127.0.0.1:40391_solr", 2> "base_url":"http://127.0.0.1:40391/solr", 2> "collection":"replicaTypesTestColl", 2> "shard":"shard2", 2> "state":"down", 2> "type":"TLOG", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 618381 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replicaTypesTestColl_shard3_replica_t4", 2> "node_name":"127.0.0.1:40391_solr", 2> "base_url":"http://127.0.0.1:40391/solr", 2> "collection":"replicaTypesTestColl", 2> "shard":"shard3", 2> "state":"down", 2> "type":"TLOG", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 618392 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replicaTypesTestColl_shard3_replica_p5", 2> "node_name":"127.0.0.1:44127_solr", 2> "base_url":"http://127.0.0.1:44127/solr", 2> "collection":"replicaTypesTestColl", 2> "shard":"shard3", 2> "state":"down", 2> "type":"PULL", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 618402 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replicaTypesTestColl_shard1_replica_p6", 2> "node_name":"127.0.0.1:44127_solr", 2> "base_url":"http://127.0.0.1:44127/solr", 2> "collection":"replicaTypesTestColl", 2> "shard":"shard1", 2> "state":"down", 2> "type":"PULL", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 618412 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replicaTypesTestColl_shard2_replica_n7", 2> "node_name":"127.0.0.1:39159_solr", 2> "base_url":"http://127.0.0.1:39159/solr", 2> "collection":"replicaTypesTestColl", 2> "shard":"shard2", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 618423 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replicaTypesTestColl_shard2_replica_p8", 2> "node_name":"127.0.0.1:44127_solr", 2> "base_url":"http://127.0.0.1:44127/solr", 2> "collection":"replicaTypesTestColl", 2> "shard":"shard2", 2> "state":"down", 2> "type":"PULL", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 618434 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"replicaTypesTestColl_shard1_replica_n9", 2> "node_name":"127.0.0.1:39159_solr", 2> "base_url":"http://127.0.0.1:39159/solr", 2> "collection":"replicaTypesTestColl", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 618468 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard1_replica_t2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=replicaTypesTestColl&version=2&replicaType=TLOG&async=5303af29-6bf9-481e-8638-bec815fe689a369420682171049&coreNodeName=core_node11&name=replicaTypesTestColl_shard1_replica_t2&action=CREATE&numShards=3&shard=shard1&wt=javabin} status=0 QTime=3 2> 618469 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard1_replica_t2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=replicaTypesTestColl&version=2&replicaType=TLOG&async=5303af29-6bf9-481e-8638-bec815fe689a369420682171049&coreNodeName=core_node11&name=replicaTypesTestColl_shard1_replica_t2&action=CREATE&numShards=3&shard=shard1&wt=javabin 2> 618483 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard2_replica_t3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=replicaTypesTestColl&version=2&replicaType=TLOG&async=5303af29-6bf9-481e-8638-bec815fe689a369420693854353&coreNodeName=core_node12&name=replicaTypesTestColl_shard2_replica_t3&action=CREATE&numShards=3&shard=shard2&wt=javabin} status=0 QTime=15 2> 618484 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-6-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 create) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard2_replica_t3] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=replicaTypesTestColl&version=2&replicaType=TLOG&async=5303af29-6bf9-481e-8638-bec815fe689a369420693854353&coreNodeName=core_node12&name=replicaTypesTestColl_shard2_replica_t3&action=CREATE&numShards=3&shard=shard2&wt=javabin 2> 618488 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard1_replica_p6] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=replicaTypesTestColl&version=2&replicaType=PULL&async=5303af29-6bf9-481e-8638-bec815fe689a369420726232531&coreNodeName=core_node15&name=replicaTypesTestColl_shard1_replica_p6&action=CREATE&numShards=3&shard=shard1&wt=javabin} status=0 QTime=8 2> 618491 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-3-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard1_replica_p6 5303af29-6bf9-481e-8638-bec815fe689a369420726232531 create) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard1_replica_p6] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=replicaTypesTestColl&version=2&replicaType=PULL&async=5303af29-6bf9-481e-8638-bec815fe689a369420726232531&coreNodeName=core_node15&name=replicaTypesTestColl_shard1_replica_p6&action=CREATE&numShards=3&shard=shard1&wt=javabin 2> 618491 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr x:replicaTypesTestColl_shard2_replica_n7] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=replicaTypesTestColl&version=2&replicaType=NRT&async=5303af29-6bf9-481e-8638-bec815fe689a369420737162482&coreNodeName=core_node16&name=replicaTypesTestColl_shard2_replica_n7&action=CREATE&numShards=3&shard=shard2&wt=javabin} status=0 QTime=5 2> 618493 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr x:replicaTypesTestColl_shard2_replica_n7] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=replicaTypesTestColl&version=2&replicaType=NRT&async=5303af29-6bf9-481e-8638-bec815fe689a369420737162482&coreNodeName=core_node16&name=replicaTypesTestColl_shard2_replica_n7&action=CREATE&numShards=3&shard=shard2&wt=javabin 2> 618497 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard3_replica_t4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=replicaTypesTestColl&version=2&replicaType=TLOG&async=5303af29-6bf9-481e-8638-bec815fe689a369420704978310&coreNodeName=core_node13&name=replicaTypesTestColl_shard3_replica_t4&action=CREATE&numShards=3&shard=shard3&wt=javabin} status=0 QTime=24 2> 618498 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-7-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 create) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard3_replica_t4] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=replicaTypesTestColl&version=2&replicaType=TLOG&async=5303af29-6bf9-481e-8638-bec815fe689a369420704978310&coreNodeName=core_node13&name=replicaTypesTestColl_shard3_replica_t4&action=CREATE&numShards=3&shard=shard3&wt=javabin 2> 618502 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-4-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard2_replica_p8 5303af29-6bf9-481e-8638-bec815fe689a369420748091101 create) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard2_replica_p8] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=replicaTypesTestColl&version=2&replicaType=PULL&async=5303af29-6bf9-481e-8638-bec815fe689a369420748091101&coreNodeName=core_node17&name=replicaTypesTestColl_shard2_replica_p8&action=CREATE&numShards=3&shard=shard2&wt=javabin 2> 618502 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr x:replicaTypesTestColl_shard3_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=replicaTypesTestColl&version=2&replicaType=NRT&async=5303af29-6bf9-481e-8638-bec815fe689a369420671288431&coreNodeName=core_node10&name=replicaTypesTestColl_shard3_replica_n1&action=CREATE&numShards=3&shard=shard3&wt=javabin 2> 618501 INFO (qtp218128001-2528) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard2_replica_p8] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=replicaTypesTestColl&version=2&replicaType=PULL&async=5303af29-6bf9-481e-8638-bec815fe689a369420748091101&coreNodeName=core_node17&name=replicaTypesTestColl_shard2_replica_p8&action=CREATE&numShards=3&shard=shard2&wt=javabin} status=0 QTime=19 2> 618501 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr x:replicaTypesTestColl_shard3_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=replicaTypesTestColl&version=2&replicaType=NRT&async=5303af29-6bf9-481e-8638-bec815fe689a369420671288431&coreNodeName=core_node10&name=replicaTypesTestColl_shard3_replica_n1&action=CREATE&numShards=3&shard=shard3&wt=javabin} status=0 QTime=36 2> 618511 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr x:replicaTypesTestColl_shard1_replica_n9] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=replicaTypesTestColl&version=2&replicaType=NRT&async=5303af29-6bf9-481e-8638-bec815fe689a369420758822714&coreNodeName=core_node18&name=replicaTypesTestColl_shard1_replica_n9&action=CREATE&numShards=3&shard=shard1&wt=javabin} status=0 QTime=25 2> 618516 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-8-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 5303af29-6bf9-481e-8638-bec815fe689a369420758822714 create) [n:127.0.0.1:39159_solr x:replicaTypesTestColl_shard1_replica_n9] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=replicaTypesTestColl&version=2&replicaType=NRT&async=5303af29-6bf9-481e-8638-bec815fe689a369420758822714&coreNodeName=core_node18&name=replicaTypesTestColl_shard1_replica_n9&action=CREATE&numShards=3&shard=shard1&wt=javabin 2> 618528 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard3_replica_p5] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=replicaTypesTestColl&version=2&replicaType=PULL&async=5303af29-6bf9-481e-8638-bec815fe689a369420715487584&coreNodeName=core_node14&name=replicaTypesTestColl_shard3_replica_p5&action=CREATE&numShards=3&shard=shard3&wt=javabin} status=0 QTime=58 2> 618530 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-5-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard3_replica_p5 5303af29-6bf9-481e-8638-bec815fe689a369420715487584 create) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard3_replica_p5] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=replicaTypesTestColl&version=2&replicaType=PULL&async=5303af29-6bf9-481e-8638-bec815fe689a369420715487584&coreNodeName=core_node14&name=replicaTypesTestColl_shard3_replica_p5&action=CREATE&numShards=3&shard=shard3&wt=javabin 2> 618536 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 2> 618537 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5303af29-6bf9-481e-8638-bec815fe689a369420671288431&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 618582 INFO (zkCallback-1166-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 618785 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 618798 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.s.IndexSchema Schema name=test 2> 618837 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-6-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 618893 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-6-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.s.IndexSchema Schema name=test 2> 618909 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 618929 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.s.IndexSchema Schema name=test 2> 618938 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-7-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 618939 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-3-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard1_replica_p6 5303af29-6bf9-481e-8638-bec815fe689a369420726232531 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard1 r:core_node15 x:replicaTypesTestColl_shard1_replica_p6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 618952 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-3-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard1_replica_p6 5303af29-6bf9-481e-8638-bec815fe689a369420726232531 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard1 r:core_node15 x:replicaTypesTestColl_shard1_replica_p6] o.a.s.s.IndexSchema Schema name=test 2> 618954 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-7-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.s.IndexSchema Schema name=test 2> 618961 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 618986 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.s.IndexSchema Schema name=test 2> 618988 WARN (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 618988 WARN (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 618989 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 618990 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.CoreContainer Creating SolrCore 'replicaTypesTestColl_shard2_replica_n7' using configuration from configset conf, trusted=true 2> 618993 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/replicaTypesTestColl_shard2_replica_n7], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/replicaTypesTestColl_shard2_replica_n7/data/] 2> 618997 WARN (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 618998 WARN (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 618999 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 619000 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.CoreContainer Creating SolrCore 'replicaTypesTestColl_shard1_replica_t2' using configuration from configset conf, trusted=true 2> 619013 WARN (parallelCoreAdminAPIBaseExecutor-1179-thread-6-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 619015 WARN (parallelCoreAdminAPIBaseExecutor-1179-thread-6-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 619016 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/replicaTypesTestColl_shard1_replica_t2], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/replicaTypesTestColl_shard1_replica_t2/data/] 2> 619017 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-6-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 619021 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-4-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard2_replica_p8 5303af29-6bf9-481e-8638-bec815fe689a369420748091101 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard2 r:core_node17 x:replicaTypesTestColl_shard2_replica_p8] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 619027 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-6-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.c.CoreContainer Creating SolrCore 'replicaTypesTestColl_shard2_replica_t3' using configuration from configset conf, trusted=true 2> 619028 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-8-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 5303af29-6bf9-481e-8638-bec815fe689a369420758822714 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 619035 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-4-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard2_replica_p8 5303af29-6bf9-481e-8638-bec815fe689a369420748091101 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard2 r:core_node17 x:replicaTypesTestColl_shard2_replica_p8] o.a.s.s.IndexSchema Schema name=test 2> 619036 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-5-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard3_replica_p5 5303af29-6bf9-481e-8638-bec815fe689a369420715487584 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard3 r:core_node14 x:replicaTypesTestColl_shard3_replica_p5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 619042 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-8-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 5303af29-6bf9-481e-8638-bec815fe689a369420758822714 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.s.IndexSchema Schema name=test 2> 619050 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 619053 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-5-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard3_replica_p5 5303af29-6bf9-481e-8638-bec815fe689a369420715487584 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard3 r:core_node14 x:replicaTypesTestColl_shard3_replica_p5] o.a.s.s.IndexSchema Schema name=test 2> 619053 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 619055 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 619055 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 619056 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 619056 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 619077 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-6-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/replicaTypesTestColl_shard2_replica_t3], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/replicaTypesTestColl_shard2_replica_t3/data/] 2> 619083 WARN (parallelCoreAdminAPIBaseExecutor-1178-thread-3-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard1_replica_p6 5303af29-6bf9-481e-8638-bec815fe689a369420726232531 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard1 r:core_node15 x:replicaTypesTestColl_shard1_replica_p6] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 619084 WARN (parallelCoreAdminAPIBaseExecutor-1178-thread-3-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard1_replica_p6 5303af29-6bf9-481e-8638-bec815fe689a369420726232531 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard1 r:core_node15 x:replicaTypesTestColl_shard1_replica_p6] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 619085 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-3-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard1_replica_p6 5303af29-6bf9-481e-8638-bec815fe689a369420726232531 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard1 r:core_node15 x:replicaTypesTestColl_shard1_replica_p6] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 619086 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-3-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard1_replica_p6 5303af29-6bf9-481e-8638-bec815fe689a369420726232531 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard1 r:core_node15 x:replicaTypesTestColl_shard1_replica_p6] o.a.s.c.CoreContainer Creating SolrCore 'replicaTypesTestColl_shard1_replica_p6' using configuration from configset conf, trusted=true 2> 619107 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5303af29-6bf9-481e-8638-bec815fe689a&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=9 2> 619128 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-3-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard1_replica_p6 5303af29-6bf9-481e-8638-bec815fe689a369420726232531 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard1 r:core_node15 x:replicaTypesTestColl_shard1_replica_p6] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node3/replicaTypesTestColl_shard1_replica_p6], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node3/replicaTypesTestColl_shard1_replica_p6/data/] 2> 619144 WARN (parallelCoreAdminAPIBaseExecutor-1178-thread-5-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard3_replica_p5 5303af29-6bf9-481e-8638-bec815fe689a369420715487584 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard3 r:core_node14 x:replicaTypesTestColl_shard3_replica_p5] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 619145 WARN (parallelCoreAdminAPIBaseExecutor-1178-thread-5-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard3_replica_p5 5303af29-6bf9-481e-8638-bec815fe689a369420715487584 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard3 r:core_node14 x:replicaTypesTestColl_shard3_replica_p5] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 619150 WARN (parallelCoreAdminAPIBaseExecutor-1179-thread-7-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 619150 WARN (parallelCoreAdminAPIBaseExecutor-1179-thread-7-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 619156 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-7-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 619158 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-7-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.c.CoreContainer Creating SolrCore 'replicaTypesTestColl_shard3_replica_t4' using configuration from configset conf, trusted=true 2> 619167 WARN (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 619167 WARN (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 619168 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 619170 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'replicaTypesTestColl_shard3_replica_n1' using configuration from configset conf, trusted=true 2> 619177 WARN (parallelCoreAdminAPIBaseExecutor-1177-thread-8-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 5303af29-6bf9-481e-8638-bec815fe689a369420758822714 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 619177 WARN (parallelCoreAdminAPIBaseExecutor-1177-thread-8-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 5303af29-6bf9-481e-8638-bec815fe689a369420758822714 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 619177 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-7-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/replicaTypesTestColl_shard3_replica_t4], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/replicaTypesTestColl_shard3_replica_t4/data/] 2> 619178 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-8-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 5303af29-6bf9-481e-8638-bec815fe689a369420758822714 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 619182 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-8-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 5303af29-6bf9-481e-8638-bec815fe689a369420758822714 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.c.CoreContainer Creating SolrCore 'replicaTypesTestColl_shard1_replica_n9' using configuration from configset conf, trusted=true 2> 619184 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-5-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard3_replica_p5 5303af29-6bf9-481e-8638-bec815fe689a369420715487584 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard3 r:core_node14 x:replicaTypesTestColl_shard3_replica_p5] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 619186 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-8-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 5303af29-6bf9-481e-8638-bec815fe689a369420758822714 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/replicaTypesTestColl_shard1_replica_n9], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/replicaTypesTestColl_shard1_replica_n9/data/] 2> 619187 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/replicaTypesTestColl_shard3_replica_n1], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/replicaTypesTestColl_shard3_replica_n1/data/] 2> 619190 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-5-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard3_replica_p5 5303af29-6bf9-481e-8638-bec815fe689a369420715487584 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard3 r:core_node14 x:replicaTypesTestColl_shard3_replica_p5] o.a.s.c.CoreContainer Creating SolrCore 'replicaTypesTestColl_shard3_replica_p5' using configuration from configset conf, trusted=true 2> 619186 WARN (parallelCoreAdminAPIBaseExecutor-1178-thread-4-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard2_replica_p8 5303af29-6bf9-481e-8638-bec815fe689a369420748091101 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard2 r:core_node17 x:replicaTypesTestColl_shard2_replica_p8] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 619195 WARN (parallelCoreAdminAPIBaseExecutor-1178-thread-4-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard2_replica_p8 5303af29-6bf9-481e-8638-bec815fe689a369420748091101 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard2 r:core_node17 x:replicaTypesTestColl_shard2_replica_p8] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 619196 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-4-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard2_replica_p8 5303af29-6bf9-481e-8638-bec815fe689a369420748091101 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard2 r:core_node17 x:replicaTypesTestColl_shard2_replica_p8] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 619198 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-4-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard2_replica_p8 5303af29-6bf9-481e-8638-bec815fe689a369420748091101 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard2 r:core_node17 x:replicaTypesTestColl_shard2_replica_p8] o.a.s.c.CoreContainer Creating SolrCore 'replicaTypesTestColl_shard2_replica_p8' using configuration from configset conf, trusted=true 2> 619201 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-4-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard2_replica_p8 5303af29-6bf9-481e-8638-bec815fe689a369420748091101 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard2 r:core_node17 x:replicaTypesTestColl_shard2_replica_p8] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node3/replicaTypesTestColl_shard2_replica_p8], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node3/replicaTypesTestColl_shard2_replica_p8/data/] 2> 619198 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-5-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard3_replica_p5 5303af29-6bf9-481e-8638-bec815fe689a369420715487584 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard3 r:core_node14 x:replicaTypesTestColl_shard3_replica_p5] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node3/replicaTypesTestColl_shard3_replica_p5], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node3/replicaTypesTestColl_shard3_replica_p5/data/] 2> 619545 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 2> 619546 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5303af29-6bf9-481e-8638-bec815fe689a369420671288431&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 620060 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 620062 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 620062 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 620062 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 620062 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 620062 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 620120 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5303af29-6bf9-481e-8638-bec815fe689a&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=8 2> 620240 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 620241 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 620294 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 620294 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 620301 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 620303 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 620348 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-6-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 620350 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-6-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 620354 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 620357 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 620387 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 620391 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 620403 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-3-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard1_replica_p6 5303af29-6bf9-481e-8638-bec815fe689a369420726232531 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard1 r:core_node15 x:replicaTypesTestColl_shard1_replica_p6] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 620403 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-3-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard1_replica_p6 5303af29-6bf9-481e-8638-bec815fe689a369420726232531 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard1 r:core_node15 x:replicaTypesTestColl_shard1_replica_p6] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 620408 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-6-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 620408 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-6-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 620420 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 620421 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736785390567424 2> 620916 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 2> 620916 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5303af29-6bf9-481e-8638-bec815fe689a369420671288431&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 620953 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 620960 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 620982 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 620982 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 620987 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replicaTypesTestColl/terms/shard1 to Terms{values={core_node11=0}, version=0} for registerTerm 2> 620988 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replicaTypesTestColl/leaders/shard1 2> 621002 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 621003 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736786000838656 2> 621019 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-3-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard1_replica_p6 5303af29-6bf9-481e-8638-bec815fe689a369420726232531 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard1 r:core_node15 x:replicaTypesTestColl_shard1_replica_p6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 621024 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-3-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard1_replica_p6 5303af29-6bf9-481e-8638-bec815fe689a369420726232531 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard1 r:core_node15 x:replicaTypesTestColl_shard1_replica_p6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 621056 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-8-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 5303af29-6bf9-481e-8638-bec815fe689a369420758822714 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 621056 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-8-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 5303af29-6bf9-481e-8638-bec815fe689a369420758822714 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 621057 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-6-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 621063 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-7-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 621068 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-6-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 621066 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-7-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 621068 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=2 2> 621074 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 621075 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 621075 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 621075 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 621075 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 621084 INFO (searcherExecutor-1368-thread-1-processing-replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 core_node11 create 127.0.0.1:40391_solr replicaTypesTestColl shard1) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 621093 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-3-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard1_replica_p6 5303af29-6bf9-481e-8638-bec815fe689a369420726232531 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard1 r:core_node15 x:replicaTypesTestColl_shard1_replica_p6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 621099 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 621100 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 621132 INFO (zkCallback-1166-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 621138 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-5-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard3_replica_p5 5303af29-6bf9-481e-8638-bec815fe689a369420715487584 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard3 r:core_node14 x:replicaTypesTestColl_shard3_replica_p5] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 621140 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-5-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard3_replica_p5 5303af29-6bf9-481e-8638-bec815fe689a369420715487584 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard3 r:core_node14 x:replicaTypesTestColl_shard3_replica_p5] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 621144 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-4-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard2_replica_p8 5303af29-6bf9-481e-8638-bec815fe689a369420748091101 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard2 r:core_node17 x:replicaTypesTestColl_shard2_replica_p8] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 621145 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-6-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 621146 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-4-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard2_replica_p8 5303af29-6bf9-481e-8638-bec815fe689a369420748091101 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard2 r:core_node17 x:replicaTypesTestColl_shard2_replica_p8] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 621147 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-6-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736786151833600 2> 621159 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5303af29-6bf9-481e-8638-bec815fe689a&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=33 2> 621162 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoute in=9972ms 2> 621164 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-8-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 5303af29-6bf9-481e-8638-bec815fe689a369420758822714 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 621164 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-8-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 5303af29-6bf9-481e-8638-bec815fe689a369420758822714 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 621177 INFO (searcherExecutor-1366-thread-1-processing-replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 core_node16 create 127.0.0.1:39159_solr replicaTypesTestColl shard2) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 621182 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-7-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 621182 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-7-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 621193 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replicaTypesTestColl/terms/shard2 to Terms{values={core_node16=0}, version=0} for registerTerm 2> 621193 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replicaTypesTestColl/leaders/shard2 2> 621226 INFO (searcherExecutor-1372-thread-1-processing-replicaTypesTestColl_shard1_replica_p6 5303af29-6bf9-481e-8638-bec815fe689a369420726232531 core_node15 create 127.0.0.1:44127_solr replicaTypesTestColl shard1) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard1 r:core_node15 x:replicaTypesTestColl_shard1_replica_p6] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 621237 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-3-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard1_replica_p6 5303af29-6bf9-481e-8638-bec815fe689a369420726232531 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard1 r:core_node15 x:replicaTypesTestColl_shard1_replica_p6] o.a.s.c.ZkController replicaTypesTestColl_shard1_replica_p6 starting background replication from leader 2> 621239 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-3-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard1_replica_p6 5303af29-6bf9-481e-8638-bec815fe689a369420726232531 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard1 r:core_node15 x:replicaTypesTestColl_shard1_replica_p6] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:01 2> 621253 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 621267 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 621267 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-4-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard2_replica_p8 5303af29-6bf9-481e-8638-bec815fe689a369420748091101 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard2 r:core_node17 x:replicaTypesTestColl_shard2_replica_p8] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 621269 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-3-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard1_replica_p6 5303af29-6bf9-481e-8638-bec815fe689a369420726232531 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard1 r:core_node15 x:replicaTypesTestColl_shard1_replica_p6] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms 2> 621270 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-3-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard1_replica_p6 5303af29-6bf9-481e-8638-bec815fe689a369420726232531 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard1 r:core_node15 x:replicaTypesTestColl_shard1_replica_p6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 621281 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 621282 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736786293391360 2> 621285 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-4-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard2_replica_p8 5303af29-6bf9-481e-8638-bec815fe689a369420748091101 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard2 r:core_node17 x:replicaTypesTestColl_shard2_replica_p8] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 621290 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-7-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 621294 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-4-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard2_replica_p8 5303af29-6bf9-481e-8638-bec815fe689a369420748091101 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard2 r:core_node17 x:replicaTypesTestColl_shard2_replica_p8] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 621295 INFO (searcherExecutor-1370-thread-1-processing-replicaTypesTestColl_shard2_replica_t3 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 core_node12 create 127.0.0.1:40391_solr replicaTypesTestColl shard2) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 621302 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-7-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 621309 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-7-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 621310 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-7-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736786322751488 2> 621318 INFO (searcherExecutor-1382-thread-1-processing-replicaTypesTestColl_shard2_replica_p8 5303af29-6bf9-481e-8638-bec815fe689a369420748091101 core_node17 create 127.0.0.1:44127_solr replicaTypesTestColl shard2) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard2 r:core_node17 x:replicaTypesTestColl_shard2_replica_p8] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 621327 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-5-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard3_replica_p5 5303af29-6bf9-481e-8638-bec815fe689a369420715487584 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard3 r:core_node14 x:replicaTypesTestColl_shard3_replica_p5] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 621338 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-8-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 5303af29-6bf9-481e-8638-bec815fe689a369420758822714 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 621346 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-5-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard3_replica_p5 5303af29-6bf9-481e-8638-bec815fe689a369420715487584 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard3 r:core_node14 x:replicaTypesTestColl_shard3_replica_p5] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 621361 INFO (searcherExecutor-1376-thread-1-processing-replicaTypesTestColl_shard3_replica_t4 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 core_node13 create 127.0.0.1:40391_solr replicaTypesTestColl shard3) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 621365 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-8-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 5303af29-6bf9-481e-8638-bec815fe689a369420758822714 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 621370 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-5-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard3_replica_p5 5303af29-6bf9-481e-8638-bec815fe689a369420715487584 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard3 r:core_node14 x:replicaTypesTestColl_shard3_replica_p5] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 621380 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-8-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 5303af29-6bf9-481e-8638-bec815fe689a369420758822714 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 621383 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-8-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 5303af29-6bf9-481e-8638-bec815fe689a369420758822714 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736786399297536 2> 621392 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-6-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replicaTypesTestColl/terms/shard2 to Terms{values={core_node16=0, core_node12=0}, version=1} for registerTerm 2> 621392 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-6-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replicaTypesTestColl/leaders/shard2 2> 621394 INFO (searcherExecutor-1374-thread-1-processing-replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 core_node10 create 127.0.0.1:39159_solr replicaTypesTestColl shard3) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 621443 INFO (searcherExecutor-1380-thread-1-processing-replicaTypesTestColl_shard3_replica_p5 5303af29-6bf9-481e-8638-bec815fe689a369420715487584 core_node14 create 127.0.0.1:44127_solr replicaTypesTestColl shard3) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard3 r:core_node14 x:replicaTypesTestColl_shard3_replica_p5] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 621444 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoute in=9956ms 2> 621480 INFO (searcherExecutor-1378-thread-1-processing-replicaTypesTestColl_shard1_replica_n9 5303af29-6bf9-481e-8638-bec815fe689a369420758822714 core_node18 create 127.0.0.1:39159_solr replicaTypesTestColl shard1) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 621483 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-4-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard2_replica_p8 5303af29-6bf9-481e-8638-bec815fe689a369420748091101 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard2 r:core_node17 x:replicaTypesTestColl_shard2_replica_p8] o.a.s.c.ZkController replicaTypesTestColl_shard2_replica_p8 starting background replication from leader 2> 621483 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-4-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard2_replica_p8 5303af29-6bf9-481e-8638-bec815fe689a369420748091101 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard2 r:core_node17 x:replicaTypesTestColl_shard2_replica_p8] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:01 2> 621498 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-4-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard2_replica_p8 5303af29-6bf9-481e-8638-bec815fe689a369420748091101 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard2 r:core_node17 x:replicaTypesTestColl_shard2_replica_p8] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms 2> 621498 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-4-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard2_replica_p8 5303af29-6bf9-481e-8638-bec815fe689a369420748091101 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard2 r:core_node17 x:replicaTypesTestColl_shard2_replica_p8] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 621505 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-7-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.c.ZkShardTerms Failed to save terms, version is not a match, retrying 2> 621507 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replicaTypesTestColl/terms/shard3 to Terms{values={core_node10=0}, version=0} for registerTerm 2> 621507 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replicaTypesTestColl/leaders/shard3 2> 621525 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-7-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replicaTypesTestColl/terms/shard3 to Terms{values={core_node10=0, core_node13=0}, version=1} for registerTerm 2> 621525 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-7-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replicaTypesTestColl/leaders/shard3 2> 621540 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-5-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard3_replica_p5 5303af29-6bf9-481e-8638-bec815fe689a369420715487584 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard3 r:core_node14 x:replicaTypesTestColl_shard3_replica_p5] o.a.s.c.ZkController replicaTypesTestColl_shard3_replica_p5 starting background replication from leader 2> 621541 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-5-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard3_replica_p5 5303af29-6bf9-481e-8638-bec815fe689a369420715487584 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard3 r:core_node14 x:replicaTypesTestColl_shard3_replica_p5] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:01 2> 621560 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-5-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard3_replica_p5 5303af29-6bf9-481e-8638-bec815fe689a369420715487584 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard3 r:core_node14 x:replicaTypesTestColl_shard3_replica_p5] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms 2> 621560 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-5-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard3_replica_p5 5303af29-6bf9-481e-8638-bec815fe689a369420715487584 create) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard3 r:core_node14 x:replicaTypesTestColl_shard3_replica_p5] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 621595 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-8-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 5303af29-6bf9-481e-8638-bec815fe689a369420758822714 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replicaTypesTestColl/terms/shard1 to Terms{values={core_node18=0, core_node11=0}, version=1} for registerTerm 2> 621595 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-8-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 5303af29-6bf9-481e-8638-bec815fe689a369420758822714 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/replicaTypesTestColl/leaders/shard1 2> 621629 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 621629 INFO (zkCallback-1166-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 621641 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 621642 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 621642 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39159/solr/replicaTypesTestColl_shard3_replica_n1/ 2> 621647 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.u.PeerSync PeerSync: core=replicaTypesTestColl_shard3_replica_n1 url=http://127.0.0.1:39159/solr START replicas=[http://127.0.0.1:40391/solr/replicaTypesTestColl_shard3_replica_t4/] nUpdates=100 2> 621650 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.u.PeerSync PeerSync: core=replicaTypesTestColl_shard3_replica_n1 url=http://127.0.0.1:39159/solr DONE. We have no versions. sync failed. 2> 621653 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.c.S.Request webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=2 2> 621656 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate 2> 621656 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway 2> 621656 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/replicaTypesTestColl/leaders/shard3/leader after winning as /collections/replicaTypesTestColl/leader_elect/shard3/election/72081801657778182-core_node10-n_0000000000 2> 621672 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 621672 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 621672 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40391/solr/replicaTypesTestColl_shard1_replica_t2/ 2> 621678 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.u.PeerSync PeerSync: core=replicaTypesTestColl_shard1_replica_t2 url=http://127.0.0.1:40391/solr START replicas=[http://127.0.0.1:39159/solr/replicaTypesTestColl_shard1_replica_n9/] nUpdates=100 2> 621684 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.c.S.Request webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1 2> 621687 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 621687 INFO (zkCallback-1166-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 621689 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.u.PeerSync PeerSync: core=replicaTypesTestColl_shard1_replica_t2 url=http://127.0.0.1:40391/solr DONE. We have no versions. sync failed. 2> 621690 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate 2> 621690 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway 2> 621690 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.ZkController replicaTypesTestColl_shard1_replica_t2 stopping background replication from leader 2> 621691 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/replicaTypesTestColl/leaders/shard1/leader after winning as /collections/replicaTypesTestColl/leader_elect/shard1/election/72081801657778183-core_node11-n_0000000000 2> 621692 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39159/solr/replicaTypesTestColl_shard3_replica_n1/ shard3 2> 621696 INFO (indexFetcher-1421-thread-1) [] o.a.s.h.IndexFetcher Replica core_node10 is leader but it's state is down, skipping replication 2> 621719 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 621719 INFO (zkCallback-1166-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 621719 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40391/solr/replicaTypesTestColl_shard1_replica_t2/ shard1 2> 621726 INFO (indexFetcher-1417-thread-1) [] o.a.s.h.IndexFetcher Replica core_node11 is leader but it's state is down, skipping replication 2> 621748 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-6-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 621769 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-5-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 621922 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5303af29-6bf9-481e-8638-bec815fe689a369420671288431 2> 621923 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5303af29-6bf9-481e-8638-bec815fe689a369420671288431&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 621928 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5303af29-6bf9-481e-8638-bec815fe689a369420682171049 2> 621928 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5303af29-6bf9-481e-8638-bec815fe689a369420682171049&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 621934 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 2> 621935 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5303af29-6bf9-481e-8638-bec815fe689a369420693854353&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 621947 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 621947 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 621947 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39159/solr/replicaTypesTestColl_shard2_replica_n7/ 2> 621949 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.u.PeerSync PeerSync: core=replicaTypesTestColl_shard2_replica_n7 url=http://127.0.0.1:39159/solr START replicas=[http://127.0.0.1:40391/solr/replicaTypesTestColl_shard2_replica_t3/] nUpdates=100 2> 621953 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.u.PeerSync PeerSync: core=replicaTypesTestColl_shard2_replica_n7 url=http://127.0.0.1:39159/solr DONE. We have no versions. sync failed. 2> 621955 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.c.S.Request webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1 2> 621958 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate 2> 621958 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway 2> 621959 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/replicaTypesTestColl/leaders/shard2/leader after winning as /collections/replicaTypesTestColl/leader_elect/shard2/election/72081801657778182-core_node16-n_0000000000 2> 621973 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 621973 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 621977 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39159/solr/replicaTypesTestColl_shard2_replica_n7/ shard2 2> 621989 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-7-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 create) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 622007 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 622007 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 622021 INFO (indexFetcher-1419-thread-1) [] o.a.s.h.IndexFetcher Updated leaderUrl to http://127.0.0.1:39159/solr/replicaTypesTestColl_shard2_replica_n7/ 2> 622028 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 622030 INFO (indexFetcher-1419-thread-1) [] o.a.s.h.IndexFetcher Leader's generation: 1 2> 622030 INFO (indexFetcher-1419-thread-1) [] o.a.s.h.IndexFetcher Leader's version: 0 2> 622031 INFO (indexFetcher-1419-thread-1) [] o.a.s.h.IndexFetcher Follower's generation: 1 2> 622031 INFO (indexFetcher-1419-thread-1) [] o.a.s.h.IndexFetcher Follower's version: 0 2> 622079 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 622081 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 622081 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 622082 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 622082 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 622082 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 622171 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5303af29-6bf9-481e-8638-bec815fe689a&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=7 2> 622320 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 622320 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 622482 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-6-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.c.ZkController replicaTypesTestColl_shard2_replica_t3 starting background replication from leader 2> 622482 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-6-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:01 2> 622489 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-6-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms 2> 622489 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-6-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 622501 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 622501 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 622502 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 622551 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 622553 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 622553 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 622611 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 622611 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 622616 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 622637 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-7-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.c.ZkController replicaTypesTestColl_shard3_replica_t4 starting background replication from leader 2> 622638 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-7-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:01 2> 622651 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-7-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms 2> 622652 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-7-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 create) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 622655 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 622655 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 622656 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 622688 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 622688 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 622692 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 622701 INFO (indexFetcher-1421-thread-1) [] o.a.s.h.IndexFetcher Last replication failed, so I'll force replication 2> 622702 INFO (indexFetcher-1421-thread-1) [] o.a.s.h.IndexFetcher Updated leaderUrl to http://127.0.0.1:39159/solr/replicaTypesTestColl_shard3_replica_n1/ 2> 622708 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=2 2> 622711 INFO (indexFetcher-1421-thread-1) [] o.a.s.h.IndexFetcher Leader's generation: 1 2> 622712 INFO (indexFetcher-1421-thread-1) [] o.a.s.h.IndexFetcher Leader's version: 0 2> 622712 INFO (indexFetcher-1421-thread-1) [] o.a.s.h.IndexFetcher Follower's generation: 1 2> 622712 INFO (indexFetcher-1421-thread-1) [] o.a.s.h.IndexFetcher Follower's version: 0 2> 622734 INFO (indexFetcher-1417-thread-1) [] o.a.s.h.IndexFetcher Last replication failed, so I'll force replication 2> 622735 INFO (indexFetcher-1417-thread-1) [] o.a.s.h.IndexFetcher Updated leaderUrl to http://127.0.0.1:40391/solr/replicaTypesTestColl_shard1_replica_t2/ 2> 622742 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 622743 INFO (indexFetcher-1417-thread-1) [] o.a.s.h.IndexFetcher Leader's generation: 1 2> 622744 INFO (indexFetcher-1417-thread-1) [] o.a.s.h.IndexFetcher Leader's version: 0 2> 622744 INFO (indexFetcher-1417-thread-1) [] o.a.s.h.IndexFetcher Follower's generation: 1 2> 622744 INFO (indexFetcher-1417-thread-1) [] o.a.s.h.IndexFetcher Follower's version: 0 2> 622884 INFO (indexFetcher-1425-thread-1) [] o.a.s.h.IndexFetcher Updated leaderUrl to http://127.0.0.1:39159/solr/replicaTypesTestColl_shard3_replica_n1/ 2> 622906 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 622909 INFO (indexFetcher-1425-thread-1) [] o.a.s.h.IndexFetcher Leader's generation: 1 2> 622910 INFO (indexFetcher-1425-thread-1) [] o.a.s.h.IndexFetcher Leader's version: 0 2> 622910 INFO (indexFetcher-1425-thread-1) [] o.a.s.h.IndexFetcher Follower's generation: 1 2> 622910 INFO (indexFetcher-1425-thread-1) [] o.a.s.h.IndexFetcher Follower's version: 0 2> 622941 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5303af29-6bf9-481e-8638-bec815fe689a369420693854353 2> 622942 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5303af29-6bf9-481e-8638-bec815fe689a369420693854353&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 622948 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5303af29-6bf9-481e-8638-bec815fe689a369420704978310 2> 622949 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5303af29-6bf9-481e-8638-bec815fe689a369420704978310&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 622956 INFO (qtp218128001-2523) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5303af29-6bf9-481e-8638-bec815fe689a369420715487584 2> 622956 INFO (qtp218128001-2523) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5303af29-6bf9-481e-8638-bec815fe689a369420715487584&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 622964 INFO (qtp218128001-2525) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5303af29-6bf9-481e-8638-bec815fe689a369420726232531 2> 622965 INFO (qtp218128001-2525) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5303af29-6bf9-481e-8638-bec815fe689a369420726232531&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 622973 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5303af29-6bf9-481e-8638-bec815fe689a369420737162482 2> 622974 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5303af29-6bf9-481e-8638-bec815fe689a369420737162482&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 622981 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5303af29-6bf9-481e-8638-bec815fe689a369420748091101 2> 622982 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5303af29-6bf9-481e-8638-bec815fe689a369420748091101&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 622988 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5303af29-6bf9-481e-8638-bec815fe689a369420758822714 2> 622989 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5303af29-6bf9-481e-8638-bec815fe689a369420758822714&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 623020 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000021 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 623036 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 623037 INFO (indexFetcher-1419-thread-1) [] o.a.s.h.IndexFetcher Leader's generation: 1 2> 623038 INFO (indexFetcher-1419-thread-1) [] o.a.s.h.IndexFetcher Leader's version: 0 2> 623038 INFO (indexFetcher-1419-thread-1) [] o.a.s.h.IndexFetcher Follower's generation: 1 2> 623038 INFO (indexFetcher-1419-thread-1) [] o.a.s.h.IndexFetcher Follower's version: 0 2> 623065 INFO (indexFetcher-1423-thread-1) [] o.a.s.h.IndexFetcher Updated leaderUrl to http://127.0.0.1:39159/solr/replicaTypesTestColl_shard2_replica_n7/ 2> 623071 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 623073 INFO (indexFetcher-1423-thread-1) [] o.a.s.h.IndexFetcher Leader's generation: 1 2> 623073 INFO (indexFetcher-1423-thread-1) [] o.a.s.h.IndexFetcher Leader's version: 0 2> 623073 INFO (indexFetcher-1423-thread-1) [] o.a.s.h.IndexFetcher Follower's generation: 1 2> 623073 INFO (indexFetcher-1423-thread-1) [] o.a.s.h.IndexFetcher Follower's version: 0 2> 623086 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 623087 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 623088 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 623088 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 623088 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 623088 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 623183 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5303af29-6bf9-481e-8638-bec815fe689a&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=7 2> 623198 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5303af29-6bf9-481e-8638-bec815fe689a&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=10 2> 623199 INFO (TEST-CloudSolrClientTest.preferReplicaTypesTest-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForActiveCollection: replicaTypesTestColl 2> 623298 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replicaTypesTestColl/terms/shard1 to Terms{values={core_node18=1, core_node11=1}, version=2} for ensureHighestTermsAreNotZero 2> 623299 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replicaTypesTestColl/terms/shard2 to Terms{values={core_node16=1, core_node12=1}, version=2} for ensureHighestTermsAreNotZero 2> 623304 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.c.S.Request webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:39159/solr/replicaTypesTestColl_shard2_replica_n7/&wt=javabin&version=2} status=0 QTime=26 2> 623312 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.c.S.Request webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:40391/solr/replicaTypesTestColl_shard1_replica_t2/&wt=javabin&version=2} status=0 QTime=30 2> 623315 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=replicaTypesTestColl:25&wt=javabin&version=2} status=0 QTime=83 2> 623317 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=replicaTypesTestColl:25&wt=javabin&version=2} status=0 QTime=82 2> 623351 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.c.S.Request webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:39159/solr/replicaTypesTestColl_shard2_replica_n7/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false} status=0 QTime=6 2> 623366 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.c.S.Request webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:39159/solr/replicaTypesTestColl_shard3_replica_n1/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false} status=0 QTime=2 2> 623371 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:40391/solr/replicaTypesTestColl_shard1_replica_t2/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false} status=0 QTime=27 2> 623450 INFO (searcherExecutor-1366-thread-1-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 replicaTypesTestColl shard2 core_node16) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 623453 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:40391/solr/replicaTypesTestColl_shard1_replica_t2/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false} status=0 QTime=120 2> 623600 INFO (searcherExecutor-1368-thread-1-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 replicaTypesTestColl shard1 core_node11) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 623624 INFO (searcherExecutor-1378-thread-1-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 replicaTypesTestColl shard1 core_node18) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 623625 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.c.S.Request webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:40391/solr/replicaTypesTestColl_shard1_replica_t2/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false} status=0 QTime=164 2> 623628 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=replicaTypesTestColl:25&commit=true&wt=javabin&version=2} status=0 QTime=304 2> 623644 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:replicaTypesTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=replicaTypesTestColl&wt=javabin&version=2} status=0 QTime=8 2> 623659 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:replicaTypesTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=replicaTypesTestColl&wt=javabin&version=2} status=0 QTime=8 2> 623679 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=14 2> 623696 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:replicaTypesTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=replicaTypesTestColl&wt=javabin&version=2} status=0 QTime=7 2> 623708 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:replicaTypesTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=replicaTypesTestColl&wt=javabin&version=2} status=0 QTime=7 2> 623717 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 623719 INFO (indexFetcher-1421-thread-1) [] o.a.s.h.IndexFetcher Leader's generation: 1 2> 623720 INFO (indexFetcher-1421-thread-1) [] o.a.s.h.IndexFetcher Leader's version: 0 2> 623720 INFO (indexFetcher-1421-thread-1) [] o.a.s.h.IndexFetcher Follower's generation: 1 2> 623720 INFO (indexFetcher-1421-thread-1) [] o.a.s.h.IndexFetcher Follower's version: 0 2> 623746 INFO (qtp218128001-2525) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard3 r:core_node14 x:replicaTypesTestColl_shard3_replica_p5] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=replicaTypesTestColl:25&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shards.info=true&shard.url=http://127.0.0.1:44127/solr/replicaTypesTestColl_shard3_replica_p5/|http://127.0.0.1:40391/solr/replicaTypesTestColl_shard3_replica_t4/|http://127.0.0.1:39159/solr/replicaTypesTestColl_shard3_replica_n1/&rid=127.0.0.1-1371&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335591174&isShard=true&shards.preference=replica.type:PULL&wt=javabin} hits=0 status=0 QTime=2 2> 623746 INFO (qtp218128001-2523) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard2 r:core_node17 x:replicaTypesTestColl_shard2_replica_p8] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=replicaTypesTestColl:25&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shards.info=true&shard.url=http://127.0.0.1:44127/solr/replicaTypesTestColl_shard2_replica_p8/|http://127.0.0.1:39159/solr/replicaTypesTestColl_shard2_replica_n7/|http://127.0.0.1:40391/solr/replicaTypesTestColl_shard2_replica_t3/&rid=127.0.0.1-1371&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335591174&isShard=true&shards.preference=replica.type:PULL&wt=javabin} hits=0 status=0 QTime=2 2> 623746 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard1 r:core_node15 x:replicaTypesTestColl_shard1_replica_p6] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=replicaTypesTestColl:25&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shards.info=true&shard.url=http://127.0.0.1:44127/solr/replicaTypesTestColl_shard1_replica_p6/|http://127.0.0.1:40391/solr/replicaTypesTestColl_shard1_replica_t2/|http://127.0.0.1:39159/solr/replicaTypesTestColl_shard1_replica_n9/&rid=127.0.0.1-1371&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335591174&isShard=true&shards.preference=replica.type:PULL&wt=javabin} hits=0 status=0 QTime=2 2> 623752 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=2 2> 623755 INFO (indexFetcher-1417-thread-1) [] o.a.s.h.IndexFetcher Leader's generation: 2 2> 623755 INFO (indexFetcher-1417-thread-1) [] o.a.s.h.IndexFetcher Leader's version: 1696335590908 2> 623756 INFO (indexFetcher-1417-thread-1) [] o.a.s.h.IndexFetcher Follower's generation: 1 2> 623756 INFO (indexFetcher-1417-thread-1) [] o.a.s.h.IndexFetcher Follower's version: 0 2> 623756 INFO (indexFetcher-1417-thread-1) [] o.a.s.h.IndexFetcher Starting replication process 2> 623758 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard2 r:core_node17 x:replicaTypesTestColl_shard2_replica_p8] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=replicaTypesTestColl:25&shards.info=true&shards.preference=replica.type:PULL&wt=javabin&version=2} rid=127.0.0.1-1371 hits=0 status=0 QTime=38 2> 623768 INFO (TEST-CloudSolrClientTest.preferReplicaTypesTest-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Shards giving the response: [http://127.0.0.1:44127/solr/replicaTypesTestColl_shard1_replica_p6, http://127.0.0.1:44127/solr/replicaTypesTestColl_shard2_replica_p8, http://127.0.0.1:44127/solr/replicaTypesTestColl_shard3_replica_p5] 2> 623780 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:replicaTypesTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=replicaTypesTestColl&wt=javabin&version=2} status=0 QTime=8 2> 623788 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&wt=javabin&version=2&command=filelist} status=0 QTime=27 2> 623793 INFO (indexFetcher-1417-thread-1) [] o.a.s.h.IndexFetcher Number of files in latest index in leader: 34 2> 623798 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:replicaTypesTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=replicaTypesTestColl&wt=javabin&version=2} status=0 QTime=12 2> 623821 INFO (indexFetcher-1417-thread-1) [] o.a.s.h.IndexFetcher Starting download (fullCopy=false) to MockDirectoryWrapper(ByteBuffersDirectory@77e8dc6c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@ee7e77b) 2> 623821 INFO (indexFetcher-1417-thread-1) [] o.a.s.h.IndexFetcher tmpIndexDir_type : class org.apache.lucene.tests.store.MockDirectoryWrapper , ByteBuffersDirectory@77e8dc6c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@ee7e77b 2> 623828 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=1 2> 623830 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=25 2> 623848 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_1.dvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 623862 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 623867 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:replicaTypesTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=replicaTypesTestColl&wt=javabin&version=2} status=0 QTime=27 2> 623890 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 623893 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:replicaTypesTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=replicaTypesTestColl&wt=javabin&version=2} status=0 QTime=10 2> 623916 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_BlockTreeOrds_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=6 2> 623916 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=2 2> 623922 INFO (indexFetcher-1425-thread-1) [] o.a.s.h.IndexFetcher Leader's generation: 1 2> 623923 INFO (indexFetcher-1425-thread-1) [] o.a.s.h.IndexFetcher Leader's version: 0 2> 623923 INFO (indexFetcher-1425-thread-1) [] o.a.s.h.IndexFetcher Follower's generation: 1 2> 623924 INFO (indexFetcher-1425-thread-1) [] o.a.s.h.IndexFetcher Follower's version: 0 2> 623928 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard2 r:core_node17 x:replicaTypesTestColl_shard2_replica_p8] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=replicaTypesTestColl:25&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shards.info=true&shard.url=http://127.0.0.1:44127/solr/replicaTypesTestColl_shard2_replica_p8/|http://127.0.0.1:40391/solr/replicaTypesTestColl_shard2_replica_t3/|http://127.0.0.1:39159/solr/replicaTypesTestColl_shard2_replica_n7/&rid=127.0.0.1-1372&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335591369&isShard=true&shards.preference=replica.type:PULL,replica.type:TLOG&wt=javabin} hits=0 status=0 QTime=3 2> 623929 INFO (qtp218128001-2523) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard3 r:core_node14 x:replicaTypesTestColl_shard3_replica_p5] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=replicaTypesTestColl:25&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shards.info=true&shard.url=http://127.0.0.1:44127/solr/replicaTypesTestColl_shard3_replica_p5/|http://127.0.0.1:40391/solr/replicaTypesTestColl_shard3_replica_t4/|http://127.0.0.1:39159/solr/replicaTypesTestColl_shard3_replica_n1/&rid=127.0.0.1-1372&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335591369&isShard=true&shards.preference=replica.type:PULL,replica.type:TLOG&wt=javabin} hits=0 status=0 QTime=2 2> 623938 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_BlockTreeOrds_0.tipo&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 623940 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard1 r:core_node15 x:replicaTypesTestColl_shard1_replica_p6] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=replicaTypesTestColl:25&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shards.info=true&shard.url=http://127.0.0.1:44127/solr/replicaTypesTestColl_shard1_replica_p6/|http://127.0.0.1:40391/solr/replicaTypesTestColl_shard1_replica_t2/|http://127.0.0.1:39159/solr/replicaTypesTestColl_shard1_replica_n9/&rid=127.0.0.1-1372&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335591369&isShard=true&shards.preference=replica.type:PULL,replica.type:TLOG&wt=javabin} hits=0 status=0 QTime=2 2> 623943 INFO (qtp218128001-2528) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard2 r:core_node17 x:replicaTypesTestColl_shard2_replica_p8] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=replicaTypesTestColl:25&shards.info=true&shards.preference=replica.type:PULL,replica.type:TLOG&wt=javabin&version=2} rid=127.0.0.1-1372 hits=0 status=0 QTime=27 2> 623948 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 623951 INFO (TEST-CloudSolrClientTest.preferReplicaTypesTest-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Shards giving the response: [http://127.0.0.1:44127/solr/replicaTypesTestColl_shard2_replica_p8, http://127.0.0.1:44127/solr/replicaTypesTestColl_shard3_replica_p5, http://127.0.0.1:44127/solr/replicaTypesTestColl_shard1_replica_p6] 2> 623966 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.dvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 623967 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=replicaTypesTestColl:25&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shards.info=true&shard.url=http://127.0.0.1:40391/solr/replicaTypesTestColl_shard2_replica_t3/|http://127.0.0.1:39159/solr/replicaTypesTestColl_shard2_replica_n7/|http://127.0.0.1:44127/solr/replicaTypesTestColl_shard2_replica_p8/&rid=127.0.0.1-1373&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335591408&isShard=true&shards.preference=replica.type:TLOG&wt=javabin} hits=0 status=0 QTime=2 2> 623971 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=replicaTypesTestColl:25&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shards.info=true&shard.url=http://127.0.0.1:40391/solr/replicaTypesTestColl_shard3_replica_t4/|http://127.0.0.1:39159/solr/replicaTypesTestColl_shard3_replica_n1/|http://127.0.0.1:44127/solr/replicaTypesTestColl_shard3_replica_p5/&rid=127.0.0.1-1373&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335591408&isShard=true&shards.preference=replica.type:TLOG&wt=javabin} hits=0 status=0 QTime=2 2> 623978 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=replicaTypesTestColl:25&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shards.info=true&shard.url=http://127.0.0.1:40391/solr/replicaTypesTestColl_shard1_replica_t2/|http://127.0.0.1:44127/solr/replicaTypesTestColl_shard1_replica_p6/|http://127.0.0.1:39159/solr/replicaTypesTestColl_shard1_replica_n9/&rid=127.0.0.1-1373&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335591408&isShard=true&shards.preference=replica.type:TLOG&wt=javabin} hits=1 status=0 QTime=4 2> 623982 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=3 2> 623988 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=replicaTypesTestColl:25&shards.purpose=64&shards.info=true&shard.url=http://127.0.0.1:40391/solr/replicaTypesTestColl_shard1_replica_t2/|http://127.0.0.1:44127/solr/replicaTypesTestColl_shard1_replica_p6/|http://127.0.0.1:39159/solr/replicaTypesTestColl_shard1_replica_n9/&rid=127.0.0.1-1373&version=2&q=*:*&omitHeader=false&NOW=1696335591408&ids=0&isShard=true&shards.preference=replica.type:TLOG&wt=javabin} status=0 QTime=2 2> 623992 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=replicaTypesTestColl:25&shards.info=true&shards.preference=replica.type:TLOG&wt=javabin&version=2} rid=127.0.0.1-1373 hits=1 status=0 QTime=37 2> 623993 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=4 2> 623999 INFO (TEST-CloudSolrClientTest.preferReplicaTypesTest-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Shards giving the response: [http://127.0.0.1:40391/solr/replicaTypesTestColl_shard2_replica_t3, http://127.0.0.1:40391/solr/replicaTypesTestColl_shard3_replica_t4, http://127.0.0.1:40391/solr/replicaTypesTestColl_shard1_replica_t2] 2> 624010 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=3 2> 624016 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:replicaTypesTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=replicaTypesTestColl&wt=javabin&version=2} status=0 QTime=14 2> 624025 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.tmd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624034 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:replicaTypesTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=replicaTypesTestColl&wt=javabin&version=2} status=0 QTime=13 2> 624041 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.kdm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624044 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=2 2> 624045 INFO (indexFetcher-1419-thread-1) [] o.a.s.h.IndexFetcher Leader's generation: 2 2> 624046 INFO (indexFetcher-1419-thread-1) [] o.a.s.h.IndexFetcher Leader's version: 1696335590792 2> 624046 INFO (indexFetcher-1419-thread-1) [] o.a.s.h.IndexFetcher Follower's generation: 1 2> 624046 INFO (indexFetcher-1419-thread-1) [] o.a.s.h.IndexFetcher Follower's version: 0 2> 624046 INFO (indexFetcher-1419-thread-1) [] o.a.s.h.IndexFetcher Starting replication process 2> 624055 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_BlockTreeOrds_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=1 2> 624059 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=20 2> 624069 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624075 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&wt=javabin&version=2&command=filelist} status=0 QTime=24 2> 624079 INFO (indexFetcher-1419-thread-1) [] o.a.s.h.IndexFetcher Number of files in latest index in leader: 34 2> 624080 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=4 2> 624084 INFO (indexFetcher-1423-thread-1) [] o.a.s.h.IndexFetcher Leader's generation: 2 2> 624084 INFO (indexFetcher-1423-thread-1) [] o.a.s.h.IndexFetcher Leader's version: 1696335590792 2> 624084 INFO (indexFetcher-1423-thread-1) [] o.a.s.h.IndexFetcher Follower's generation: 1 2> 624084 INFO (indexFetcher-1423-thread-1) [] o.a.s.h.IndexFetcher Follower's version: 0 2> 624085 INFO (indexFetcher-1423-thread-1) [] o.a.s.h.IndexFetcher Starting replication process 2> 624085 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:replicaTypesTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=replicaTypesTestColl&wt=javabin&version=2} status=0 QTime=16 2> 624093 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 624095 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 624096 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 624096 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 624096 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 624096 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 624107 INFO (indexFetcher-1419-thread-1) [] o.a.s.h.IndexFetcher Starting download (fullCopy=false) to MockDirectoryWrapper(ByteBuffersDirectory@63c77e8f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@33c40e51) 2> 624107 INFO (indexFetcher-1419-thread-1) [] o.a.s.h.IndexFetcher tmpIndexDir_type : class org.apache.lucene.tests.store.MockDirectoryWrapper , ByteBuffersDirectory@63c77e8f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@33c40e51 2> 624112 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624117 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=3 2> 624122 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:replicaTypesTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=replicaTypesTestColl&wt=javabin&version=2} status=0 QTime=32 2> 624126 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000021 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 624134 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.kdi&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624138 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&wt=javabin&version=2&command=filelist} status=0 QTime=50 2> 624142 INFO (indexFetcher-1423-thread-1) [] o.a.s.h.IndexFetcher Number of files in latest index in leader: 34 2> 624148 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=replicaTypesTestColl:25&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shards.info=true&shard.url=http://127.0.0.1:40391/solr/replicaTypesTestColl_shard2_replica_t3/|http://127.0.0.1:44127/solr/replicaTypesTestColl_shard2_replica_p8/|http://127.0.0.1:39159/solr/replicaTypesTestColl_shard2_replica_n7/&rid=127.0.0.1-1374&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335591585&isShard=true&shards.preference=replica.type:TLOG,replica.type:PULL&wt=javabin} hits=0 status=0 QTime=2 2> 624158 INFO (indexFetcher-1423-thread-1) [] o.a.s.h.IndexFetcher Starting download (fullCopy=false) to MockDirectoryWrapper(ByteBuffersDirectory@37f0446f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1bf16408) 2> 624160 INFO (indexFetcher-1423-thread-1) [] o.a.s.h.IndexFetcher tmpIndexDir_type : class org.apache.lucene.tests.store.MockDirectoryWrapper , ByteBuffersDirectory@37f0446f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1bf16408 2> 624167 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=replicaTypesTestColl:25&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shards.info=true&shard.url=http://127.0.0.1:40391/solr/replicaTypesTestColl_shard1_replica_t2/|http://127.0.0.1:44127/solr/replicaTypesTestColl_shard1_replica_p6/|http://127.0.0.1:39159/solr/replicaTypesTestColl_shard1_replica_n9/&rid=127.0.0.1-1374&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335591585&isShard=true&shards.preference=replica.type:TLOG,replica.type:PULL&wt=javabin} hits=1 status=0 QTime=3 2> 624171 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard3 r:core_node13 x:replicaTypesTestColl_shard3_replica_t4] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=replicaTypesTestColl:25&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shards.info=true&shard.url=http://127.0.0.1:40391/solr/replicaTypesTestColl_shard3_replica_t4/|http://127.0.0.1:44127/solr/replicaTypesTestColl_shard3_replica_p5/|http://127.0.0.1:39159/solr/replicaTypesTestColl_shard3_replica_n1/&rid=127.0.0.1-1374&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335591585&isShard=true&shards.preference=replica.type:TLOG,replica.type:PULL&wt=javabin} hits=0 status=0 QTime=8 2> 624171 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=7 2> 624173 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_1.dvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=11 2> 624186 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624186 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=replicaTypesTestColl:25&shards.purpose=64&shards.info=true&shard.url=http://127.0.0.1:40391/solr/replicaTypesTestColl_shard1_replica_t2/|http://127.0.0.1:44127/solr/replicaTypesTestColl_shard1_replica_p6/|http://127.0.0.1:39159/solr/replicaTypesTestColl_shard1_replica_n9/&rid=127.0.0.1-1374&version=2&q=*:*&omitHeader=false&NOW=1696335591585&ids=0&isShard=true&shards.preference=replica.type:TLOG,replica.type:PULL&wt=javabin} status=0 QTime=2 2> 624190 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=9 2> 624191 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=3 2> 624194 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard2 r:core_node12 x:replicaTypesTestColl_shard2_replica_t3] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=replicaTypesTestColl:25&shards.info=true&shards.preference=replica.type:TLOG,replica.type:PULL&wt=javabin&version=2} rid=127.0.0.1-1374 hits=1 status=0 QTime=63 2> 624201 INFO (TEST-CloudSolrClientTest.preferReplicaTypesTest-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Shards giving the response: [http://127.0.0.1:40391/solr/replicaTypesTestColl_shard2_replica_t3, http://127.0.0.1:40391/solr/replicaTypesTestColl_shard1_replica_t2, http://127.0.0.1:40391/solr/replicaTypesTestColl_shard3_replica_t4] 2> 624203 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=3 2> 624207 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_1.dvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624207 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624218 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:replicaTypesTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=replicaTypesTestColl&wt=javabin&version=2} status=0 QTime=14 2> 624224 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624229 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdx&checksum=true&wt=filestream&command=filecontent} status=0 QTime=6 2> 624234 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_BlockTreeOrds_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=5 2> 624240 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:replicaTypesTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=replicaTypesTestColl&wt=javabin&version=2} status=0 QTime=16 2> 624248 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624248 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_BlockTreeOrds_0.tio&checksum=true&wt=filestream&command=filecontent} status=0 QTime=5 2> 624255 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_BlockTreeOrds_0.tipo&checksum=true&wt=filestream&command=filecontent} status=0 QTime=8 2> 624266 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_BlockTreeOrds_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=8 2> 624266 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=21 2> 624273 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624277 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdt&checksum=true&wt=filestream&command=filecontent} status=0 QTime=4 2> 624280 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_BlockTreeOrds_0.tipo&checksum=true&wt=filestream&command=filecontent} status=0 QTime=3 2> 624287 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.dvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624293 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.tmd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624295 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=3 2> 624300 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:replicaTypesTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=replicaTypesTestColl&wt=javabin&version=2} status=0 QTime=23 2> 624304 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.dvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624306 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624318 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624319 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.dvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=3 2> 624319 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=3 2> 624329 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.tmd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624330 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:replicaTypesTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=replicaTypesTestColl&wt=javabin&version=2} status=0 QTime=24 2> 624338 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=3 2> 624340 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=9 2> 624340 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.kdd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=3 2> 624353 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=replicaTypesTestColl:25&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shards.info=true&shard.url=http://127.0.0.1:39159/solr/replicaTypesTestColl_shard2_replica_n7/|http://127.0.0.1:40391/solr/replicaTypesTestColl_shard2_replica_t3/|http://127.0.0.1:44127/solr/replicaTypesTestColl_shard2_replica_p8/&rid=127.0.0.1-1375&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335591791&isShard=true&shards.preference=replica.type:NRT&wt=javabin} hits=2 status=0 QTime=2 2> 624354 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624782 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.tmd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624783 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=replicaTypesTestColl:25&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shards.info=true&shard.url=http://127.0.0.1:39159/solr/replicaTypesTestColl_shard3_replica_n1/|http://127.0.0.1:44127/solr/replicaTypesTestColl_shard3_replica_p5/|http://127.0.0.1:40391/solr/replicaTypesTestColl_shard3_replica_t4/&rid=127.0.0.1-1375&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335591791&isShard=true&shards.preference=replica.type:NRT&wt=javabin} hits=0 status=0 QTime=3 2> 624790 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=6 2> 624792 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=replicaTypesTestColl:25&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shards.info=true&shard.url=http://127.0.0.1:39159/solr/replicaTypesTestColl_shard1_replica_n9/|http://127.0.0.1:44127/solr/replicaTypesTestColl_shard1_replica_p6/|http://127.0.0.1:40391/solr/replicaTypesTestColl_shard1_replica_t2/&rid=127.0.0.1-1375&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335591791&isShard=true&shards.preference=replica.type:NRT&wt=javabin} hits=1 status=0 QTime=5 2> 624794 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=6 2> 624795 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=6 2> 624797 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.kdm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624797 INFO (indexFetcher-1421-thread-1) [] o.a.s.h.IndexFetcher Leader's generation: 1 2> 624797 INFO (indexFetcher-1421-thread-1) [] o.a.s.h.IndexFetcher Leader's version: 0 2> 624797 INFO (indexFetcher-1421-thread-1) [] o.a.s.h.IndexFetcher Follower's generation: 1 2> 624798 INFO (indexFetcher-1421-thread-1) [] o.a.s.h.IndexFetcher Follower's version: 0 2> 624801 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_1.dvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624809 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=replicaTypesTestColl:25&shards.purpose=64&shards.info=true&shard.url=http://127.0.0.1:39159/solr/replicaTypesTestColl_shard1_replica_n9/|http://127.0.0.1:44127/solr/replicaTypesTestColl_shard1_replica_p6/|http://127.0.0.1:40391/solr/replicaTypesTestColl_shard1_replica_t2/&rid=127.0.0.1-1375&version=2&q=*:*&omitHeader=false&NOW=1696335591791&ids=0&isShard=true&shards.preference=replica.type:NRT&wt=javabin} status=0 QTime=5 2> 624812 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.tmd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=3 2> 624813 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624821 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_BlockTreeOrds_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624823 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=replicaTypesTestColl:25&shards.purpose=64&shards.info=true&shard.url=http://127.0.0.1:39159/solr/replicaTypesTestColl_shard2_replica_n7/|http://127.0.0.1:40391/solr/replicaTypesTestColl_shard2_replica_t3/|http://127.0.0.1:44127/solr/replicaTypesTestColl_shard2_replica_p8/&rid=127.0.0.1-1375&version=2&q=*:*&omitHeader=false&NOW=1696335591791&ids=2,3&isShard=true&shards.preference=replica.type:NRT&wt=javabin} status=0 QTime=12 2> 624826 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=1 2> 624833 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=replicaTypesTestColl:25&shards.info=true&shards.preference=replica.type:NRT&wt=javabin&version=2} rid=127.0.0.1-1375 hits=3 status=0 QTime=496 2> 624833 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.kdm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=10 2> 624836 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fnm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624839 INFO (TEST-CloudSolrClientTest.preferReplicaTypesTest-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Shards giving the response: [http://127.0.0.1:39159/solr/replicaTypesTestColl_shard2_replica_n7, http://127.0.0.1:39159/solr/replicaTypesTestColl_shard3_replica_n1, http://127.0.0.1:39159/solr/replicaTypesTestColl_shard1_replica_n9] 2> 624849 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:replicaTypesTestColl s:shard1 r:core_node11 x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=segments_2&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624849 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624851 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_BlockTreeOrds_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624860 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624863 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624871 INFO (indexFetcher-1417-thread-1) [] o.a.s.h.IndexFetcher Bytes downloaded: 5866, Bytes skipped downloading: 0 2> 624871 INFO (indexFetcher-1417-thread-1) [] o.a.s.h.IndexFetcher Total time taken for download (fullCopy=false,bytesDownloaded=5866) : 1 secs (5866 bytes/sec) to MockDirectoryWrapper(ByteBuffersDirectory@77e8dc6c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@ee7e77b) 2> 624890 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.kdi&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624890 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624893 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:replicaTypesTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=replicaTypesTestColl&wt=javabin&version=2} status=0 QTime=28 2> 624908 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.kdi&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624910 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624913 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:replicaTypesTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=replicaTypesTestColl&wt=javabin&version=2} status=0 QTime=12 2> 624922 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624933 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=3 2> 624939 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 624940 INFO (indexFetcher-1425-thread-1) [] o.a.s.h.IndexFetcher Leader's generation: 1 2> 624940 INFO (indexFetcher-1425-thread-1) [] o.a.s.h.IndexFetcher Leader's version: 0 2> 624940 INFO (indexFetcher-1425-thread-1) [] o.a.s.h.IndexFetcher Follower's generation: 1 2> 624941 INFO (indexFetcher-1425-thread-1) [] o.a.s.h.IndexFetcher Follower's version: 0 2> 624944 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624950 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624950 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=28 2> 624956 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624964 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdx&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624976 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdx&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 624982 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_BlockTreeOrds_0.tio&checksum=true&wt=filestream&command=filecontent} status=0 QTime=3 2> 624983 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:replicaTypesTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=replicaTypesTestColl&wt=javabin&version=2} status=0 QTime=14 2> 624994 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_BlockTreeOrds_0.tio&checksum=true&wt=filestream&command=filecontent} status=0 QTime=3 2> 624996 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdt&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 625006 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.tmd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 625030 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:replicaTypesTestColl] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=replicaTypesTestColl&wt=javabin&version=2} status=0 QTime=39 2> 625033 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdt&checksum=true&wt=filestream&command=filecontent} status=0 QTime=10 2> 625039 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.dvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=3 2> 625046 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.tmd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 625049 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 625051 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=replicaTypesTestColl:25&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shards.info=true&shard.url=http://127.0.0.1:39159/solr/replicaTypesTestColl_shard2_replica_n7/|http://127.0.0.1:44127/solr/replicaTypesTestColl_shard2_replica_p8/|http://127.0.0.1:40391/solr/replicaTypesTestColl_shard2_replica_t3/&rid=127.0.0.1-1376&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335592493&isShard=true&shards.preference=replica.type:NRT,replica.type:PULL&wt=javabin} hits=2 status=0 QTime=2 2> 625054 INFO (indexFetcher-1417-thread-1) [] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used. 2> 625066 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.dvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 625066 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard3 r:core_node10 x:replicaTypesTestColl_shard3_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=replicaTypesTestColl:25&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shards.info=true&shard.url=http://127.0.0.1:39159/solr/replicaTypesTestColl_shard3_replica_n1/|http://127.0.0.1:44127/solr/replicaTypesTestColl_shard3_replica_p5/|http://127.0.0.1:40391/solr/replicaTypesTestColl_shard3_replica_t4/&rid=127.0.0.1-1376&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335592493&isShard=true&shards.preference=replica.type:NRT,replica.type:PULL&wt=javabin} hits=0 status=0 QTime=2 2> 625073 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.tmd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 625073 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=replicaTypesTestColl:25&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shards.info=true&shard.url=http://127.0.0.1:39159/solr/replicaTypesTestColl_shard1_replica_n9/|http://127.0.0.1:44127/solr/replicaTypesTestColl_shard1_replica_p6/|http://127.0.0.1:40391/solr/replicaTypesTestColl_shard1_replica_t2/&rid=127.0.0.1-1376&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335592493&isShard=true&shards.preference=replica.type:NRT,replica.type:PULL&wt=javabin} hits=1 status=0 QTime=2 2> 625083 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.kdd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 625085 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=3 2> 625092 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard1 r:core_node18 x:replicaTypesTestColl_shard1_replica_n9] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=replicaTypesTestColl:25&shards.purpose=64&shards.info=true&shard.url=http://127.0.0.1:39159/solr/replicaTypesTestColl_shard1_replica_n9/|http://127.0.0.1:44127/solr/replicaTypesTestColl_shard1_replica_p6/|http://127.0.0.1:40391/solr/replicaTypesTestColl_shard1_replica_t2/&rid=127.0.0.1-1376&version=2&q=*:*&omitHeader=false&NOW=1696335592493&ids=0&isShard=true&shards.preference=replica.type:NRT,replica.type:PULL&wt=javabin} status=0 QTime=2 2> 625102 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 625102 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=replicaTypesTestColl:25&shards.purpose=64&shards.info=true&shard.url=http://127.0.0.1:39159/solr/replicaTypesTestColl_shard2_replica_n7/|http://127.0.0.1:44127/solr/replicaTypesTestColl_shard2_replica_p8/|http://127.0.0.1:40391/solr/replicaTypesTestColl_shard2_replica_t3/&rid=127.0.0.1-1376&version=2&q=*:*&omitHeader=false&NOW=1696335592493&ids=2,3&isShard=true&shards.preference=replica.type:NRT,replica.type:PULL&wt=javabin} status=0 QTime=10 2> 625104 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=3 2> 625107 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 625107 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 625107 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 625108 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 625108 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 625113 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.tmd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 625122 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=replicaTypesTestColl:25&shards.info=true&shards.preference=replica.type:NRT,replica.type:PULL&wt=javabin&version=2} rid=127.0.0.1-1376 hits=3 status=0 QTime=82 2> 625126 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_1.dvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=3 2> 625128 INFO (TEST-CloudSolrClientTest.preferReplicaTypesTest-seed#[9CAA27B60427D55F]) [] o.a.s.c.s.i.CloudSolrClientTest Shards giving the response: [http://127.0.0.1:39159/solr/replicaTypesTestColl_shard2_replica_n7, http://127.0.0.1:39159/solr/replicaTypesTestColl_shard3_replica_n1, http://127.0.0.1:39159/solr/replicaTypesTestColl_shard1_replica_n9] 2> 625137 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 625146 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.kdd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 625152 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=3 2> 625158 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=bb78f355-2b3f-4f1f-862b-fef25a581d53&name=replicaTypesTestColl&action=DELETE&wt=javabin&version=2} status=0 QTime=25 2> 625178 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=11 2> 625185 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fnm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 625196 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=bb78f355-2b3f-4f1f-862b-fef25a581d53&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=32 2> 625201 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene90_1.dvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=6 2> 625201 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=segments_2&checksum=true&wt=filestream&command=filecontent} status=0 QTime=4 2> 625205 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CollectionHandlingUtils Executing Collection Cmd=action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true, asyncId=bb78f355-2b3f-4f1f-862b-fef25a581d53 2> 625208 INFO (indexFetcher-1419-thread-1) [] o.a.s.h.IndexFetcher Bytes downloaded: 5940, Bytes skipped downloading: 0 2> 625217 INFO (indexFetcher-1419-thread-1) [] o.a.s.h.IndexFetcher Total time taken for download (fullCopy=false,bytesDownloaded=5940) : 1 secs (5940 bytes/sec) to MockDirectoryWrapper(ByteBuffersDirectory@63c77e8f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@33c40e51) 2> 625218 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard1_replica_t2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=replicaTypesTestColl_shard1_replica_t2&async=bb78f355-2b3f-4f1f-862b-fef25a581d53369427618582558&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=9 2> 625219 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr x:replicaTypesTestColl_shard1_replica_n9] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=replicaTypesTestColl_shard1_replica_n9&async=bb78f355-2b3f-4f1f-862b-fef25a581d53369427622339323&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=4 2> 625216 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 625222 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-8-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 bb78f355-2b3f-4f1f-862b-fef25a581d53369427618582558 unload) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard1_replica_t2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replicaTypesTestColl.shard1.replica_t2 tag=null 2> 625223 INFO (qtp218128001-2525) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard1_replica_p6] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=replicaTypesTestColl_shard1_replica_p6&async=bb78f355-2b3f-4f1f-862b-fef25a581d53369427620487941&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=13 2> 625233 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-9-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 bb78f355-2b3f-4f1f-862b-fef25a581d53369427622339323 unload) [n:127.0.0.1:39159_solr x:replicaTypesTestColl_shard1_replica_n9] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replicaTypesTestColl.shard1.replica_n9 tag=null 2> 625235 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-6-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard1_replica_p6 bb78f355-2b3f-4f1f-862b-fef25a581d53369427620487941 unload) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard1_replica_p6] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replicaTypesTestColl.shard1.replica_p6 tag=null 2> 625237 INFO (searcherExecutor-1372-thread-1-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard1_replica_p6 replicaTypesTestColl shard1 core_node15) [n:127.0.0.1:44127_solr c:replicaTypesTestColl s:shard1 r:core_node15 x:replicaTypesTestColl_shard1_replica_p6] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 625241 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard3_replica_t4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=replicaTypesTestColl_shard3_replica_t4&async=bb78f355-2b3f-4f1f-862b-fef25a581d53369427641790830&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=7 2> 625241 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard2_replica_t3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=replicaTypesTestColl_shard2_replica_t3&async=bb78f355-2b3f-4f1f-862b-fef25a581d53369427624301374&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=24 2> 625246 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=15 2> 625259 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:replicaTypesTestColl s:shard2 r:core_node16 x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fnm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=2 2> 625260 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr x:replicaTypesTestColl_shard2_replica_n7] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=replicaTypesTestColl_shard2_replica_n7&async=bb78f355-2b3f-4f1f-862b-fef25a581d53369427626122088&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=44 2> 625270 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard3_replica_p5] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=replicaTypesTestColl_shard3_replica_p5&async=bb78f355-2b3f-4f1f-862b-fef25a581d53369427643443873&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=32 2> 625275 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr x:replicaTypesTestColl_shard3_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=replicaTypesTestColl_shard3_replica_n1&async=bb78f355-2b3f-4f1f-862b-fef25a581d53369427640181788&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=44 2> 625286 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard2_replica_p8] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=replicaTypesTestColl_shard2_replica_p8&async=bb78f355-2b3f-4f1f-862b-fef25a581d53369427627687796&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=55 2> 625296 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : bb78f355-2b3f-4f1f-862b-fef25a581d53369427618582558 2> 625297 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=bb78f355-2b3f-4f1f-862b-fef25a581d53369427618582558&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=3 2> 625400 ERROR (indexFetcher-1423-thread-1) [] o.a.s.h.IndexFetcher Error fetching file, doing one retry... 2> => org.apache.solr.common.SolrException: Unable to download segments_2 completely. Downloaded 0!=221 2> at org.apache.solr.handler.IndexFetcher$FileFetcher.cleanup(IndexFetcher.java:1949) 2> org.apache.solr.common.SolrException: Unable to download segments_2 completely. Downloaded 0!=221 2> at org.apache.solr.handler.IndexFetcher$FileFetcher.cleanup(IndexFetcher.java:1949) ~[solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.IndexFetcher$FileFetcher.fetch(IndexFetcher.java:1812) ~[solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.IndexFetcher$FileFetcher.fetchFile(IndexFetcher.java:1782) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.IndexFetcher.downloadIndexFiles(IndexFetcher.java:1203) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:687) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:424) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:485) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1308) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?] 2> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> 625433 ERROR (indexFetcher-1423-thread-1) [] o.a.s.h.IndexFetcher Error deleting file: segments_2 2> => java.nio.file.NoSuchFileException: segments_2 2> at org.apache.lucene.store.ByteBuffersDirectory.deleteFile(ByteBuffersDirectory.java:168) 2> java.nio.file.NoSuchFileException: segments_2 2> at org.apache.lucene.store.ByteBuffersDirectory.deleteFile(ByteBuffersDirectory.java:168) ~[lucene-core-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39] 2> at org.apache.lucene.tests.store.MockDirectoryWrapper.deleteFile(MockDirectoryWrapper.java:651) ~[lucene-test-framework-9.4.2.jar:9.4.2 858d9b437047a577fa9457089afff43eefa461db - jpountz - 2022-11-17 12:56:39] 2> at org.apache.solr.handler.IndexFetcher$DirectoryFile.delete(IndexFetcher.java:2040) ~[solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.IndexFetcher$FileFetcher.cleanup(IndexFetcher.java:1943) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.IndexFetcher$FileFetcher.fetch(IndexFetcher.java:1812) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.IndexFetcher$FileFetcher.fetchFile(IndexFetcher.java:1787) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.IndexFetcher.downloadIndexFiles(IndexFetcher.java:1203) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:687) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:424) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:485) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1308) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?] 2> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> 625460 INFO (indexFetcher-1419-thread-1) [] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used. 2> 625464 INFO (indexFetcher-1423-thread-1) [] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used. 2> 625469 ERROR (indexFetcher-1423-thread-1) [] o.a.s.h.ReplicationHandler Index fetch failed 2> => org.apache.solr.common.SolrException: Unable to download segments_2 completely. Downloaded 0!=221 2> at org.apache.solr.handler.IndexFetcher$FileFetcher.cleanup(IndexFetcher.java:1949) 2> org.apache.solr.common.SolrException: Unable to download segments_2 completely. Downloaded 0!=221 2> at org.apache.solr.handler.IndexFetcher$FileFetcher.cleanup(IndexFetcher.java:1949) ~[solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.IndexFetcher$FileFetcher.fetch(IndexFetcher.java:1812) ~[solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.IndexFetcher$FileFetcher.fetchFile(IndexFetcher.java:1787) ~[solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.IndexFetcher.downloadIndexFiles(IndexFetcher.java:1203) ~[solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:687) ~[solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:424) ~[solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:485) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1308) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?] 2> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> 625810 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-10-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 bb78f355-2b3f-4f1f-862b-fef25a581d53369427626122088 unload) [n:127.0.0.1:39159_solr x:replicaTypesTestColl_shard2_replica_n7] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replicaTypesTestColl.shard2.replica_n7 tag=null 2> 625810 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-9-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 bb78f355-2b3f-4f1f-862b-fef25a581d53369427622339323 unload) [n:127.0.0.1:39159_solr x:replicaTypesTestColl_shard1_replica_n9] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@6e3f5983 replicaTypesTestColl_shard1_replica_n9 2> 625823 WARN (indexFetcher-1421-thread-1) [] o.a.s.h.IndexFetcher Leader at: http://127.0.0.1:39159/solr/replicaTypesTestColl_shard3_replica_n1/ is not available. Index fetch failed by exception: org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39159/solr/replicaTypesTestColl_shard3_replica_n1: Expected mime type in [application/octet-stream, application/vnd.apache.solr.javabin] but got text/html. 2> 2> 2> Error 404 Can not find: /solr/replicaTypesTestColl_shard3_replica_n1/replication 2> 2>

HTTP ERROR 404 Can not find: /solr/replicaTypesTestColl_shard3_replica_n1/replication

2> 2> 2> 2> 2> 2>
URI:/solr/replicaTypesTestColl_shard3_replica_n1/replication
STATUS:404
MESSAGE:Can not find: /solr/replicaTypesTestColl_shard3_replica_n1/replication
SERVLET:org.apache.solr.embedded.JettySolrRunner$Servlet404-26636c0b
2>
Powered by Jetty:// 10.0.13
2> 2> 2> 2> 2> 625868 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-6-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard1_replica_p6 bb78f355-2b3f-4f1f-862b-fef25a581d53369427620487941 unload) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard1_replica_p6] o.a.s.c.ZkController replicaTypesTestColl_shard1_replica_p6 stopping background replication from leader 2> 625868 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-8-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard3_replica_p5 bb78f355-2b3f-4f1f-862b-fef25a581d53369427643443873 unload) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard3_replica_p5] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replicaTypesTestColl.shard3.replica_p5 tag=null 2> 625890 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-6-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard1_replica_p6 bb78f355-2b3f-4f1f-862b-fef25a581d53369427620487941 unload) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard1_replica_p6] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@69a40f80 replicaTypesTestColl_shard1_replica_p6 2> 625909 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-8-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 bb78f355-2b3f-4f1f-862b-fef25a581d53369427618582558 unload) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.ZkController replicaTypesTestColl_shard1_replica_t2 stopping background replication from leader 2> 625909 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-8-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 bb78f355-2b3f-4f1f-862b-fef25a581d53369427618582558 unload) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard1_replica_t2] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@22b68ee7 replicaTypesTestColl_shard1_replica_t2 2> 625910 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-8-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 bb78f355-2b3f-4f1f-862b-fef25a581d53369427618582558 unload) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard1_replica_t2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replicaTypesTestColl.shard1.replica_t2 tag=SolrCore@22b68ee7 2> 625912 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-8-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 bb78f355-2b3f-4f1f-862b-fef25a581d53369427618582558 unload) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard1_replica_t2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replicaTypesTestColl.shard1.leader tag=SolrCore@22b68ee7 2> 625929 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-10-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 bb78f355-2b3f-4f1f-862b-fef25a581d53369427641790830 unload) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard3_replica_t4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replicaTypesTestColl.shard3.replica_t4 tag=null 2> 625936 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-8-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 bb78f355-2b3f-4f1f-862b-fef25a581d53369427618582558 unload) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard1_replica_t2] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 625960 WARN (indexFetcher-1425-thread-1) [] o.a.s.h.IndexFetcher Leader at: http://127.0.0.1:39159/solr/replicaTypesTestColl_shard3_replica_n1/ is not available. Index fetch failed by exception: org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39159/solr/replicaTypesTestColl_shard3_replica_n1: Expected mime type in [application/octet-stream, application/vnd.apache.solr.javabin] but got text/html. 2> 2> 2> Error 404 Can not find: /solr/replicaTypesTestColl_shard3_replica_n1/replication 2> 2>

HTTP ERROR 404 Can not find: /solr/replicaTypesTestColl_shard3_replica_n1/replication

2> 2> 2> 2> 2> 2>
URI:/solr/replicaTypesTestColl_shard3_replica_n1/replication
STATUS:404
MESSAGE:Can not find: /solr/replicaTypesTestColl_shard3_replica_n1/replication
SERVLET:org.apache.solr.embedded.JettySolrRunner$Servlet404-26636c0b
2>
Powered by Jetty:// 10.0.13
2> 2> 2> 2> 2> 626009 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-8-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard1_replica_t2 bb78f355-2b3f-4f1f-862b-fef25a581d53369427618582558 unload) [n:127.0.0.1:40391_solr] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replicaTypesTestColl/terms/shard1 to Terms{values={core_node18=1}, version=3} for removeTerm 2> 626041 INFO (zkCallback-1164-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 626043 INFO (zkCallback-1166-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 626043 INFO (zkCallback-1160-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replicaTypesTestColl/state.json] for collection [replicaTypesTestColl] has occurred - updating... (live nodes size: [3]) 2> 626114 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=2 2> 626116 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 626116 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 626116 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 626116 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 626117 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 626210 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=bb78f355-2b3f-4f1f-862b-fef25a581d53&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=8 2> 626306 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : bb78f355-2b3f-4f1f-862b-fef25a581d53369427618582558 2> 626308 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=bb78f355-2b3f-4f1f-862b-fef25a581d53369427618582558&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=4 2> 626316 INFO (qtp218128001-2523) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : bb78f355-2b3f-4f1f-862b-fef25a581d53369427620487941 2> 626318 INFO (qtp218128001-2523) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=bb78f355-2b3f-4f1f-862b-fef25a581d53369427620487941&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=3 2> 626386 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-11-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 bb78f355-2b3f-4f1f-862b-fef25a581d53369427640181788 unload) [n:127.0.0.1:39159_solr x:replicaTypesTestColl_shard3_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replicaTypesTestColl.shard3.replica_n1 tag=null 2> 626399 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-10-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 bb78f355-2b3f-4f1f-862b-fef25a581d53369427626122088 unload) [n:127.0.0.1:39159_solr x:replicaTypesTestColl_shard2_replica_n7] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@3e81b1c5 replicaTypesTestColl_shard2_replica_n7 2> 626402 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-8-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard3_replica_p5 bb78f355-2b3f-4f1f-862b-fef25a581d53369427643443873 unload) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard3_replica_p5] o.a.s.c.ZkController replicaTypesTestColl_shard3_replica_p5 stopping background replication from leader 2> 626402 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-7-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard2_replica_p8 bb78f355-2b3f-4f1f-862b-fef25a581d53369427627687796 unload) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard2_replica_p8] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replicaTypesTestColl.shard2.replica_p8 tag=null 2> 626413 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-8-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard3_replica_p5 bb78f355-2b3f-4f1f-862b-fef25a581d53369427643443873 unload) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard3_replica_p5] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@7fae68af replicaTypesTestColl_shard3_replica_p5 2> 626479 INFO (indexFetcher-1423-thread-1) [] o.a.s.h.IndexFetcher Last replication failed, so I'll force replication 2> 626526 WARN (indexFetcher-1419-thread-1) [] o.a.s.h.IndexFetcher Leader at: http://127.0.0.1:39159/solr/replicaTypesTestColl_shard2_replica_n7/ is not available. Index fetch failed by exception: org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39159/solr/replicaTypesTestColl_shard2_replica_n7: Expected mime type in [application/octet-stream, application/vnd.apache.solr.javabin] but got text/html. 2> 2> 2> Error 404 Can not find: /solr/replicaTypesTestColl_shard2_replica_n7/replication 2> 2>

HTTP ERROR 404 Can not find: /solr/replicaTypesTestColl_shard2_replica_n7/replication

2> 2> 2> 2> 2> 2>
URI:/solr/replicaTypesTestColl_shard2_replica_n7/replication
STATUS:404
MESSAGE:Can not find: /solr/replicaTypesTestColl_shard2_replica_n7/replication
SERVLET:org.apache.solr.embedded.JettySolrRunner$Servlet404-26636c0b
2>
Powered by Jetty:// 10.0.13
2> 2> 2> 2> 2> 626531 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-9-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 bb78f355-2b3f-4f1f-862b-fef25a581d53369427624301374 unload) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard2_replica_t3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replicaTypesTestColl.shard2.replica_t3 tag=null 2> 626529 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-10-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 bb78f355-2b3f-4f1f-862b-fef25a581d53369427641790830 unload) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard3_replica_t4] o.a.s.c.ZkController replicaTypesTestColl_shard3_replica_t4 stopping background replication from leader 2> 626537 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-10-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 bb78f355-2b3f-4f1f-862b-fef25a581d53369427641790830 unload) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard3_replica_t4] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@51d66871 replicaTypesTestColl_shard3_replica_t4 2> 626545 WARN (indexFetcher-1423-thread-1) [] o.a.s.h.IndexFetcher Leader at: http://127.0.0.1:39159/solr/replicaTypesTestColl_shard2_replica_n7/ is not available. Index fetch failed by exception: org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39159/solr/replicaTypesTestColl_shard2_replica_n7: Expected mime type in [application/octet-stream, application/vnd.apache.solr.javabin] but got text/html. 2> 2> 2> Error 404 Can not find: /solr/replicaTypesTestColl_shard2_replica_n7/replication 2> 2>

HTTP ERROR 404 Can not find: /solr/replicaTypesTestColl_shard2_replica_n7/replication

2> 2> 2> 2> 2> 2>
URI:/solr/replicaTypesTestColl_shard2_replica_n7/replication
STATUS:404
MESSAGE:Can not find: /solr/replicaTypesTestColl_shard2_replica_n7/replication
SERVLET:org.apache.solr.embedded.JettySolrRunner$Servlet404-26636c0b
2>
Powered by Jetty:// 10.0.13
2> 2> 2> 2> 2> 626811 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-9-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 bb78f355-2b3f-4f1f-862b-fef25a581d53369427622339323 unload) [n:127.0.0.1:39159_solr x:replicaTypesTestColl_shard1_replica_n9] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replicaTypesTestColl.shard1.replica_n9 tag=SolrCore@6e3f5983 2> 626811 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-11-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 bb78f355-2b3f-4f1f-862b-fef25a581d53369427640181788 unload) [n:127.0.0.1:39159_solr x:replicaTypesTestColl_shard3_replica_n1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@5126838b replicaTypesTestColl_shard3_replica_n1 2> 626811 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-9-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 bb78f355-2b3f-4f1f-862b-fef25a581d53369427622339323 unload) [n:127.0.0.1:39159_solr x:replicaTypesTestColl_shard1_replica_n9] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replicaTypesTestColl.shard1.leader tag=SolrCore@6e3f5983 2> 626811 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-11-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 bb78f355-2b3f-4f1f-862b-fef25a581d53369427640181788 unload) [n:127.0.0.1:39159_solr x:replicaTypesTestColl_shard3_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replicaTypesTestColl.shard3.replica_n1 tag=SolrCore@5126838b 2> 626812 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-10-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 bb78f355-2b3f-4f1f-862b-fef25a581d53369427626122088 unload) [n:127.0.0.1:39159_solr x:replicaTypesTestColl_shard2_replica_n7] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replicaTypesTestColl.shard2.replica_n7 tag=SolrCore@3e81b1c5 2> 626812 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-10-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 bb78f355-2b3f-4f1f-862b-fef25a581d53369427626122088 unload) [n:127.0.0.1:39159_solr x:replicaTypesTestColl_shard2_replica_n7] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replicaTypesTestColl.shard2.leader tag=SolrCore@3e81b1c5 2> 626812 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-11-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 bb78f355-2b3f-4f1f-862b-fef25a581d53369427640181788 unload) [n:127.0.0.1:39159_solr x:replicaTypesTestColl_shard3_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replicaTypesTestColl.shard3.leader tag=SolrCore@5126838b 2> 626825 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-10-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 bb78f355-2b3f-4f1f-862b-fef25a581d53369427626122088 unload) [n:127.0.0.1:39159_solr x:replicaTypesTestColl_shard2_replica_n7] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 626825 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-11-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 bb78f355-2b3f-4f1f-862b-fef25a581d53369427640181788 unload) [n:127.0.0.1:39159_solr x:replicaTypesTestColl_shard3_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 626833 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-9-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 bb78f355-2b3f-4f1f-862b-fef25a581d53369427622339323 unload) [n:127.0.0.1:39159_solr x:replicaTypesTestColl_shard1_replica_n9] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 626844 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-7-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard2_replica_p8 bb78f355-2b3f-4f1f-862b-fef25a581d53369427627687796 unload) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard2_replica_p8] o.a.s.c.ZkController replicaTypesTestColl_shard2_replica_p8 stopping background replication from leader 2> 626844 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-6-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard1_replica_p6 bb78f355-2b3f-4f1f-862b-fef25a581d53369427620487941 unload) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard1_replica_p6] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replicaTypesTestColl.shard1.replica_p6 tag=SolrCore@69a40f80 2> 626845 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-6-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard1_replica_p6 bb78f355-2b3f-4f1f-862b-fef25a581d53369427620487941 unload) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard1_replica_p6] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replicaTypesTestColl.shard1.leader tag=SolrCore@69a40f80 2> 626845 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-8-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard3_replica_p5 bb78f355-2b3f-4f1f-862b-fef25a581d53369427643443873 unload) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard3_replica_p5] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replicaTypesTestColl.shard3.replica_p5 tag=SolrCore@7fae68af 2> 626845 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-8-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard3_replica_p5 bb78f355-2b3f-4f1f-862b-fef25a581d53369427643443873 unload) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard3_replica_p5] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replicaTypesTestColl.shard3.leader tag=SolrCore@7fae68af 2> 626846 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-6-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard1_replica_p6 bb78f355-2b3f-4f1f-862b-fef25a581d53369427620487941 unload) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard1_replica_p6] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 626874 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-7-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard2_replica_p8 bb78f355-2b3f-4f1f-862b-fef25a581d53369427627687796 unload) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard2_replica_p8] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@8c59a1d replicaTypesTestColl_shard2_replica_p8 2> 626874 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-7-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard2_replica_p8 bb78f355-2b3f-4f1f-862b-fef25a581d53369427627687796 unload) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard2_replica_p8] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replicaTypesTestColl.shard2.replica_p8 tag=SolrCore@8c59a1d 2> 626875 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-7-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard2_replica_p8 bb78f355-2b3f-4f1f-862b-fef25a581d53369427627687796 unload) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard2_replica_p8] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replicaTypesTestColl.shard2.leader tag=SolrCore@8c59a1d 2> 626883 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-8-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard3_replica_p5 bb78f355-2b3f-4f1f-862b-fef25a581d53369427643443873 unload) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard3_replica_p5] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 626899 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-7-processing-127.0.0.1:44127_solr replicaTypesTestColl_shard2_replica_p8 bb78f355-2b3f-4f1f-862b-fef25a581d53369427627687796 unload) [n:127.0.0.1:44127_solr x:replicaTypesTestColl_shard2_replica_p8] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 626911 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-10-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard2_replica_n7 bb78f355-2b3f-4f1f-862b-fef25a581d53369427626122088 unload) [n:127.0.0.1:39159_solr] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replicaTypesTestColl/terms/shard2 to Terms{values={core_node12=1}, version=3} for removeTerm 2> 626939 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-9-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 bb78f355-2b3f-4f1f-862b-fef25a581d53369427622339323 unload) [n:127.0.0.1:39159_solr] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replicaTypesTestColl/terms/shard1 to Terms{values={}, version=4} for removeTerm 2> 626959 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-9-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard1_replica_n9 bb78f355-2b3f-4f1f-862b-fef25a581d53369427622339323 unload) [n:127.0.0.1:39159_solr] o.a.s.c.ShardLeaderElectionContextBase No version found for ephemeral leader parent node, won't remove previous leader registration. 2> 626980 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-11-processing-127.0.0.1:39159_solr replicaTypesTestColl_shard3_replica_n1 bb78f355-2b3f-4f1f-862b-fef25a581d53369427640181788 unload) [n:127.0.0.1:39159_solr] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replicaTypesTestColl/terms/shard3 to Terms{values={core_node13=0}, version=2} for removeTerm 2> 626999 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-9-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 bb78f355-2b3f-4f1f-862b-fef25a581d53369427624301374 unload) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard2_replica_t3] o.a.s.c.ZkController replicaTypesTestColl_shard2_replica_t3 stopping background replication from leader 2> 626999 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-10-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 bb78f355-2b3f-4f1f-862b-fef25a581d53369427641790830 unload) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard3_replica_t4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replicaTypesTestColl.shard3.replica_t4 tag=SolrCore@51d66871 2> 626999 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-10-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 bb78f355-2b3f-4f1f-862b-fef25a581d53369427641790830 unload) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard3_replica_t4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replicaTypesTestColl.shard3.leader tag=SolrCore@51d66871 2> 627004 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-9-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 bb78f355-2b3f-4f1f-862b-fef25a581d53369427624301374 unload) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard2_replica_t3] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@37c26397 replicaTypesTestColl_shard2_replica_t3 2> 627006 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-10-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 bb78f355-2b3f-4f1f-862b-fef25a581d53369427641790830 unload) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard3_replica_t4] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 627006 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-9-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 bb78f355-2b3f-4f1f-862b-fef25a581d53369427624301374 unload) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard2_replica_t3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.replicaTypesTestColl.shard2.replica_t3 tag=SolrCore@37c26397 2> 627008 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-9-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 bb78f355-2b3f-4f1f-862b-fef25a581d53369427624301374 unload) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard2_replica_t3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replicaTypesTestColl.shard2.leader tag=SolrCore@37c26397 2> 627024 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-9-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 bb78f355-2b3f-4f1f-862b-fef25a581d53369427624301374 unload) [n:127.0.0.1:40391_solr x:replicaTypesTestColl_shard2_replica_t3] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() . 2> 627076 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-10-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 bb78f355-2b3f-4f1f-862b-fef25a581d53369427641790830 unload) [n:127.0.0.1:40391_solr] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replicaTypesTestColl/terms/shard3 to Terms{values={}, version=3} for removeTerm 2> 627089 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-9-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 bb78f355-2b3f-4f1f-862b-fef25a581d53369427624301374 unload) [n:127.0.0.1:40391_solr] o.a.s.c.ZkShardTerms Successful update of terms at /collections/replicaTypesTestColl/terms/shard2 to Terms{values={}, version=4} for removeTerm 2> 627098 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-10-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard3_replica_t4 bb78f355-2b3f-4f1f-862b-fef25a581d53369427641790830 unload) [n:127.0.0.1:40391_solr] o.a.s.c.ShardLeaderElectionContextBase No version found for ephemeral leader parent node, won't remove previous leader registration. 2> 627219 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 627220 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 627220 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 627220 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 627220 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 627221 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 627224 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-9-processing-127.0.0.1:40391_solr replicaTypesTestColl_shard2_replica_t3 bb78f355-2b3f-4f1f-862b-fef25a581d53369427624301374 unload) [n:127.0.0.1:40391_solr] o.a.s.c.ShardLeaderElectionContextBase No version found for ephemeral leader parent node, won't remove previous leader registration. 2> 627231 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=bb78f355-2b3f-4f1f-862b-fef25a581d53&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=14 2> 627324 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : bb78f355-2b3f-4f1f-862b-fef25a581d53369427620487941 2> 627325 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=bb78f355-2b3f-4f1f-862b-fef25a581d53369427620487941&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 627330 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : bb78f355-2b3f-4f1f-862b-fef25a581d53369427622339323 2> 627330 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=bb78f355-2b3f-4f1f-862b-fef25a581d53369427622339323&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 627336 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : bb78f355-2b3f-4f1f-862b-fef25a581d53369427624301374 2> 627336 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=bb78f355-2b3f-4f1f-862b-fef25a581d53369427624301374&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 627341 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : bb78f355-2b3f-4f1f-862b-fef25a581d53369427626122088 2> 627342 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=bb78f355-2b3f-4f1f-862b-fef25a581d53369427626122088&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 627347 INFO (qtp218128001-2528) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : bb78f355-2b3f-4f1f-862b-fef25a581d53369427627687796 2> 627348 INFO (qtp218128001-2528) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=bb78f355-2b3f-4f1f-862b-fef25a581d53369427627687796&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 627353 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : bb78f355-2b3f-4f1f-862b-fef25a581d53369427640181788 2> 627354 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=bb78f355-2b3f-4f1f-862b-fef25a581d53369427640181788&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 627359 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : bb78f355-2b3f-4f1f-862b-fef25a581d53369427641790830 2> 627360 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=bb78f355-2b3f-4f1f-862b-fef25a581d53369427641790830&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 627365 INFO (qtp218128001-2525) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : bb78f355-2b3f-4f1f-862b-fef25a581d53369427643443873 2> 627365 INFO (qtp218128001-2525) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=bb78f355-2b3f-4f1f-862b-fef25a581d53369427643443873&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 627382 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.c.ZkStateReader /collections/replicaTypesTestColl/state.json is deleted, stop watching children 2> 627508 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000022 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 628224 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 628226 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 628226 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 628226 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 628226 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 628226 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 628241 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=bb78f355-2b3f-4f1f-862b-fef25a581d53&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=5 2> 628252 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=bb78f355-2b3f-4f1f-862b-fef25a581d53&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=7 2> 628255 INFO (TEST-CloudSolrClientTest.preferReplicaTypesTest-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Ending preferReplicaTypesTest 2> 628588 INFO (TEST-CloudSolrClientTest.testAliasHandling-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Starting testAliasHandling 2> 628614 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CreateCollectionCmd Create collection testAliasHandling 2> 628617 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000022 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 628787 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"testAliasHandling_shard1_replica_n1", 2> "node_name":"127.0.0.1:40391_solr", 2> "base_url":"http://127.0.0.1:40391/solr", 2> "collection":"testAliasHandling", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 628798 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"testAliasHandling_shard2_replica_n2", 2> "node_name":"127.0.0.1:39159_solr", 2> "base_url":"http://127.0.0.1:39159/solr", 2> "collection":"testAliasHandling", 2> "shard":"shard2", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 628828 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr x:testAliasHandling_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=testAliasHandling_shard1_replica_n1&action=CREATE&numShards=2&collection=testAliasHandling&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 628833 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr x:testAliasHandling_shard2_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=testAliasHandling_shard2_replica_n2&action=CREATE&numShards=2&collection=testAliasHandling&shard=shard2&wt=javabin&version=2&replicaType=NRT 2> 628945 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 628955 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.s.IndexSchema Schema name=test 2> 628963 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 628970 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.s.IndexSchema Schema name=test 2> 628995 WARN (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 628996 WARN (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 628997 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 628999 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'testAliasHandling_shard2_replica_n2' using configuration from configset conf, trusted=true 2> 629002 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/testAliasHandling_shard2_replica_n2], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/testAliasHandling_shard2_replica_n2/data/] 2> 629005 WARN (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 629005 WARN (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 629006 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 629008 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testAliasHandling_shard1_replica_n1' using configuration from configset conf, trusted=true 2> 629012 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/testAliasHandling_shard1_replica_n1], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/testAliasHandling_shard1_replica_n1/data/] 2> 629231 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 629232 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 629233 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 629233 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 629233 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 629233 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 629240 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 629241 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 629252 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 629252 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 629259 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 629261 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 629267 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 629268 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 629286 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 629290 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 629292 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 629293 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 629298 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 629299 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736794699825152 2> 629302 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 629304 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736794705068032 2> 629343 INFO (searcherExecutor-1430-thread-1-processing-127.0.0.1:39159_solr testAliasHandling_shard2_replica_n2 testAliasHandling shard2 core_node4) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 629355 INFO (searcherExecutor-1432-thread-1-processing-127.0.0.1:40391_solr testAliasHandling_shard1_replica_n1 testAliasHandling shard1 core_node3) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 629386 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testAliasHandling/terms/shard1 to Terms{values={core_node3=0}, version=0} for registerTerm 2> 629386 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/testAliasHandling/leaders/shard1 2> 629396 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testAliasHandling/terms/shard2 to Terms{values={core_node4=0}, version=0} for registerTerm 2> 629400 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/testAliasHandling/leaders/shard2 2> 629437 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 629437 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 629437 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40391/solr/testAliasHandling_shard1_replica_n1/ 2> 629442 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 629445 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:40391/solr/testAliasHandling_shard1_replica_n1/ has no replicas 2> 629445 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/testAliasHandling/leaders/shard1/leader after winning as /collections/testAliasHandling/leader_elect/shard1/election/72081801657778183-core_node3-n_0000000000 2> 629452 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 629453 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 629453 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39159/solr/testAliasHandling_shard2_replica_n2/ 2> 629456 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 629460 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:39159/solr/testAliasHandling_shard2_replica_n2/ has no replicas 2> 629461 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/testAliasHandling/leaders/shard2/leader after winning as /collections/testAliasHandling/leader_elect/shard2/election/72081801657778182-core_node4-n_0000000000 2> 629471 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40391/solr/testAliasHandling_shard1_replica_n1/ shard1 2> 629487 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39159/solr/testAliasHandling_shard2_replica_n2/ shard2 2> 629500 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 629521 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 629534 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=testAliasHandling_shard1_replica_n1&action=CREATE&numShards=2&collection=testAliasHandling&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=708 2> 629539 INFO (zkCallback-1164-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testAliasHandling/state.json] for collection [testAliasHandling] has occurred - updating... (live nodes size: [3]) 2> 629561 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=testAliasHandling_shard2_replica_n2&action=CREATE&numShards=2&collection=testAliasHandling&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=729 2> 629581 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas 2> 629582 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf&name=testAliasHandling&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=991 2> 629584 INFO (TEST-CloudSolrClientTest.testAliasHandling-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForActiveCollection: testAliasHandling 2> 629628 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CreateCollectionCmd Create collection 2nd_collection 2> 629630 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000023 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 629797 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"2nd_collection_shard2_replica_n1", 2> "node_name":"127.0.0.1:39159_solr", 2> "base_url":"http://127.0.0.1:39159/solr", 2> "collection":"2nd_collection", 2> "shard":"shard2", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 629808 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"2nd_collection_shard1_replica_n2", 2> "node_name":"127.0.0.1:40391_solr", 2> "base_url":"http://127.0.0.1:40391/solr", 2> "collection":"2nd_collection", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 629839 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr x:2nd_collection_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=2nd_collection_shard1_replica_n2&action=CREATE&numShards=2&collection=2nd_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 629845 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr x:2nd_collection_shard2_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=2nd_collection_shard2_replica_n1&action=CREATE&numShards=2&collection=2nd_collection&shard=shard2&wt=javabin&version=2&replicaType=NRT 2> 629972 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 629979 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.s.IndexSchema Schema name=test 2> 629984 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 629991 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.s.IndexSchema Schema name=test 2> 630014 WARN (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 630014 WARN (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 630015 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 630018 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore '2nd_collection_shard1_replica_n2' using configuration from configset conf, trusted=true 2> 630021 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/2nd_collection_shard1_replica_n2], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/2nd_collection_shard1_replica_n2/data/] 2> 630028 WARN (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 630029 WARN (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 630031 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 630033 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.CoreContainer Creating SolrCore '2nd_collection_shard2_replica_n1' using configuration from configset conf, trusted=true 2> 630037 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/2nd_collection_shard2_replica_n1], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/2nd_collection_shard2_replica_n1/data/] 2> 630238 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 630240 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 630240 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 630240 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 630240 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 630241 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 630264 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 630265 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 630283 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 630284 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 630294 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 630295 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 630306 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 630309 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 630314 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 630314 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 630314 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 630316 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736795766226944 2> 630356 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 630360 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 630361 INFO (searcherExecutor-1442-thread-1-processing-127.0.0.1:40391_solr 2nd_collection_shard1_replica_n2 2nd_collection shard1 core_node4) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 630369 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 630370 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736795822850048 2> 630404 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/2nd_collection/terms/shard1 to Terms{values={core_node4=0}, version=0} for registerTerm 2> 630404 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/2nd_collection/leaders/shard1 2> 630411 INFO (searcherExecutor-1444-thread-1-processing-127.0.0.1:39159_solr 2nd_collection_shard2_replica_n1 2nd_collection shard2 core_node3) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 630451 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/2nd_collection/terms/shard2 to Terms{values={core_node3=0}, version=0} for registerTerm 2> 630454 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/2nd_collection/leaders/shard2 2> 630457 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 630457 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 630457 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40391/solr/2nd_collection_shard1_replica_n2/ 2> 630463 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 630468 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:40391/solr/2nd_collection_shard1_replica_n2/ has no replicas 2> 630468 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/2nd_collection/leaders/shard1/leader after winning as /collections/2nd_collection/leader_elect/shard1/election/72081801657778183-core_node4-n_0000000000 2> 630493 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40391/solr/2nd_collection_shard1_replica_n2/ shard1 2> 630506 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 630506 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 630506 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39159/solr/2nd_collection_shard2_replica_n1/ 2> 630509 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 630514 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:39159/solr/2nd_collection_shard2_replica_n1/ has no replicas 2> 630514 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/2nd_collection/leaders/shard2/leader after winning as /collections/2nd_collection/leader_elect/shard2/election/72081801657778182-core_node3-n_0000000000 2> 630523 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 630545 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39159/solr/2nd_collection_shard2_replica_n1/ shard2 2> 630559 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=2nd_collection_shard1_replica_n2&action=CREATE&numShards=2&collection=2nd_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=721 2> 630584 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 630594 INFO (zkCallback-1164-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/2nd_collection/state.json] for collection [2nd_collection] has occurred - updating... (live nodes size: [3]) 2> 630616 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=2nd_collection_shard2_replica_n1&action=CREATE&numShards=2&collection=2nd_collection&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=773 2> 630630 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas 2> 630630 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf&name=2nd_collection&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=1026 2> 630632 INFO (TEST-CloudSolrClientTest.testAliasHandling-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForActiveCollection: 2nd_collection 2> 630658 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=4 2> 630667 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=4 2> 630675 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LISTALIASES&wt=javabin&version=2} status=0 QTime=4 2> 630684 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=4 2> 630711 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testAliasHandling/terms/shard1 to Terms{values={core_node3=1}, version=1} for ensureHighestTermsAreNotZero 2> 630712 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={wt=javabin&version=2} status=0 QTime=21 2> 630723 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=4 2> 630732 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=4 2> 630741 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=4 2> 630754 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=9 2> 630765 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=4 2> 630774 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=4 2> 630877 INFO (searcherExecutor-1432-thread-1-processing-127.0.0.1:40391_solr testAliasHandling_shard1_replica_n1 testAliasHandling shard1 core_node3) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 630879 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:39159/solr/testAliasHandling_shard2_replica_n2/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false} status=0 QTime=84 2> 630887 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=testAliasHandling:7&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=107 2> 630927 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000025 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 631046 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collections=testAliasHandling&name=testalias&action=CREATEALIAS&wt=javabin&version=2} status=0 QTime=154 2> 631055 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=5 2> 631064 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=5 2> 631073 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=4 2> 631110 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/2nd_collection/terms/shard2 to Terms{values={core_node3=1}, version=1} for ensureHighestTermsAreNotZero 2> 631110 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={wt=javabin&version=2} status=0 QTime=32 2> 631121 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=5 2> 631132 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=4 2> 631141 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=4 2> 631155 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=10 2> 631166 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=4 2> 631175 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=4 2> 631244 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 631247 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 631247 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 631247 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 631247 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 631247 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 631262 INFO (searcherExecutor-1444-thread-1-processing-127.0.0.1:39159_solr 2nd_collection_shard2_replica_n1 2nd_collection shard2 core_node3) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 631266 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:40391/solr/2nd_collection_shard1_replica_n2/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false} status=0 QTime=71 2> 631269 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/update params={_stateVer_=2nd_collection:7&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=89 2> 631302 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000027 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 631426 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collections=2nd_collection&name=testalias2&action=CREATEALIAS&wt=javabin&version=2} status=0 QTime=152 2> 631462 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000029 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 631590 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collections=testAliasHandling,2nd_collection&name=testaliascombined&action=CREATEALIAS&wt=javabin&version=2} status=0 QTime=157 2> 631601 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LISTALIASES&wt=javabin&version=2} status=0 QTime=6 2> 631611 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=6 2> 631622 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=6 2> 631640 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=13 2> 631653 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=5 2> 631663 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=5 2> 631675 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:39159/solr/testAliasHandling_shard2_replica_n2/&rid=127.0.0.1-1377&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335599120&isShard=true&wt=javabin} hits=0 status=0 QTime=2 2> 631680 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:40391/solr/testAliasHandling_shard1_replica_n1/&rid=127.0.0.1-1377&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335599120&isShard=true&wt=javabin} hits=1 status=0 QTime=2 2> 631688 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7&shards.purpose=64&shard.url=http://127.0.0.1:40391/solr/testAliasHandling_shard1_replica_n1/&rid=127.0.0.1-1377&version=2&q=*:*&omitHeader=false&NOW=1696335599120&ids=1&isShard=true&wt=javabin} status=0 QTime=1 2> 631691 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=testAliasHandling:7&wt=javabin&version=2} rid=127.0.0.1-1377 hits=1 status=0 QTime=24 2> 631705 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testalias] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testalias&wt=javabin&version=2} status=0 QTime=11 2> 631718 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LISTALIASES&wt=javabin&version=2} status=0 QTime=6 2> 631730 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=7 2> 631753 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=14 2> 631778 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testalias] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testalias&wt=javabin&version=2} status=0 QTime=14 2> 631797 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LISTALIASES&wt=javabin&version=2} status=0 QTime=5 2> 631809 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=7 2> 631825 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:39159/solr/testAliasHandling_shard2_replica_n2/&rid=127.0.0.1-1378&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335599268&isShard=true&wt=javabin} hits=0 status=0 QTime=1 2> 631825 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:40391/solr/testAliasHandling_shard1_replica_n1/&rid=127.0.0.1-1378&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335599268&isShard=true&wt=javabin} hits=1 status=0 QTime=2 2> 631836 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7&shards.purpose=64&shard.url=http://127.0.0.1:40391/solr/testAliasHandling_shard1_replica_n1/&rid=127.0.0.1-1378&version=2&q=*:*&omitHeader=false&NOW=1696335599268&ids=1&isShard=true&wt=javabin} status=0 QTime=2 2> 631844 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=testAliasHandling:7&wt=javabin&version=2} rid=127.0.0.1-1378 hits=1 status=0 QTime=29 2> 631868 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=19 2> 631881 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=8 2> 631906 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=14 2> 631924 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=8 2> 631942 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=10 2> 631959 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=2nd_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:39159/solr/2nd_collection_shard2_replica_n1/&rid=127.0.0.1-1379&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335599401&isShard=true&wt=javabin} hits=1 status=0 QTime=3 2> 631964 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=2nd_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:40391/solr/2nd_collection_shard1_replica_n2/&rid=127.0.0.1-1379&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335599401&isShard=true&wt=javabin} hits=0 status=0 QTime=3 2> 631975 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=2nd_collection:7&shards.purpose=64&shard.url=http://127.0.0.1:39159/solr/2nd_collection_shard2_replica_n1/&rid=127.0.0.1-1379&version=2&q=*:*&omitHeader=false&NOW=1696335599401&ids=2&isShard=true&wt=javabin} status=0 QTime=2 2> 631979 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=2nd_collection:7&wt=javabin&version=2} rid=127.0.0.1-1379 hits=1 status=0 QTime=32 2> 631994 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:testalias2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testalias2&wt=javabin&version=2} status=0 QTime=10 2> 632007 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LISTALIASES&wt=javabin&version=2} status=0 QTime=7 2> 632020 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=8 2> 632041 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=17 2> 632059 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testalias2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testalias2&wt=javabin&version=2} status=0 QTime=9 2> 632071 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LISTALIASES&wt=javabin&version=2} status=0 QTime=6 2> 632083 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=7 2> 632100 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=2nd_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:39159/solr/2nd_collection_shard2_replica_n1/&rid=127.0.0.1-1380&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335599542&isShard=true&wt=javabin} hits=1 status=0 QTime=3 2> 632112 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=2nd_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=http://127.0.0.1:40391/solr/2nd_collection_shard1_replica_n2/&rid=127.0.0.1-1380&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335599542&isShard=true&wt=javabin} hits=0 status=0 QTime=2 2> 632121 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=2nd_collection:7&shards.purpose=64&shard.url=http://127.0.0.1:39159/solr/2nd_collection_shard2_replica_n1/&rid=127.0.0.1-1380&version=2&q=*:*&omitHeader=false&NOW=1696335599542&ids=2&isShard=true&wt=javabin} status=0 QTime=2 2> 632128 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=2nd_collection:7&wt=javabin&version=2} rid=127.0.0.1-1380 hits=1 status=0 QTime=38 2> 632140 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testaliascombined] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testaliascombined&wt=javabin&version=2} status=0 QTime=8 2> 632150 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LISTALIASES&wt=javabin&version=2} status=0 QTime=5 2> 632159 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=5 2> 632169 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=6 2> 632185 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=11 2> 632200 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:testaliascombined] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testaliascombined&wt=javabin&version=2} status=0 QTime=7 2> 632209 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LISTALIASES&wt=javabin&version=2} status=0 QTime=5 2> 632219 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=5 2> 632229 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=5 2> 632243 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:40391/solr/testAliasHandling_shard1_replica_n1/&rid=127.0.0.1-1381&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335599687&isShard=true&wt=javabin} hits=1 status=0 QTime=2 2> 632247 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:40391/solr/2nd_collection_shard1_replica_n2/&rid=127.0.0.1-1381&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335599687&isShard=true&wt=javabin} hits=0 status=0 QTime=2 2> 632252 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 632254 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 632254 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 632254 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 632254 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 632254 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 632258 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:39159/solr/2nd_collection_shard2_replica_n1/&rid=127.0.0.1-1381&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335599687&isShard=true&wt=javabin} hits=1 status=0 QTime=10 2> 632261 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:39159/solr/testAliasHandling_shard2_replica_n2/&rid=127.0.0.1-1381&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335599687&isShard=true&wt=javabin} hits=0 status=0 QTime=3 2> 632270 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&shards.purpose=64&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:40391/solr/testAliasHandling_shard1_replica_n1/&rid=127.0.0.1-1381&version=2&q=*:*&omitHeader=false&NOW=1696335599687&ids=1&isShard=true&wt=javabin} status=0 QTime=2 2> 632275 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&shards.purpose=64&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:39159/solr/2nd_collection_shard2_replica_n1/&rid=127.0.0.1-1381&version=2&q=*:*&omitHeader=false&NOW=1696335599687&ids=2&isShard=true&wt=javabin} status=0 QTime=8 2> 632278 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=testAliasHandling:7|2nd_collection:7&wt=javabin&version=2} rid=127.0.0.1-1381 hits=2 status=0 QTime=44 2> 632288 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=5 2> 632298 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=5 2> 632309 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=6 2> 632319 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=6 2> 632334 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=11 2> 632347 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=5 2> 632357 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=5 2> 632366 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=5 2> 632376 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=5 2> 632388 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:40391/solr/testAliasHandling_shard1_replica_n1/&rid=127.0.0.1-1382&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335599834&isShard=true&wt=javabin} hits=1 status=0 QTime=2 2> 632395 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:40391/solr/2nd_collection_shard1_replica_n2/&rid=127.0.0.1-1382&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335599834&isShard=true&wt=javabin} hits=0 status=0 QTime=2 2> 632397 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:39159/solr/testAliasHandling_shard2_replica_n2/&rid=127.0.0.1-1382&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335599834&isShard=true&wt=javabin} hits=0 status=0 QTime=2 2> 632400 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:39159/solr/2nd_collection_shard2_replica_n1/&rid=127.0.0.1-1382&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335599834&isShard=true&wt=javabin} hits=1 status=0 QTime=5 2> 632408 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&shards.purpose=64&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:39159/solr/2nd_collection_shard2_replica_n1/&rid=127.0.0.1-1382&version=2&q=*:*&omitHeader=false&NOW=1696335599834&ids=2&isShard=true&wt=javabin} status=0 QTime=3 2> 632410 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&shards.purpose=64&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:40391/solr/testAliasHandling_shard1_replica_n1/&rid=127.0.0.1-1382&version=2&q=*:*&omitHeader=false&NOW=1696335599834&ids=1&isShard=true&wt=javabin} status=0 QTime=2 2> 632415 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=testAliasHandling:7|2nd_collection:7&collection=testAliasHandling,2nd_collection&wt=javabin&version=2} rid=127.0.0.1-1382 hits=2 status=0 QTime=34 2> 632428 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testalias] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testalias&wt=javabin&version=2} status=0 QTime=9 2> 632438 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LISTALIASES&wt=javabin&version=2} status=0 QTime=5 2> 632450 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:testalias2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testalias2&wt=javabin&version=2} status=0 QTime=8 2> 632459 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LISTALIASES&wt=javabin&version=2} status=0 QTime=5 2> 632468 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=5 2> 632479 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=6 2> 632496 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=11 2> 632514 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:testalias] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testalias&wt=javabin&version=2} status=0 QTime=10 2> 632525 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LISTALIASES&wt=javabin&version=2} status=0 QTime=6 2> 632540 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testalias2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testalias2&wt=javabin&version=2} status=0 QTime=10 2> 632564 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LISTALIASES&wt=javabin&version=2} status=0 QTime=6 2> 632580 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=8 2> 632599 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=11 2> 632615 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:40391/solr/testAliasHandling_shard1_replica_n1/&rid=127.0.0.1-1383&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335600057&isShard=true&wt=javabin} hits=1 status=0 QTime=2 2> 632622 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:40391/solr/2nd_collection_shard1_replica_n2/&rid=127.0.0.1-1383&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335600057&isShard=true&wt=javabin} hits=0 status=0 QTime=1 2> 632629 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:39159/solr/2nd_collection_shard2_replica_n1/&rid=127.0.0.1-1383&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335600057&isShard=true&wt=javabin} hits=1 status=0 QTime=2 2> 632636 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:39159/solr/testAliasHandling_shard2_replica_n2/&rid=127.0.0.1-1383&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335600057&isShard=true&wt=javabin} hits=0 status=0 QTime=2 2> 632646 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&shards.purpose=64&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:39159/solr/2nd_collection_shard2_replica_n1/&rid=127.0.0.1-1383&version=2&q=*:*&omitHeader=false&NOW=1696335600057&ids=2&isShard=true&wt=javabin} status=0 QTime=3 2> 632647 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&shards.purpose=64&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:40391/solr/testAliasHandling_shard1_replica_n1/&rid=127.0.0.1-1383&version=2&q=*:*&omitHeader=false&NOW=1696335600057&ids=1&isShard=true&wt=javabin} status=0 QTime=2 2> 632650 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=testAliasHandling:7|2nd_collection:7&collection=testalias,testalias2&wt=javabin&version=2} rid=127.0.0.1-1383 hits=2 status=0 QTime=46 2> 632665 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:testaliascombined] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testaliascombined&wt=javabin&version=2} status=0 QTime=11 2> 632676 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LISTALIASES&wt=javabin&version=2} status=0 QTime=6 2> 632688 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=7 2> 632699 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=7 2> 632718 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=14 2> 632738 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testaliascombined] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testaliascombined&wt=javabin&version=2} status=0 QTime=10 2> 632750 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LISTALIASES&wt=javabin&version=2} status=0 QTime=6 2> 632763 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=7 2> 632775 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=7 2> 632794 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:40391/solr/testAliasHandling_shard1_replica_n1/&rid=127.0.0.1-1384&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335600237&isShard=true&wt=javabin} hits=1 status=0 QTime=2 2> 632796 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:40391/solr/2nd_collection_shard1_replica_n2/&rid=127.0.0.1-1384&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335600237&isShard=true&wt=javabin} hits=0 status=0 QTime=2 2> 632797 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:39159/solr/2nd_collection_shard2_replica_n1/&rid=127.0.0.1-1384&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335600237&isShard=true&wt=javabin} hits=1 status=0 QTime=2 2> 632808 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:39159/solr/testAliasHandling_shard2_replica_n2/&rid=127.0.0.1-1384&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335600237&isShard=true&wt=javabin} hits=0 status=0 QTime=2 2> 632815 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&shards.purpose=64&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:39159/solr/2nd_collection_shard2_replica_n1/&rid=127.0.0.1-1384&version=2&q=*:*&omitHeader=false&NOW=1696335600237&ids=2&isShard=true&wt=javabin} status=0 QTime=2 2> 632823 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&shards.purpose=64&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:40391/solr/testAliasHandling_shard1_replica_n1/&rid=127.0.0.1-1384&version=2&q=*:*&omitHeader=false&NOW=1696335600237&ids=1&isShard=true&wt=javabin} status=0 QTime=2 2> 632827 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=testAliasHandling:7|2nd_collection:7&collection=testaliascombined&wt=javabin&version=2} rid=127.0.0.1-1384 hits=2 status=0 QTime=42 2> 632844 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testalias] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testalias&wt=javabin&version=2} status=0 QTime=11 2> 632880 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LISTALIASES&wt=javabin&version=2} status=0 QTime=12 2> 632918 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=34 2> 632939 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=6 2> 632950 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=6 2> 632971 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=15 2> 632996 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:testalias] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testalias&wt=javabin&version=2} status=0 QTime=14 2> 633006 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LISTALIASES&wt=javabin&version=2} status=0 QTime=5 2> 633017 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=7 2> 633029 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:testAliasHandling] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=testAliasHandling&wt=javabin&version=2} status=0 QTime=7 2> 633043 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:2nd_collection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=2nd_collection&wt=javabin&version=2} status=0 QTime=8 2> 633058 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:40391/solr/testAliasHandling_shard1_replica_n1/&rid=127.0.0.1-1385&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335600502&isShard=true&wt=javabin} hits=1 status=0 QTime=2 2> 633065 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:40391/solr/2nd_collection_shard1_replica_n2/&rid=127.0.0.1-1385&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335600502&isShard=true&wt=javabin} hits=0 status=0 QTime=1 2> 633067 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:39159/solr/2nd_collection_shard2_replica_n1/&rid=127.0.0.1-1385&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335600502&isShard=true&wt=javabin} hits=1 status=0 QTime=2 2> 633073 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:39159/solr/testAliasHandling_shard2_replica_n2/&rid=127.0.0.1-1385&rows=10&version=2&q=*:*&omitHeader=false&NOW=1696335600502&isShard=true&wt=javabin} hits=0 status=0 QTime=1 2> 633080 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&shards.purpose=64&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:39159/solr/2nd_collection_shard2_replica_n1/&rid=127.0.0.1-1385&version=2&q=*:*&omitHeader=false&NOW=1696335600502&ids=2&isShard=true&wt=javabin} status=0 QTime=2 2> 633088 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.S.Request webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testAliasHandling:7|2nd_collection:7&shards.purpose=64&collection=testAliasHandling,2nd_collection&shard.url=http://127.0.0.1:40391/solr/testAliasHandling_shard1_replica_n1/&rid=127.0.0.1-1385&version=2&q=*:*&omitHeader=false&NOW=1696335600502&ids=1&isShard=true&wt=javabin} status=0 QTime=2 2> 633092 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.S.Request webapp=/solr path=/select params={q=*:*&_stateVer_=testAliasHandling:7|2nd_collection:7&collection=testalias,2nd_collection&wt=javabin&version=2} rid=127.0.0.1-1385 hits=2 status=0 QTime=43 2> 633104 INFO (TEST-CloudSolrClientTest.testAliasHandling-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Ending testAliasHandling 2> 633286 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=12 2> 633293 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 633293 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 633293 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 633293 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 633293 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 633529 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000031 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 633640 INFO (TEST-CloudSolrClientTest.testPing-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Starting testPing 2> 633678 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CreateCollectionCmd Create collection ping_test 2> 633883 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"ping_test_shard2_replica_n1", 2> "node_name":"127.0.0.1:39159_solr", 2> "base_url":"http://127.0.0.1:39159/solr", 2> "collection":"ping_test", 2> "shard":"shard2", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 633900 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"ping_test_shard1_replica_n2", 2> "node_name":"127.0.0.1:40391_solr", 2> "base_url":"http://127.0.0.1:40391/solr", 2> "collection":"ping_test", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 633943 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr x:ping_test_shard2_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=ping_test_shard2_replica_n1&action=CREATE&numShards=2&collection=ping_test&shard=shard2&wt=javabin&version=2&replicaType=NRT 2> 633945 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr x:ping_test_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=ping_test_shard1_replica_n2&action=CREATE&numShards=2&collection=ping_test&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 634076 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 634099 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.s.IndexSchema Schema name=test 2> 634107 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 634116 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.s.IndexSchema Schema name=test 2> 634148 WARN (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 634148 WARN (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 634150 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 634152 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'ping_test_shard2_replica_n1' using configuration from configset conf, trusted=true 2> 634156 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/ping_test_shard2_replica_n1], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/ping_test_shard2_replica_n1/data/] 2> 634159 WARN (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 634159 WARN (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 634160 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 634162 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'ping_test_shard1_replica_n2' using configuration from configset conf, trusted=true 2> 634166 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/ping_test_shard1_replica_n2], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/ping_test_shard1_replica_n2/data/] 2> 634299 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=2 2> 634303 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 634303 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 634303 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 634303 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 634303 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 634476 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 634476 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 634480 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 634481 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 634495 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 634495 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 634502 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 634502 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 634526 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 634531 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 634536 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 634540 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 634541 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 634543 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736800198557696 2> 634547 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 634548 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736800203800576 2> 634596 INFO (searcherExecutor-1456-thread-1-processing-127.0.0.1:40391_solr ping_test_shard1_replica_n2 ping_test shard1 core_node4) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 634598 INFO (searcherExecutor-1454-thread-1-processing-127.0.0.1:39159_solr ping_test_shard2_replica_n1 ping_test shard2 core_node3) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 634641 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/ping_test/terms/shard1 to Terms{values={core_node4=0}, version=0} for registerTerm 2> 634641 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/ping_test/leaders/shard1 2> 634656 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/ping_test/terms/shard2 to Terms{values={core_node3=0}, version=0} for registerTerm 2> 634657 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/ping_test/leaders/shard2 2> 634709 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 634709 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 634710 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40391/solr/ping_test_shard1_replica_n2/ 2> 634715 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 634722 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:40391/solr/ping_test_shard1_replica_n2/ has no replicas 2> 634722 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 634722 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/ping_test/leaders/shard1/leader after winning as /collections/ping_test/leader_elect/shard1/election/72081801657778183-core_node4-n_0000000000 2> 634723 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 634723 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39159/solr/ping_test_shard2_replica_n1/ 2> 634733 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 634737 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:39159/solr/ping_test_shard2_replica_n1/ has no replicas 2> 634738 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/ping_test/leaders/shard2/leader after winning as /collections/ping_test/leader_elect/shard2/election/72081801657778182-core_node3-n_0000000000 2> 634756 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40391/solr/ping_test_shard1_replica_n2/ shard1 2> 634769 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39159/solr/ping_test_shard2_replica_n1/ shard2 2> 634797 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 634801 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 634846 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=ping_test_shard1_replica_n2&action=CREATE&numShards=2&collection=ping_test&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=904 2> 634850 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=ping_test_shard2_replica_n1&action=CREATE&numShards=2&collection=ping_test&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=909 2> 634876 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas 2> 634877 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf&name=ping_test&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=1232 2> 634878 INFO (TEST-CloudSolrClientTest.testPing-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForActiveCollection: ping_test 2> 634921 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:ping_test] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=ping_test&wt=javabin&version=2} status=0 QTime=15 2> 634934 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:ping_test] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=ping_test&wt=javabin&version=2} status=0 QTime=8 2> 634959 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=20 2> 634979 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:ping_test] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=ping_test&wt=javabin&version=2} status=0 QTime=8 2> 634991 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:ping_test] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=ping_test&wt=javabin&version=2} status=0 QTime=6 2> 634999 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/admin/ping params={_stateVer_=ping_test:7&wt=javabin&version=2} hits=0 status=0 QTime=3 2> 635000 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/admin/ping params={_stateVer_=ping_test:7&wt=javabin&version=2} status=0 QTime=3 2> 635005 INFO (TEST-CloudSolrClientTest.testPing-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Ending testPing 2> 635337 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=10 2> 635345 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 635350 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 635350 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 635350 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 635350 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 635511 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Starting testPerReplicaStateCollection 2> 635545 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CreateCollectionCmd Create collection testPerReplicaStateCollection 2> 635548 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000033 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 635753 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"testPerReplicaStateCollection_shard2_replica_n1", 2> "node_name":"127.0.0.1:39159_solr", 2> "base_url":"http://127.0.0.1:39159/solr", 2> "collection":"testPerReplicaStateCollection", 2> "shard":"shard2", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 635766 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"testPerReplicaStateCollection_shard1_replica_n2", 2> "node_name":"127.0.0.1:40391_solr", 2> "base_url":"http://127.0.0.1:40391/solr", 2> "collection":"testPerReplicaStateCollection", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 635802 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=testPerReplicaStateCollection_shard2_replica_n1&action=CREATE&numShards=2&collection=testPerReplicaStateCollection&shard=shard2&wt=javabin&version=2&replicaType=NRT 2> 635804 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=testPerReplicaStateCollection_shard1_replica_n2&action=CREATE&numShards=2&collection=testPerReplicaStateCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 636240 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 636249 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 636249 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.s.IndexSchema Schema name=test 2> 636259 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.s.IndexSchema Schema name=test 2> 636295 WARN (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 636296 WARN (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 636297 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 636299 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testPerReplicaStateCollection_shard2_replica_n1' using configuration from configset conf, trusted=true 2> 636301 WARN (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 636301 WARN (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 636302 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 636304 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'testPerReplicaStateCollection_shard1_replica_n2' using configuration from configset conf, trusted=true 2> 636305 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/testPerReplicaStateCollection_shard2_replica_n1], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/testPerReplicaStateCollection_shard2_replica_n1/data/] 2> 636308 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/testPerReplicaStateCollection_shard1_replica_n2], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/testPerReplicaStateCollection_shard1_replica_n2/data/] 2> 636355 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 636356 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 636357 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 636357 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 636357 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 636357 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 636625 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 636626 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 636636 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 636637 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 636647 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 636647 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 636663 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 636664 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 636678 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 636681 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 636689 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 636690 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736802449850368 2> 636694 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 636698 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 636708 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 636709 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736802469773312 2> 636731 INFO (searcherExecutor-1468-thread-1-processing-127.0.0.1:40391_solr testPerReplicaStateCollection_shard1_replica_n2 testPerReplicaStateCollection shard1 core_node4) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 636745 INFO (searcherExecutor-1466-thread-1-processing-127.0.0.1:39159_solr testPerReplicaStateCollection_shard2_replica_n1 testPerReplicaStateCollection shard2 core_node3) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 636781 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testPerReplicaStateCollection/terms/shard1 to Terms{values={core_node4=0}, version=0} for registerTerm 2> 636782 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/testPerReplicaStateCollection/leaders/shard1 2> 636786 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testPerReplicaStateCollection/terms/shard2 to Terms{values={core_node3=0}, version=0} for registerTerm 2> 636786 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/testPerReplicaStateCollection/leaders/shard2 2> 636870 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 636870 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 636871 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39159/solr/testPerReplicaStateCollection_shard2_replica_n1/ 2> 636876 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 636876 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 636877 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 636877 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40391/solr/testPerReplicaStateCollection_shard1_replica_n2/ 2> 636880 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:39159/solr/testPerReplicaStateCollection_shard2_replica_n1/ has no replicas 2> 636881 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/testPerReplicaStateCollection/leaders/shard2/leader after winning as /collections/testPerReplicaStateCollection/leader_elect/shard2/election/72081801657778182-core_node3-n_0000000000 2> 636884 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 636890 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:40391/solr/testPerReplicaStateCollection_shard1_replica_n2/ has no replicas 2> 636890 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/testPerReplicaStateCollection/leaders/shard1/leader after winning as /collections/testPerReplicaStateCollection/leader_elect/shard1/election/72081801657778183-core_node4-n_0000000000 2> 636931 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39159/solr/testPerReplicaStateCollection_shard2_replica_n1/ shard2 2> 636939 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40391/solr/testPerReplicaStateCollection_shard1_replica_n2/ shard1 2> 636975 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 636985 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 637022 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=testPerReplicaStateCollection_shard2_replica_n1&action=CREATE&numShards=2&collection=testPerReplicaStateCollection&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1222 2> 637032 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=testPerReplicaStateCollection_shard1_replica_n2&action=CREATE&numShards=2&collection=testPerReplicaStateCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1230 2> 637048 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas 2> 637049 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf&name=testPerReplicaStateCollection&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=1534 2> 637097 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CreateCollectionCmd Create collection perReplicaState_test 2> 637100 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000035 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 637164 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"perReplicaState_test_shard2_replica_n1", 2> "node_name":"127.0.0.1:40391_solr", 2> "base_url":"http://127.0.0.1:40391/solr", 2> "collection":"perReplicaState_test", 2> "shard":"shard2", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 637204 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"perReplicaState_test_shard1_replica_n3", 2> "node_name":"127.0.0.1:39159_solr", 2> "base_url":"http://127.0.0.1:39159/solr", 2> "collection":"perReplicaState_test", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 637233 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"perReplicaState_test_shard2_replica_n5", 2> "node_name":"127.0.0.1:39159_solr", 2> "base_url":"http://127.0.0.1:39159/solr", 2> "collection":"perReplicaState_test", 2> "shard":"shard2", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 637267 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"perReplicaState_test_shard1_replica_n7", 2> "node_name":"127.0.0.1:44127_solr", 2> "base_url":"http://127.0.0.1:44127/solr", 2> "collection":"perReplicaState_test", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 637298 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr x:perReplicaState_test_shard2_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=perReplicaState_test_shard2_replica_n1&action=CREATE&numShards=2&collection=perReplicaState_test&shard=shard2&wt=javabin&version=2&replicaType=NRT 2> 637300 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr x:perReplicaState_test_shard1_replica_n3] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=perReplicaState_test_shard1_replica_n3&action=CREATE&numShards=2&collection=perReplicaState_test&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 637302 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr x:perReplicaState_test_shard2_replica_n5] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf&newCollection=true&name=perReplicaState_test_shard2_replica_n5&action=CREATE&numShards=2&collection=perReplicaState_test&shard=shard2&wt=javabin&version=2&replicaType=NRT 2> 637309 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr x:perReplicaState_test_shard1_replica_n7] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf&newCollection=true&name=perReplicaState_test_shard1_replica_n7&action=CREATE&numShards=2&collection=perReplicaState_test&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 637362 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 637364 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 637365 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 637365 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 637365 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 637365 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 637447 INFO (zkCallback-1160-thread-5) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [3]) 2> 637547 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard2 r:core_node6 x:perReplicaState_test_shard2_replica_n5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 637586 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard2 r:core_node6 x:perReplicaState_test_shard2_replica_n5] o.a.s.s.IndexSchema Schema name=test 2> 637608 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 637615 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 637628 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.s.IndexSchema Schema name=test 2> 637628 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.s.IndexSchema Schema name=test 2> 637632 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 637638 WARN (qtp217923472-2538) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard2 r:core_node6 x:perReplicaState_test_shard2_replica_n5] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 637638 WARN (qtp217923472-2538) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard2 r:core_node6 x:perReplicaState_test_shard2_replica_n5] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 637639 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard2 r:core_node6 x:perReplicaState_test_shard2_replica_n5] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 637640 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard2 r:core_node6 x:perReplicaState_test_shard2_replica_n5] o.a.s.c.CoreContainer Creating SolrCore 'perReplicaState_test_shard2_replica_n5' using configuration from configset conf, trusted=true 2> 637644 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard2 r:core_node6 x:perReplicaState_test_shard2_replica_n5] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/perReplicaState_test_shard2_replica_n5], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/perReplicaState_test_shard2_replica_n5/data/] 2> 637645 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.s.IndexSchema Schema name=test 2> 637681 WARN (qtp218128001-2526) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 637681 WARN (qtp218128001-2526) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 637682 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 637684 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.c.CoreContainer Creating SolrCore 'perReplicaState_test_shard1_replica_n7' using configuration from configset conf, trusted=true 2> 637689 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node3/perReplicaState_test_shard1_replica_n7], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node3/perReplicaState_test_shard1_replica_n7/data/] 2> 637717 WARN (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 637717 WARN (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 637719 WARN (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 637721 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 637722 WARN (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 637723 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 637725 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'perReplicaState_test_shard1_replica_n3' using configuration from configset conf, trusted=true 2> 637725 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'perReplicaState_test_shard2_replica_n1' using configuration from configset conf, trusted=true 2> 637729 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/perReplicaState_test_shard2_replica_n1], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/perReplicaState_test_shard2_replica_n1/data/] 2> 637734 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/perReplicaState_test_shard1_replica_n3], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/perReplicaState_test_shard1_replica_n3/data/] 2> 638253 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard2 r:core_node6 x:perReplicaState_test_shard2_replica_n5] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 638253 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard2 r:core_node6 x:perReplicaState_test_shard2_replica_n5] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 638297 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard2 r:core_node6 x:perReplicaState_test_shard2_replica_n5] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 638298 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard2 r:core_node6 x:perReplicaState_test_shard2_replica_n5] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 638333 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 638334 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 638352 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 638353 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 638358 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 638358 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 638367 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 638368 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 638371 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 638373 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 638373 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 638373 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 638373 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 638373 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 638375 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard2 r:core_node6 x:perReplicaState_test_shard2_replica_n5] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 638376 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 638377 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 638381 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard2 r:core_node6 x:perReplicaState_test_shard2_replica_n5] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 638387 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard2 r:core_node6 x:perReplicaState_test_shard2_replica_n5] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 638389 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard2 r:core_node6 x:perReplicaState_test_shard2_replica_n5] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736804231380992 2> 638420 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 638426 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 638429 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 638441 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 638443 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736804288004096 2> 638450 INFO (searcherExecutor-1478-thread-1-processing-127.0.0.1:39159_solr perReplicaState_test_shard2_replica_n5 perReplicaState_test shard2 core_node6) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard2 r:core_node6 x:perReplicaState_test_shard2_replica_n5] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 638452 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 638453 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 638454 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 638468 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 638469 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736804315267072 2> 638497 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 638501 INFO (searcherExecutor-1484-thread-1-processing-127.0.0.1:40391_solr perReplicaState_test_shard2_replica_n1 perReplicaState_test shard2 core_node2) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 638505 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 638531 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 638535 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736804384473088 2> 638540 INFO (searcherExecutor-1482-thread-1-processing-127.0.0.1:39159_solr perReplicaState_test_shard1_replica_n3 perReplicaState_test shard1 core_node4) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 638559 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard2 r:core_node6 x:perReplicaState_test_shard2_replica_n5] o.a.s.c.ZkShardTerms Successful update of terms at /collections/perReplicaState_test/terms/shard2 to Terms{values={core_node6=0}, version=0} for registerTerm 2> 638560 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard2 r:core_node6 x:perReplicaState_test_shard2_replica_n5] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/perReplicaState_test/leaders/shard2 2> 638589 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/perReplicaState_test/terms/shard2 to Terms{values={core_node6=0, core_node2=0}, version=1} for registerTerm 2> 638590 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/perReplicaState_test/leaders/shard2 2> 638597 INFO (searcherExecutor-1480-thread-1-processing-127.0.0.1:44127_solr perReplicaState_test_shard1_replica_n7 perReplicaState_test shard1 core_node8) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 638684 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.c.ZkShardTerms Successful update of terms at /collections/perReplicaState_test/terms/shard1 to Terms{values={core_node4=0}, version=0} for registerTerm 2> 638685 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/perReplicaState_test/leaders/shard1 2> 638723 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.c.ZkShardTerms Successful update of terms at /collections/perReplicaState_test/terms/shard1 to Terms{values={core_node4=0, core_node8=0}, version=1} for registerTerm 2> 638723 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/perReplicaState_test/leaders/shard1 2> 638732 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 638733 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 638733 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40391/solr/perReplicaState_test_shard2_replica_n1/ 2> 638742 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.u.PeerSync PeerSync: core=perReplicaState_test_shard2_replica_n1 url=http://127.0.0.1:40391/solr START replicas=[http://127.0.0.1:39159/solr/perReplicaState_test_shard2_replica_n5/] nUpdates=100 2> 638745 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.u.PeerSync PeerSync: core=perReplicaState_test_shard2_replica_n1 url=http://127.0.0.1:40391/solr DONE. We have no versions. sync failed. 2> 638749 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard2 r:core_node6 x:perReplicaState_test_shard2_replica_n5] o.a.s.c.S.Request webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=2 2> 638754 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate 2> 638755 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway 2> 638755 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/perReplicaState_test/leaders/shard2/leader after winning as /collections/perReplicaState_test/leader_elect/shard2/election/72081801657778183-core_node2-n_0000000000 2> 638793 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40391/solr/perReplicaState_test_shard2_replica_n1/ shard2 2> 638807 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 638808 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 638808 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39159/solr/perReplicaState_test_shard1_replica_n3/ 2> 638814 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.u.PeerSync PeerSync: core=perReplicaState_test_shard1_replica_n3 url=http://127.0.0.1:39159/solr START replicas=[http://127.0.0.1:44127/solr/perReplicaState_test_shard1_replica_n7/] nUpdates=100 2> 638818 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.u.PeerSync PeerSync: core=perReplicaState_test_shard1_replica_n3 url=http://127.0.0.1:39159/solr DONE. We have no versions. sync failed. 2> 638835 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.c.S.Request webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=2 2> 638839 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate 2> 638839 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway 2> 638839 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/perReplicaState_test/leaders/shard1/leader after winning as /collections/perReplicaState_test/leader_elect/shard1/election/72081801657778182-core_node4-n_0000000000 2> 638851 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 638880 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39159/solr/perReplicaState_test_shard1_replica_n3/ shard1 2> 638954 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=perReplicaState_test_shard2_replica_n1&action=CREATE&numShards=2&collection=perReplicaState_test&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1659 2> 638968 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 638980 INFO (zkCallback-1164-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [3]) 2> 639015 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=perReplicaState_test_shard1_replica_n3&action=CREATE&numShards=2&collection=perReplicaState_test&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1718 2> 639378 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 639380 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 639380 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 639380 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 639380 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 639381 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 639700 INFO (zkCallback-1164-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [3]) 2> 639700 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [3]) 2> 639713 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf&newCollection=true&name=perReplicaState_test_shard2_replica_n5&action=CREATE&numShards=2&collection=perReplicaState_test&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2413 2> 639819 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [3]) 2> 639820 INFO (zkCallback-1164-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [3]) 2> 639878 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf&newCollection=true&name=perReplicaState_test_shard1_replica_n7&action=CREATE&numShards=2&collection=perReplicaState_test&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2572 2> 639912 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas 2> 639912 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf&name=perReplicaState_test&nrtReplicas=2&action=CREATE&numShards=2&perReplicaState=true&wt=javabin&version=2} status=0 QTime=2858 2> 639914 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForActiveCollection: perReplicaState_test 2> 639969 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:perReplicaState_test] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=perReplicaState_test&wt=javabin&version=2} status=0 QTime=11 2> 640011 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:perReplicaState_test] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=perReplicaState_test&wt=javabin&version=2} status=0 QTime=7 2> 640032 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=16 2> 640057 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=perReplicaState_test&wt=javabin&version=2} status=0 QTime=12 2> 640071 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:perReplicaState_test] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={prs=true&action=CLUSTERSTATUS&collection=perReplicaState_test&wt=javabin&version=2} status=0 QTime=8 2> 640082 INFO (qtp218128001-2528) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.c.S.Request webapp=/solr path=/admin/ping params={_stateVer_=perReplicaState_test:3&wt=javabin&version=2} hits=0 status=0 QTime=3 2> 640083 INFO (qtp218128001-2528) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.c.S.Request webapp=/solr path=/admin/ping params={_stateVer_=perReplicaState_test:3&wt=javabin&version=2} status=0 QTime=3 2> 640110 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0) 2> 640111 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ... 2> 640112 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.e.j.s.Server jetty-10.0.13; built: 2022-12-07T20:13:20.134Z; git: 1c2636ea05c0ca8de1ffd6ca7f3a98ac084c766d; jvm 11.0.20+8 2> 640119 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.e.j.s.s.DefaultSessionIdManager Session workerName=node0 2> 640124 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1aaf7568{/solr,file:///build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-cwd/,AVAILABLE} 2> 640128 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.e.j.s.AbstractConnector Started ServerConnector@26e657de{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:42669} 2> 640129 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.e.j.s.Server Started Server@72c831f3{STARTING}[10.0.13,sto=0] @640352ms 2> 640131 ERROR (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete. 2> 640132 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.s.CoreContainerProvider Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory 2> 640133 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.s.CoreContainerProvider ___ _ Welcome to Apache Solr™ version 9.2.1 2> 640134 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.s.CoreContainerProvider / __| ___| |_ _ Starting in cloud mode on port null 2> 640134 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.s.CoreContainerProvider \__ \/ _ \ | '_| Install dir: /build/solr/src/solr-9.2.1/solr 2> 640135 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.s.CoreContainerProvider |___/\___/_|_| Start time: 2023-10-03T12:20:07.588522Z 2> 640137 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.s.CoreContainerProvider Solr Home: /build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node4 (source: servlet config: solr.solr.home) 2> 640138 WARN (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 640158 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 640176 INFO (zkConnectionManagerCallback-1502-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 640181 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 640182 WARN (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes 2> 640189 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.c.NodeConfig solr.xml found in ZooKeeper. Loading... 2> 640375 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40255/solr 2> 640376 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=true. Solr will be using distributed cluster state updates. 2> 640386 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 640387 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 640388 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 640388 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 640388 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 640388 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 640390 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [n:127.0.0.1:42669_solr] o.a.s.c.c.ConnectionManager Waiting up to 30000ms for client to connect to ZooKeeper 2> 640404 INFO (zkConnectionManagerCallback-1512-thread-1) [] o.a.s.c.c.ConnectionManager zkClient has connected 2> 640405 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [n:127.0.0.1:42669_solr] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper 2> 640459 WARN (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [n:127.0.0.1:42669_solr] o.a.s.c.ZkController Contents of zookeeper /security.json are world-readable; consider setting up ACLs as described in https://solr.apache.org/guide/solr/latest/deployment-guide/zookeeper-access-control.html 2> 640467 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [n:127.0.0.1:42669_solr] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3) 2> 640486 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [n:127.0.0.1:42669_solr] o.a.s.c.DistributedClusterStateUpdater Creating DistributedClusterStateUpdater with useDistributedStateUpdate=true. Solr will be using distributed cluster state updates. 2> 640505 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [n:127.0.0.1:42669_solr] o.a.s.c.ZkController Publish node=127.0.0.1:42669_solr as DOWN 2> 640580 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [n:127.0.0.1:42669_solr] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42669_solr 2> 640601 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4) 2> 640603 INFO (zkCallback-1186-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4) 2> 640610 INFO (zkCallback-1164-thread-2) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4) 2> 640610 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4) 2> 640613 INFO (zkCallback-1511-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4) 2> 640632 WARN (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [n:127.0.0.1:42669_solr] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info 2> 641130 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000037 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 641393 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 641395 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 641395 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 641395 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 641395 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 641396 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 641396 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [n:127.0.0.1:42669_solr] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node4 2> 641615 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [n:127.0.0.1:42669_solr] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 641671 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [n:127.0.0.1:42669_solr] o.a.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42669, zkHost=127.0.0.1:40255/solr} 2> 641705 INFO (OverseerThreadFactory-1176-thread-5) [ c:perReplicaState_test s:shard1] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:42669_solr for creating new replica of shard shard1 for collection perReplicaState_test 2> 641714 INFO (OverseerThreadFactory-1176-thread-5) [ c:perReplicaState_test s:shard1] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command. 2> 641720 INFO (OverseerThreadFactory-1176-thread-5) [ c:perReplicaState_test s:shard1] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"perReplicaState_test_shard1_replica_n9", 2> "node_name":"127.0.0.1:42669_solr", 2> "base_url":"http://127.0.0.1:42669/solr", 2> "collection":"perReplicaState_test", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"addreplica"} 2> 641735 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [4]) 2> 641736 INFO (zkCallback-1166-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [4]) 2> 641736 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [4]) 2> 641735 INFO (zkCallback-1160-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [4]) 2> 641745 INFO (zkCallback-1164-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [4]) 2> 641745 INFO (zkCallback-1164-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [4]) 2> 641770 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr x:perReplicaState_test_shard1_replica_n9] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node10&collection.configName=conf&name=perReplicaState_test_shard1_replica_n9&action=CREATE&collection=perReplicaState_test&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 641812 INFO (zkCallback-1166-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [4]) 2> 641813 INFO (zkCallback-1160-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [4]) 2> 641813 INFO (zkCallback-1164-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [4]) 2> 641898 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 641941 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.s.IndexSchema Schema name=test 2> 641967 WARN (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 641970 WARN (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 641971 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 641973 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.c.CoreContainer Creating SolrCore 'perReplicaState_test_shard1_replica_n9' using configuration from configset conf, trusted=true 2> 641979 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node4/perReplicaState_test_shard1_replica_n9], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node4/perReplicaState_test_shard1_replica_n9/data/] 2> 642344 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.j.SolrRequestAuthorizer Creating a new SolrRequestAuthorizer 2> 642359 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 642359 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 642379 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 642380 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 642401 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=2 2> 642403 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 642403 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 642403 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 642404 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 642404 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 642407 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 642410 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 642416 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 642417 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736808455045120 2> 642452 INFO (searcherExecutor-1519-thread-1-processing-127.0.0.1:42669_solr perReplicaState_test_shard1_replica_n9 perReplicaState_test shard1 core_node10) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 642492 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.c.ZkShardTerms Successful update of terms at /collections/perReplicaState_test/terms/shard1 to Terms{values={core_node10=0, core_node4=0, core_node8=0}, version=2} for registerTerm 2> 642493 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/perReplicaState_test/leaders/shard1 2> 642544 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.c.ZkController Core needs to recover:perReplicaState_test_shard1_replica_n9 2> 642546 INFO (updateExecutor-1508-thread-1-processing-127.0.0.1:42669_solr perReplicaState_test_shard1_replica_n9 perReplicaState_test shard1 core_node10) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.u.DefaultSolrCoreState Running recovery 2> 642550 INFO (recoveryExecutor-1510-thread-1-processing-127.0.0.1:42669_solr perReplicaState_test_shard1_replica_n9 perReplicaState_test shard1 core_node10) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true 2> 642551 INFO (recoveryExecutor-1510-thread-1-processing-127.0.0.1:42669_solr perReplicaState_test_shard1_replica_n9 perReplicaState_test shard1 core_node10) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.c.RecoveryStrategy startupVersions is empty 2> 642573 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node10&collection.configName=conf&name=perReplicaState_test_shard1_replica_n9&action=CREATE&collection=perReplicaState_test&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=805 2> 642583 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.c.S.Request webapp=/solr path=/admin/ping params={wt=javabin&version=2} hits=0 status=0 QTime=2 2> 642584 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.c.S.Request webapp=/solr path=/admin/ping params={wt=javabin&version=2} status=0 QTime=3 2> 642588 INFO (recoveryExecutor-1510-thread-1-processing-127.0.0.1:42669_solr perReplicaState_test_shard1_replica_n9 perReplicaState_test shard1 core_node10) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[perReplicaState_test_shard1_replica_n9] 2> 642596 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:perReplicaState_test] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=ADDREPLICA&collection=perReplicaState_test&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=919 2> 642597 INFO (recoveryExecutor-1510-thread-1-processing-127.0.0.1:42669_solr perReplicaState_test_shard1_replica_n9 perReplicaState_test shard1 core_node10) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null} 2> 642597 INFO (recoveryExecutor-1510-thread-1-processing-127.0.0.1:42669_solr perReplicaState_test_shard1_replica_n9 perReplicaState_test shard1 core_node10) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.c.RecoveryStrategy Publishing state of core [perReplicaState_test_shard1_replica_n9] as recovering, leader is [http://127.0.0.1:39159/solr/perReplicaState_test_shard1_replica_n3/] and I am [http://127.0.0.1:42669/solr/perReplicaState_test_shard1_replica_n9/] 2> 642630 INFO (zkCallback-1166-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [4]) 2> 642630 INFO (zkCallback-1160-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [4]) 2> 642633 INFO (zkCallback-1511-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [4]) 2> 642637 INFO (zkCallback-1164-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [4]) 2> 642643 INFO (zkCallback-1511-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [4]) 2> 642651 INFO (recoveryExecutor-1510-thread-1-processing-127.0.0.1:42669_solr perReplicaState_test_shard1_replica_n9 perReplicaState_test shard1 core_node10) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:39159/solr]; [WaitForState: action=PREPRECOVERY&core=perReplicaState_test_shard1_replica_n3&nodeName=127.0.0.1:42669_solr&coreNodeName=core_node10&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true] 2> 642669 INFO (zkCallback-1511-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [4]) 2> 642672 INFO (zkCallback-1511-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [4]) 2> 642674 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr x:perReplicaState_test_shard1_replica_n3] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node10, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true 2> 642688 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=perReplicaState_test, shard=shard1, thisCore=perReplicaState_test_shard1_replica_n3, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:42669_solr, coreNodeName=core_node10, onlyIfActiveCheckResult=false, nodeProps: core_node10:{ 2> "core":"perReplicaState_test_shard1_replica_n9", 2> "node_name":"127.0.0.1:42669_solr", 2> "base_url":"http://127.0.0.1:42669/solr", 2> "state":"down", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 642689 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=perReplicaState_test, shard=shard1, thisCore=perReplicaState_test_shard1_replica_n3, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:42669_solr, coreNodeName=core_node10, onlyIfActiveCheckResult=false, nodeProps: core_node10:{ 2> "core":"perReplicaState_test_shard1_replica_n9", 2> "node_name":"127.0.0.1:42669_solr", 2> "base_url":"http://127.0.0.1:42669/solr", 2> "state":"down", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 642690 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=perReplicaState_test, shard=shard1, thisCore=perReplicaState_test_shard1_replica_n3, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:42669_solr, coreNodeName=core_node10, onlyIfActiveCheckResult=false, nodeProps: core_node10:{ 2> "core":"perReplicaState_test_shard1_replica_n9", 2> "node_name":"127.0.0.1:42669_solr", 2> "base_url":"http://127.0.0.1:42669/solr", 2> "state":"down", 2> "type":"NRT", 2> "force_set_state":"false"} 2> 642691 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:42669_solr&onlyIfLeaderActive=true&core=perReplicaState_test_shard1_replica_n3&coreNodeName=core_node10&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=19 2> 643399 INFO (recoveryExecutor-1510-thread-1-processing-127.0.0.1:42669_solr perReplicaState_test_shard1_replica_n9 perReplicaState_test shard1 core_node10) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:39159/solr/perReplicaState_test_shard1_replica_n3/] - recoveringAfterStartup=[true] 2> 643401 WARN (recoveryExecutor-1510-thread-1-processing-127.0.0.1:42669_solr perReplicaState_test_shard1_replica_n9 perReplicaState_test shard1 core_node10) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.u.PeerSyncWithLeader no frame of reference to tell if we've missed updates 2> 643402 INFO (recoveryExecutor-1510-thread-1-processing-127.0.0.1:42669_solr perReplicaState_test_shard1_replica_n9 perReplicaState_test shard1 core_node10) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.c.RecoveryStrategy PeerSync Recovery was not successful - trying replication. 2> 643403 INFO (recoveryExecutor-1510-thread-1-processing-127.0.0.1:42669_solr perReplicaState_test_shard1_replica_n9 perReplicaState_test shard1 core_node10) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.c.RecoveryStrategy Starting Replication Recovery. 2> 643403 INFO (recoveryExecutor-1510-thread-1-processing-127.0.0.1:42669_solr perReplicaState_test_shard1_replica_n9 perReplicaState_test shard1 core_node10) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.c.RecoveryStrategy Attempting to replicate from [http://127.0.0.1:39159/solr/perReplicaState_test_shard1_replica_n3/]. 2> 643417 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=2 2> 643422 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 643422 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 643423 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 643423 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 643423 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 643469 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard2 r:core_node6 x:perReplicaState_test_shard2_replica_n5] o.a.s.c.S.Request webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:40391/solr/perReplicaState_test_shard2_replica_n1/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false} status=0 QTime=3 2> 643473 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.c.S.Request webapp=/solr path=/update params={update.distrib=TOLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:39159/solr/perReplicaState_test_shard1_replica_n3/&commit_end_point=leaders&wt=javabin&version=2&expungeDeletes=false} status=0 QTime=29 2> 643508 INFO (qtp434474682-2866) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.u.p.DistributedUpdateProcessor Ignoring commit while not ACTIVE - state: BUFFERING replay: false 2> 643521 INFO (qtp434474682-2866) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.c.S.Request webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:39159/solr/perReplicaState_test_shard1_replica_n3/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false} status=0 QTime=21 2> 643521 INFO (qtp218128001-2525) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.c.S.Request webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=http://127.0.0.1:39159/solr/perReplicaState_test_shard1_replica_n3/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false} status=0 QTime=34 2> 643527 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.c.S.Request webapp=/solr path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=111 2> 643539 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=2 2> 643541 INFO (recoveryExecutor-1510-thread-1-processing-127.0.0.1:42669_solr perReplicaState_test_shard1_replica_n9 perReplicaState_test shard1 core_node10) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.h.IndexFetcher Leader's generation: 1 2> 643542 INFO (recoveryExecutor-1510-thread-1-processing-127.0.0.1:42669_solr perReplicaState_test_shard1_replica_n9 perReplicaState_test shard1 core_node10) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.h.IndexFetcher Leader's version: 0 2> 643542 INFO (recoveryExecutor-1510-thread-1-processing-127.0.0.1:42669_solr perReplicaState_test_shard1_replica_n9 perReplicaState_test shard1 core_node10) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.h.IndexFetcher Follower's generation: 1 2> 643542 INFO (recoveryExecutor-1510-thread-1-processing-127.0.0.1:42669_solr perReplicaState_test_shard1_replica_n9 perReplicaState_test shard1 core_node10) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.h.IndexFetcher Follower's version: 0 2> 643542 INFO (recoveryExecutor-1510-thread-1-processing-127.0.0.1:42669_solr perReplicaState_test_shard1_replica_n9 perReplicaState_test shard1 core_node10) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.c.RecoveryStrategy No replay needed. 2> 643549 INFO (recoveryExecutor-1510-thread-1-processing-127.0.0.1:42669_solr perReplicaState_test_shard1_replica_n9 perReplicaState_test shard1 core_node10) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.c.RecoveryStrategy Replication Recovery was successful. 2> 643549 INFO (recoveryExecutor-1510-thread-1-processing-127.0.0.1:42669_solr perReplicaState_test_shard1_replica_n9 perReplicaState_test shard1 core_node10) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.c.RecoveryStrategy Registering as Active after recovery. 2> 643565 INFO (zkCallback-1511-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [4]) 2> 643565 INFO (zkCallback-1166-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [4]) 2> 643565 INFO (zkCallback-1164-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [4]) 2> 643571 INFO (zkCallback-1511-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [4]) 2> 643574 INFO (zkCallback-1511-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [4]) 2> 643578 INFO (zkCallback-1511-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [4]) 2> 643579 INFO (zkCallback-1160-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [4]) 2> 643581 INFO (recoveryExecutor-1510-thread-1-processing-127.0.0.1:42669_solr perReplicaState_test_shard1_replica_n9 perReplicaState_test shard1 core_node10) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.c.RecoveryStrategy Updating version bucket highest from index after successful recovery. 2> 643581 INFO (recoveryExecutor-1510-thread-1-processing-127.0.0.1:42669_solr perReplicaState_test_shard1_replica_n9 perReplicaState_test shard1 core_node10) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736809675587584 2> 643595 INFO (recoveryExecutor-1510-thread-1-processing-127.0.0.1:42669_solr perReplicaState_test_shard1_replica_n9 perReplicaState_test shard1 core_node10) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.c.RecoveryStrategy Finished recovery process, successful=[true] msTimeTaken=1043.0 2> 643595 INFO (recoveryExecutor-1510-thread-1-processing-127.0.0.1:42669_solr perReplicaState_test_shard1_replica_n9 perReplicaState_test shard1 core_node10) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.c.RecoveryStrategy Finished recovery process. recoveringAfterStartup=true msTimeTaken=1044.0 2> 643707 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000039 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 643828 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CreateCollectionCmd Create collection perReplicaState_testv2 2> 643889 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"perReplicaState_testv2_shard2_replica_n1", 2> "node_name":"127.0.0.1:40391_solr", 2> "base_url":"http://127.0.0.1:40391/solr", 2> "collection":"perReplicaState_testv2", 2> "shard":"shard2", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 643920 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"perReplicaState_testv2_shard1_replica_n3", 2> "node_name":"127.0.0.1:42669_solr", 2> "base_url":"http://127.0.0.1:42669/solr", 2> "collection":"perReplicaState_testv2", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 643943 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"perReplicaState_testv2_shard1_replica_n5", 2> "node_name":"127.0.0.1:44127_solr", 2> "base_url":"http://127.0.0.1:44127/solr", 2> "collection":"perReplicaState_testv2", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 643966 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"perReplicaState_testv2_shard2_replica_n7", 2> "node_name":"127.0.0.1:39159_solr", 2> "base_url":"http://127.0.0.1:39159/solr", 2> "collection":"perReplicaState_testv2", 2> "shard":"shard2", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 643991 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr x:perReplicaState_testv2_shard1_replica_n3] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=perReplicaState_testv2_shard1_replica_n3&action=CREATE&numShards=2&collection=perReplicaState_testv2&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 643993 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr x:perReplicaState_testv2_shard1_replica_n5] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf&newCollection=true&name=perReplicaState_testv2_shard1_replica_n5&action=CREATE&numShards=2&collection=perReplicaState_testv2&shard=shard1&wt=javabin&version=2&replicaType=NRT 2> 643994 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr x:perReplicaState_testv2_shard2_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=perReplicaState_testv2_shard2_replica_n1&action=CREATE&numShards=2&collection=perReplicaState_testv2&shard=shard2&wt=javabin&version=2&replicaType=NRT 2> 643996 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr x:perReplicaState_testv2_shard2_replica_n7] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf&newCollection=true&name=perReplicaState_testv2_shard2_replica_n7&action=CREATE&numShards=2&collection=perReplicaState_testv2&shard=shard2&wt=javabin&version=2&replicaType=NRT 2> 644195 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 644204 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.s.IndexSchema Schema name=test 2> 644213 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:perReplicaState_testv2 s:shard2 r:core_node8 x:perReplicaState_testv2_shard2_replica_n7] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 644216 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 644225 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:perReplicaState_testv2 s:shard2 r:core_node8 x:perReplicaState_testv2_shard2_replica_n7] o.a.s.s.IndexSchema Schema name=test 2> 644231 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.s.IndexSchema Schema name=test 2> 644239 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 644248 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.s.IndexSchema Schema name=test 2> 644263 WARN (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 644264 WARN (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 644264 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 644271 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'perReplicaState_testv2_shard1_replica_n3' using configuration from configset conf, trusted=true 2> 644275 WARN (qtp217923472-2529) [n:127.0.0.1:39159_solr c:perReplicaState_testv2 s:shard2 r:core_node8 x:perReplicaState_testv2_shard2_replica_n7] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 644275 WARN (qtp217923472-2529) [n:127.0.0.1:39159_solr c:perReplicaState_testv2 s:shard2 r:core_node8 x:perReplicaState_testv2_shard2_replica_n7] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 644276 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:perReplicaState_testv2 s:shard2 r:core_node8 x:perReplicaState_testv2_shard2_replica_n7] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 644278 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:perReplicaState_testv2 s:shard2 r:core_node8 x:perReplicaState_testv2_shard2_replica_n7] o.a.s.c.CoreContainer Creating SolrCore 'perReplicaState_testv2_shard2_replica_n7' using configuration from configset conf, trusted=true 2> 644278 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node4/perReplicaState_testv2_shard1_replica_n3], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node4/perReplicaState_testv2_shard1_replica_n3/data/] 2> 644282 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:perReplicaState_testv2 s:shard2 r:core_node8 x:perReplicaState_testv2_shard2_replica_n7] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/perReplicaState_testv2_shard2_replica_n7], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/perReplicaState_testv2_shard2_replica_n7/data/] 2> 644290 WARN (qtp218128001-2527) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 644290 WARN (qtp218128001-2527) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 644291 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 644294 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.c.CoreContainer Creating SolrCore 'perReplicaState_testv2_shard1_replica_n5' using configuration from configset conf, trusted=true 2> 644298 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node3/perReplicaState_testv2_shard1_replica_n5], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node3/perReplicaState_testv2_shard1_replica_n5/data/] 2> 644344 WARN (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 644344 WARN (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 644345 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 644348 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'perReplicaState_testv2_shard2_replica_n1' using configuration from configset conf, trusted=true 2> 644352 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/perReplicaState_testv2_shard2_replica_n1], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/perReplicaState_testv2_shard2_replica_n1/data/] 2> 644428 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=2 2> 644430 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 644431 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 644431 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 644431 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 644432 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 644797 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 644797 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 644828 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:perReplicaState_testv2 s:shard2 r:core_node8 x:perReplicaState_testv2_shard2_replica_n7] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 644829 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:perReplicaState_testv2 s:shard2 r:core_node8 x:perReplicaState_testv2_shard2_replica_n7] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 644841 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 644841 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 644845 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 644846 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 644878 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 644878 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 644898 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:perReplicaState_testv2 s:shard2 r:core_node8 x:perReplicaState_testv2_shard2_replica_n7] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 644899 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:perReplicaState_testv2 s:shard2 r:core_node8 x:perReplicaState_testv2_shard2_replica_n7] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 644976 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 644980 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 644980 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 644984 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 644992 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 644994 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736811157225472 2> 644999 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:perReplicaState_testv2 s:shard2 r:core_node8 x:perReplicaState_testv2_shard2_replica_n7] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 645003 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:perReplicaState_testv2 s:shard2 r:core_node8 x:perReplicaState_testv2_shard2_replica_n7] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 645027 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:perReplicaState_testv2 s:shard2 r:core_node8 x:perReplicaState_testv2_shard2_replica_n7] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 645031 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:perReplicaState_testv2 s:shard2 r:core_node8 x:perReplicaState_testv2_shard2_replica_n7] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736811196022784 2> 645033 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 645033 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 645035 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 645037 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736811202314240 2> 645085 INFO (searcherExecutor-1532-thread-1-processing-127.0.0.1:40391_solr perReplicaState_testv2_shard2_replica_n1 perReplicaState_testv2 shard2 core_node2) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 645100 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 645102 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 645095 INFO (searcherExecutor-1526-thread-1-processing-127.0.0.1:42669_solr perReplicaState_testv2_shard1_replica_n3 perReplicaState_testv2 shard1 core_node4) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 645138 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/perReplicaState_testv2/terms/shard2 to Terms{values={core_node2=0}, version=0} for registerTerm 2> 645138 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/perReplicaState_testv2/leaders/shard2 2> 645156 INFO (searcherExecutor-1528-thread-1-processing-127.0.0.1:39159_solr perReplicaState_testv2_shard2_replica_n7 perReplicaState_testv2 shard2 core_node8) [n:127.0.0.1:39159_solr c:perReplicaState_testv2 s:shard2 r:core_node8 x:perReplicaState_testv2_shard2_replica_n7] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 645172 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 645176 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 645188 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 645190 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736811362746368 2> 645220 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:perReplicaState_testv2 s:shard2 r:core_node8 x:perReplicaState_testv2_shard2_replica_n7] o.a.s.c.ZkShardTerms Successful update of terms at /collections/perReplicaState_testv2/terms/shard2 to Terms{values={core_node2=0, core_node8=0}, version=1} for registerTerm 2> 645221 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr c:perReplicaState_testv2 s:shard2 r:core_node8 x:perReplicaState_testv2_shard2_replica_n7] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/perReplicaState_testv2/leaders/shard2 2> 645228 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.c.ZkShardTerms Successful update of terms at /collections/perReplicaState_testv2/terms/shard1 to Terms{values={core_node4=0}, version=0} for registerTerm 2> 645228 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/perReplicaState_testv2/leaders/shard1 2> 645248 INFO (searcherExecutor-1530-thread-1-processing-127.0.0.1:44127_solr perReplicaState_testv2_shard1_replica_n5 perReplicaState_testv2 shard1 core_node6) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 645302 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 645302 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 645303 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40391/solr/perReplicaState_testv2_shard2_replica_n1/ 2> 645316 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.u.PeerSync PeerSync: core=perReplicaState_testv2_shard2_replica_n1 url=http://127.0.0.1:40391/solr START replicas=[http://127.0.0.1:39159/solr/perReplicaState_testv2_shard2_replica_n7/] nUpdates=100 2> 645325 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr c:perReplicaState_testv2 s:shard2 r:core_node8 x:perReplicaState_testv2_shard2_replica_n7] o.a.s.c.S.Request webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=3 2> 645331 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.u.PeerSync PeerSync: core=perReplicaState_testv2_shard2_replica_n1 url=http://127.0.0.1:40391/solr DONE. We have no versions. sync failed. 2> 645333 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate 2> 645333 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway 2> 645334 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/perReplicaState_testv2/leaders/shard2/leader after winning as /collections/perReplicaState_testv2/leader_elect/shard2/election/72081801657778183-core_node2-n_0000000000 2> 645349 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.c.ZkShardTerms Successful update of terms at /collections/perReplicaState_testv2/terms/shard1 to Terms{values={core_node6=0, core_node4=0}, version=1} for registerTerm 2> 645349 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/perReplicaState_testv2/leaders/shard1 2> 645389 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 645389 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 645390 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42669/solr/perReplicaState_testv2_shard1_replica_n3/ 2> 645398 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.u.PeerSync PeerSync: core=perReplicaState_testv2_shard1_replica_n3 url=http://127.0.0.1:42669/solr START replicas=[http://127.0.0.1:44127/solr/perReplicaState_testv2_shard1_replica_n5/] nUpdates=100 2> 645401 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40391/solr/perReplicaState_testv2_shard2_replica_n1/ shard2 2> 645408 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.u.PeerSync PeerSync: core=perReplicaState_testv2_shard1_replica_n3 url=http://127.0.0.1:42669/solr DONE. We have no versions. sync failed. 2> 645424 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.c.S.Request webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=2 2> 645427 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate 2> 645430 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway 2> 645430 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/perReplicaState_testv2/leaders/shard1/leader after winning as /collections/perReplicaState_testv2/leader_elect/shard1/election/72081801657778193-core_node4-n_0000000000 2> 645437 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=2 2> 645440 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 645440 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 645440 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 645440 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 645440 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 645456 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 645472 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42669/solr/perReplicaState_testv2_shard1_replica_n3/ shard1 2> 645538 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=perReplicaState_testv2_shard2_replica_n1&action=CREATE&numShards=2&collection=perReplicaState_testv2&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1553 2> 645544 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 645557 INFO (zkCallback-1164-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [4]) 2> 645610 INFO (qtp434474682-2867) [n:127.0.0.1:42669_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=perReplicaState_testv2_shard1_replica_n3&action=CREATE&numShards=2&collection=perReplicaState_testv2&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1623 2> 646322 INFO (zkCallback-1511-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [4]) 2> 646322 INFO (zkCallback-1164-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [4]) 2> 646362 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf&newCollection=true&name=perReplicaState_testv2_shard2_replica_n7&action=CREATE&numShards=2&collection=perReplicaState_testv2&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2370 2> 646437 INFO (zkCallback-1164-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [4]) 2> 646438 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [4]) 2> 646438 INFO (zkCallback-1511-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [4]) 2> 646446 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 646448 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 646448 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 646448 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 646448 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 646448 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 646492 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf&newCollection=true&name=perReplicaState_testv2_shard1_replica_n5&action=CREATE&numShards=2&collection=perReplicaState_testv2&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2503 2> 646517 INFO (qtp434474682-2869) [n:127.0.0.1:42669_solr] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas 2> 646518 INFO (qtp434474682-2869) [n:127.0.0.1:42669_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/collections params={wt=javabin&version=2} status=0 QTime=3884 2> 646520 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForActiveCollection: perReplicaState_testv2 2> 646569 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.e.j.s.Server Stopped Server@72c831f3{STOPPING}[10.0.13,sto=0] 2> 646585 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.e.j.s.AbstractConnector Stopped ServerConnector@26e657de{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0} 2> 646587 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1aaf7568{/solr,file:///build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-cwd/,STOPPED} 2> 646592 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2030076892 2> 646593 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:42669_solr 2> 646612 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkController Publish this node as DOWN... 2> 646612 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.c.ZkController Publish node=127.0.0.1:42669_solr as DOWN 2> 646614 INFO (zkCallback-1511-thread-4) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3) 2> 646615 INFO (zkCallback-1164-thread-3) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3) 2> 646616 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3) 2> 646618 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3) 2> 646620 INFO (zkCallback-1186-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3) 2> 646641 INFO (zkCallback-1164-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [3]) 2> 646641 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [3]) 2> 646641 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [3]) 2> 646645 INFO (zkCallback-1511-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [3]) 2> 646646 INFO (zkCallback-1511-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [3]) 2> 646646 INFO (zkCallback-1511-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [3]) 2> 646647 INFO (zkCallback-1511-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [3]) 2> 646663 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [3]) 2> 646663 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [3]) 2> 646664 INFO (zkCallback-1160-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [3]) 2> 646664 INFO (zkCallback-1166-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [3]) 2> 646673 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [3]) 2> 646674 INFO (zkCallback-1164-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [3]) 2> 646679 INFO (zkCallback-1511-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [3]) 2> 646680 INFO (zkCallback-1511-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [3]) 2> 646681 INFO (zkCallback-1511-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [3]) 2> 646681 INFO (zkCallback-1511-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [3]) 2> 646683 INFO (zkCallback-1511-thread-5) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [3]) 2> 646732 INFO (zkCallback-1160-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [3]) 2> 646733 INFO (zkCallback-1166-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [3]) 2> 646736 INFO (zkCallback-1511-thread-5) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [3]) 2> 646740 INFO (zkCallback-1164-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [3]) 2> 646753 INFO (zkCallback-1511-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [3]) 2> 646754 INFO (zkCallback-1166-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [3]) 2> 646754 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [3]) 2> 646756 INFO (zkCallback-1164-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [3]) 2> 646756 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [3]) 2> 646756 INFO (zkCallback-1160-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [3]) 2> 646756 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [3]) 2> 646760 INFO (zkCallback-1511-thread-5) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [3]) 2> 646825 INFO (coreCloseExecutor-1551-thread-1) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@1029b98c perReplicaState_test_shard1_replica_n9 2> 646826 INFO (coreCloseExecutor-1551-thread-2) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@4bbf370f perReplicaState_testv2_shard1_replica_n3 2> 646827 INFO (coreCloseExecutor-1551-thread-1) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.perReplicaState_test.shard1.replica_n9 tag=SolrCore@1029b98c 2> 647160 INFO (coreCloseExecutor-1551-thread-1) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.perReplicaState_test.shard1.leader tag=SolrCore@1029b98c 2> 647160 INFO (coreCloseExecutor-1551-thread-2) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.perReplicaState_testv2.shard1.replica_n3 tag=SolrCore@4bbf370f 2> 647169 INFO (coreCloseExecutor-1551-thread-1) [n:127.0.0.1:42669_solr c:perReplicaState_test s:shard1 r:core_node10 x:perReplicaState_test_shard1_replica_n9] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 647454 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=2 2> 647458 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 647458 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 647458 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 647458 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 647458 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 647480 INFO (coreCloseExecutor-1551-thread-2) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.perReplicaState_testv2.shard1.leader tag=SolrCore@4bbf370f 2> 647489 INFO (coreCloseExecutor-1551-thread-2) [n:127.0.0.1:42669_solr c:perReplicaState_testv2 s:shard1 r:core_node4 x:perReplicaState_testv2_shard1_replica_n3] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 647517 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null 2> 647687 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null 2> 647825 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null 2> 647826 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null 2> 647845 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000041 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 647890 INFO (zkCallback-1166-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [3]) 2> 647891 INFO (zkCallback-1166-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [3]) 2> 647896 INFO (zkCallback-1160-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [3]) 2> 647896 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [3]) 2> 647897 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [3]) 2> 647898 INFO (zkCallback-1164-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [3]) 2> 647924 INFO (zkCallback-1166-thread-2) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 647986 INFO (TEST-CloudSolrClientTest.testPerReplicaStateCollection-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Ending testPerReplicaStateCollection 2> 648469 INFO (TEST-CloudSolrClientTest.queryWithLocalShardsPreferenceRulesTest-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Starting queryWithLocalShardsPreferenceRulesTest 2> 648472 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 648474 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 648475 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 648475 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 648475 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 648475 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 648501 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=5570a893-eeb3-4a6f-b218-de371c7a71b3&collection.configName=conf&name=localShardsTestColl&nrtReplicas=3&action=CREATE&numShards=3&wt=javabin&version=2} status=0 QTime=26 2> 648525 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=15 2> 648530 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CreateCollectionCmd Create collection localShardsTestColl 2> 648838 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"localShardsTestColl_shard2_replica_n1", 2> "node_name":"127.0.0.1:44127_solr", 2> "base_url":"http://127.0.0.1:44127/solr", 2> "collection":"localShardsTestColl", 2> "shard":"shard2", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 648865 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"localShardsTestColl_shard2_replica_n2", 2> "node_name":"127.0.0.1:39159_solr", 2> "base_url":"http://127.0.0.1:39159/solr", 2> "collection":"localShardsTestColl", 2> "shard":"shard2", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 648879 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"localShardsTestColl_shard3_replica_n3", 2> "node_name":"127.0.0.1:44127_solr", 2> "base_url":"http://127.0.0.1:44127/solr", 2> "collection":"localShardsTestColl", 2> "shard":"shard3", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 648898 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"localShardsTestColl_shard1_replica_n4", 2> "node_name":"127.0.0.1:40391_solr", 2> "base_url":"http://127.0.0.1:40391/solr", 2> "collection":"localShardsTestColl", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 648917 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"localShardsTestColl_shard1_replica_n5", 2> "node_name":"127.0.0.1:44127_solr", 2> "base_url":"http://127.0.0.1:44127/solr", 2> "collection":"localShardsTestColl", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 648931 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"localShardsTestColl_shard3_replica_n6", 2> "node_name":"127.0.0.1:39159_solr", 2> "base_url":"http://127.0.0.1:39159/solr", 2> "collection":"localShardsTestColl", 2> "shard":"shard3", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 648944 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"localShardsTestColl_shard1_replica_n7", 2> "node_name":"127.0.0.1:39159_solr", 2> "base_url":"http://127.0.0.1:39159/solr", 2> "collection":"localShardsTestColl", 2> "shard":"shard1", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 648956 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"localShardsTestColl_shard2_replica_n8", 2> "node_name":"127.0.0.1:40391_solr", 2> "base_url":"http://127.0.0.1:40391/solr", 2> "collection":"localShardsTestColl", 2> "shard":"shard2", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 648968 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.o.SliceMutator createReplica() { 2> "core":"localShardsTestColl_shard3_replica_n9", 2> "node_name":"127.0.0.1:40391_solr", 2> "base_url":"http://127.0.0.1:40391/solr", 2> "collection":"localShardsTestColl", 2> "shard":"shard3", 2> "state":"down", 2> "type":"NRT", 2> "operation":"ADDREPLICA", 2> "waitForFinalState":"false"} 2> 649006 INFO (qtp218128001-2525) [n:127.0.0.1:44127_solr x:localShardsTestColl_shard2_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=localShardsTestColl&version=2&replicaType=NRT&async=5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542&coreNodeName=core_node10&name=localShardsTestColl_shard2_replica_n1&action=CREATE&numShards=3&shard=shard2&wt=javabin} status=0 QTime=3 2> 649007 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr x:localShardsTestColl_shard2_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=localShardsTestColl&version=2&replicaType=NRT&async=5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542&coreNodeName=core_node10&name=localShardsTestColl_shard2_replica_n1&action=CREATE&numShards=3&shard=shard2&wt=javabin 2> 649011 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr x:localShardsTestColl_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=localShardsTestColl&version=2&replicaType=NRT&async=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&coreNodeName=core_node11&name=localShardsTestColl_shard2_replica_n2&action=CREATE&numShards=3&shard=shard2&wt=javabin} status=0 QTime=6 2> 649020 INFO (qtp218128001-2523) [n:127.0.0.1:44127_solr x:localShardsTestColl_shard3_replica_n3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=localShardsTestColl&version=2&replicaType=NRT&async=5570a893-eeb3-4a6f-b218-de371c7a71b3369451156746236&coreNodeName=core_node12&name=localShardsTestColl_shard3_replica_n3&action=CREATE&numShards=3&shard=shard3&wt=javabin} status=0 QTime=6 2> 649020 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-12-processing-127.0.0.1:39159_solr localShardsTestColl_shard2_replica_n2 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 create) [n:127.0.0.1:39159_solr x:localShardsTestColl_shard2_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=localShardsTestColl&version=2&replicaType=NRT&async=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&coreNodeName=core_node11&name=localShardsTestColl_shard2_replica_n2&action=CREATE&numShards=3&shard=shard2&wt=javabin 2> 649023 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-10-processing-127.0.0.1:44127_solr localShardsTestColl_shard3_replica_n3 5570a893-eeb3-4a6f-b218-de371c7a71b3369451156746236 create) [n:127.0.0.1:44127_solr x:localShardsTestColl_shard3_replica_n3] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=localShardsTestColl&version=2&replicaType=NRT&async=5570a893-eeb3-4a6f-b218-de371c7a71b3369451156746236&coreNodeName=core_node12&name=localShardsTestColl_shard3_replica_n3&action=CREATE&numShards=3&shard=shard3&wt=javabin 2> 649036 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr x:localShardsTestColl_shard1_replica_n4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=localShardsTestColl&version=2&replicaType=NRT&async=5570a893-eeb3-4a6f-b218-de371c7a71b3369451169187560&coreNodeName=core_node13&name=localShardsTestColl_shard1_replica_n4&action=CREATE&numShards=3&shard=shard1&wt=javabin} status=0 QTime=19 2> 649041 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr x:localShardsTestColl_shard1_replica_n7] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=localShardsTestColl&version=2&replicaType=NRT&async=5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409&coreNodeName=core_node16&name=localShardsTestColl_shard1_replica_n7&action=CREATE&numShards=3&shard=shard1&wt=javabin} status=0 QTime=12 2> 649041 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-11-processing-127.0.0.1:40391_solr localShardsTestColl_shard1_replica_n4 5570a893-eeb3-4a6f-b218-de371c7a71b3369451169187560 create) [n:127.0.0.1:40391_solr x:localShardsTestColl_shard1_replica_n4] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=localShardsTestColl&version=2&replicaType=NRT&async=5570a893-eeb3-4a6f-b218-de371c7a71b3369451169187560&coreNodeName=core_node13&name=localShardsTestColl_shard1_replica_n4&action=CREATE&numShards=3&shard=shard1&wt=javabin 2> 649048 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr x:localShardsTestColl_shard1_replica_n7] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=localShardsTestColl&version=2&replicaType=NRT&async=5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409&coreNodeName=core_node16&name=localShardsTestColl_shard1_replica_n7&action=CREATE&numShards=3&shard=shard1&wt=javabin 2> 649054 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr x:localShardsTestColl_shard2_replica_n8] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=localShardsTestColl&version=2&replicaType=NRT&async=5570a893-eeb3-4a6f-b218-de371c7a71b3369451229983812&coreNodeName=core_node17&name=localShardsTestColl_shard2_replica_n8&action=CREATE&numShards=3&shard=shard2&wt=javabin} status=0 QTime=25 2> 649054 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-12-processing-127.0.0.1:40391_solr localShardsTestColl_shard2_replica_n8 5570a893-eeb3-4a6f-b218-de371c7a71b3369451229983812 create) [n:127.0.0.1:40391_solr x:localShardsTestColl_shard2_replica_n8] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=localShardsTestColl&version=2&replicaType=NRT&async=5570a893-eeb3-4a6f-b218-de371c7a71b3369451229983812&coreNodeName=core_node17&name=localShardsTestColl_shard2_replica_n8&action=CREATE&numShards=3&shard=shard2&wt=javabin 2> 649058 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr x:localShardsTestColl_shard3_replica_n6] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=localShardsTestColl&version=2&replicaType=NRT&async=5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045&coreNodeName=core_node15&name=localShardsTestColl_shard3_replica_n6&action=CREATE&numShards=3&shard=shard3&wt=javabin} status=0 QTime=29 2> 649058 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr x:localShardsTestColl_shard3_replica_n6] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=localShardsTestColl&version=2&replicaType=NRT&async=5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045&coreNodeName=core_node15&name=localShardsTestColl_shard3_replica_n6&action=CREATE&numShards=3&shard=shard3&wt=javabin 2> 649064 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr x:localShardsTestColl_shard1_replica_n5] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=localShardsTestColl&version=2&replicaType=NRT&async=5570a893-eeb3-4a6f-b218-de371c7a71b3369451184841301&coreNodeName=core_node14&name=localShardsTestColl_shard1_replica_n5&action=CREATE&numShards=3&shard=shard1&wt=javabin} status=0 QTime=47 2> 649067 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-11-processing-127.0.0.1:44127_solr localShardsTestColl_shard1_replica_n5 5570a893-eeb3-4a6f-b218-de371c7a71b3369451184841301 create) [n:127.0.0.1:44127_solr x:localShardsTestColl_shard1_replica_n5] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=localShardsTestColl&version=2&replicaType=NRT&async=5570a893-eeb3-4a6f-b218-de371c7a71b3369451184841301&coreNodeName=core_node14&name=localShardsTestColl_shard1_replica_n5&action=CREATE&numShards=3&shard=shard1&wt=javabin 2> 649069 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr x:localShardsTestColl_shard3_replica_n9] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=localShardsTestColl&version=2&replicaType=NRT&async=5570a893-eeb3-4a6f-b218-de371c7a71b3369451246322238&coreNodeName=core_node18&name=localShardsTestColl_shard3_replica_n9&action=CREATE&numShards=3&shard=shard3&wt=javabin} status=0 QTime=33 2> 649070 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-13-processing-127.0.0.1:40391_solr localShardsTestColl_shard3_replica_n9 5570a893-eeb3-4a6f-b218-de371c7a71b3369451246322238 create) [n:127.0.0.1:40391_solr x:localShardsTestColl_shard3_replica_n9] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=localShardsTestColl&version=2&replicaType=NRT&async=5570a893-eeb3-4a6f-b218-de371c7a71b3369451246322238&coreNodeName=core_node18&name=localShardsTestColl_shard3_replica_n9&action=CREATE&numShards=3&shard=shard3&wt=javabin 2> 649085 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 2> 649086 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 649118 INFO (zkCallback-1164-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [3]) 2> 649119 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [3]) 2> 649313 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 649328 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.s.IndexSchema Schema name=test 2> 649355 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 649361 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-13-processing-127.0.0.1:40391_solr localShardsTestColl_shard3_replica_n9 5570a893-eeb3-4a6f-b218-de371c7a71b3369451246322238 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n9] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 649371 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.s.IndexSchema Schema name=test 2> 649374 WARN (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 649374 WARN (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 649375 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 649377 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'localShardsTestColl_shard3_replica_n6' using configuration from configset conf, trusted=true 2> 649381 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/localShardsTestColl_shard3_replica_n6], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/localShardsTestColl_shard3_replica_n6/data/] 2> 649384 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-13-processing-127.0.0.1:40391_solr localShardsTestColl_shard3_replica_n9 5570a893-eeb3-4a6f-b218-de371c7a71b3369451246322238 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n9] o.a.s.s.IndexSchema Schema name=test 2> 649391 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 649403 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.s.IndexSchema Schema name=test 2> 649410 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-12-processing-127.0.0.1:39159_solr localShardsTestColl_shard2_replica_n2 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard2 r:core_node11 x:localShardsTestColl_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 649432 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-12-processing-127.0.0.1:39159_solr localShardsTestColl_shard2_replica_n2 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard2 r:core_node11 x:localShardsTestColl_shard2_replica_n2] o.a.s.s.IndexSchema Schema name=test 2> 649452 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-10-processing-127.0.0.1:44127_solr localShardsTestColl_shard3_replica_n3 5570a893-eeb3-4a6f-b218-de371c7a71b3369451156746236 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard3 r:core_node12 x:localShardsTestColl_shard3_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 649480 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=2 2> 649482 WARN (parallelCoreAdminAPIBaseExecutor-1179-thread-13-processing-127.0.0.1:40391_solr localShardsTestColl_shard3_replica_n9 5570a893-eeb3-4a6f-b218-de371c7a71b3369451246322238 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n9] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 649482 WARN (parallelCoreAdminAPIBaseExecutor-1179-thread-13-processing-127.0.0.1:40391_solr localShardsTestColl_shard3_replica_n9 5570a893-eeb3-4a6f-b218-de371c7a71b3369451246322238 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n9] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 649483 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-13-processing-127.0.0.1:40391_solr localShardsTestColl_shard3_replica_n9 5570a893-eeb3-4a6f-b218-de371c7a71b3369451246322238 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n9] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 649483 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 649483 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 649483 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 649484 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 649484 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 649485 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-13-processing-127.0.0.1:40391_solr localShardsTestColl_shard3_replica_n9 5570a893-eeb3-4a6f-b218-de371c7a71b3369451246322238 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n9] o.a.s.c.CoreContainer Creating SolrCore 'localShardsTestColl_shard3_replica_n9' using configuration from configset conf, trusted=true 2> 649490 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-10-processing-127.0.0.1:44127_solr localShardsTestColl_shard3_replica_n3 5570a893-eeb3-4a6f-b218-de371c7a71b3369451156746236 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard3 r:core_node12 x:localShardsTestColl_shard3_replica_n3] o.a.s.s.IndexSchema Schema name=test 2> 649498 WARN (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 649498 WARN (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 649499 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 649503 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-11-processing-127.0.0.1:40391_solr localShardsTestColl_shard1_replica_n4 5570a893-eeb3-4a6f-b218-de371c7a71b3369451169187560 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard1 r:core_node13 x:localShardsTestColl_shard1_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 649504 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'localShardsTestColl_shard2_replica_n1' using configuration from configset conf, trusted=true 2> 649507 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node3/localShardsTestColl_shard2_replica_n1], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node3/localShardsTestColl_shard2_replica_n1/data/] 2> 649509 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-11-processing-127.0.0.1:44127_solr localShardsTestColl_shard1_replica_n5 5570a893-eeb3-4a6f-b218-de371c7a71b3369451184841301 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard1 r:core_node14 x:localShardsTestColl_shard1_replica_n5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 649510 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-13-processing-127.0.0.1:40391_solr localShardsTestColl_shard3_replica_n9 5570a893-eeb3-4a6f-b218-de371c7a71b3369451246322238 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n9] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/localShardsTestColl_shard3_replica_n9], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/localShardsTestColl_shard3_replica_n9/data/] 2> 649517 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-11-processing-127.0.0.1:40391_solr localShardsTestColl_shard1_replica_n4 5570a893-eeb3-4a6f-b218-de371c7a71b3369451169187560 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard1 r:core_node13 x:localShardsTestColl_shard1_replica_n4] o.a.s.s.IndexSchema Schema name=test 2> 649534 WARN (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 649539 WARN (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 649541 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 649544 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.c.CoreContainer Creating SolrCore 'localShardsTestColl_shard1_replica_n7' using configuration from configset conf, trusted=true 2> 649547 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-11-processing-127.0.0.1:44127_solr localShardsTestColl_shard1_replica_n5 5570a893-eeb3-4a6f-b218-de371c7a71b3369451184841301 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard1 r:core_node14 x:localShardsTestColl_shard1_replica_n5] o.a.s.s.IndexSchema Schema name=test 2> 649546 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-12-processing-127.0.0.1:40391_solr localShardsTestColl_shard2_replica_n8 5570a893-eeb3-4a6f-b218-de371c7a71b3369451229983812 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard2 r:core_node17 x:localShardsTestColl_shard2_replica_n8] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.4.2 2> 649560 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/localShardsTestColl_shard1_replica_n7], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/localShardsTestColl_shard1_replica_n7/data/] 2> 649545 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=14 2> 649586 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-12-processing-127.0.0.1:40391_solr localShardsTestColl_shard2_replica_n8 5570a893-eeb3-4a6f-b218-de371c7a71b3369451229983812 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard2 r:core_node17 x:localShardsTestColl_shard2_replica_n8] o.a.s.s.IndexSchema Schema name=test 2> 649612 WARN (parallelCoreAdminAPIBaseExecutor-1177-thread-12-processing-127.0.0.1:39159_solr localShardsTestColl_shard2_replica_n2 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard2 r:core_node11 x:localShardsTestColl_shard2_replica_n2] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 649613 WARN (parallelCoreAdminAPIBaseExecutor-1177-thread-12-processing-127.0.0.1:39159_solr localShardsTestColl_shard2_replica_n2 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard2 r:core_node11 x:localShardsTestColl_shard2_replica_n2] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 649616 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-12-processing-127.0.0.1:39159_solr localShardsTestColl_shard2_replica_n2 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard2 r:core_node11 x:localShardsTestColl_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 649624 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-12-processing-127.0.0.1:39159_solr localShardsTestColl_shard2_replica_n2 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard2 r:core_node11 x:localShardsTestColl_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'localShardsTestColl_shard2_replica_n2' using configuration from configset conf, trusted=true 2> 649630 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-12-processing-127.0.0.1:39159_solr localShardsTestColl_shard2_replica_n2 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard2 r:core_node11 x:localShardsTestColl_shard2_replica_n2] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/localShardsTestColl_shard2_replica_n2], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node2/localShardsTestColl_shard2_replica_n2/data/] 2> 649644 WARN (parallelCoreAdminAPIBaseExecutor-1178-thread-10-processing-127.0.0.1:44127_solr localShardsTestColl_shard3_replica_n3 5570a893-eeb3-4a6f-b218-de371c7a71b3369451156746236 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard3 r:core_node12 x:localShardsTestColl_shard3_replica_n3] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 649645 WARN (parallelCoreAdminAPIBaseExecutor-1178-thread-10-processing-127.0.0.1:44127_solr localShardsTestColl_shard3_replica_n3 5570a893-eeb3-4a6f-b218-de371c7a71b3369451156746236 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard3 r:core_node12 x:localShardsTestColl_shard3_replica_n3] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 649645 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-10-processing-127.0.0.1:44127_solr localShardsTestColl_shard3_replica_n3 5570a893-eeb3-4a6f-b218-de371c7a71b3369451156746236 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard3 r:core_node12 x:localShardsTestColl_shard3_replica_n3] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 649649 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-10-processing-127.0.0.1:44127_solr localShardsTestColl_shard3_replica_n3 5570a893-eeb3-4a6f-b218-de371c7a71b3369451156746236 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard3 r:core_node12 x:localShardsTestColl_shard3_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'localShardsTestColl_shard3_replica_n3' using configuration from configset conf, trusted=true 2> 649654 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-10-processing-127.0.0.1:44127_solr localShardsTestColl_shard3_replica_n3 5570a893-eeb3-4a6f-b218-de371c7a71b3369451156746236 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard3 r:core_node12 x:localShardsTestColl_shard3_replica_n3] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node3/localShardsTestColl_shard3_replica_n3], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node3/localShardsTestColl_shard3_replica_n3/data/] 2> 649660 WARN (parallelCoreAdminAPIBaseExecutor-1179-thread-11-processing-127.0.0.1:40391_solr localShardsTestColl_shard1_replica_n4 5570a893-eeb3-4a6f-b218-de371c7a71b3369451169187560 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard1 r:core_node13 x:localShardsTestColl_shard1_replica_n4] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 649661 WARN (parallelCoreAdminAPIBaseExecutor-1179-thread-11-processing-127.0.0.1:40391_solr localShardsTestColl_shard1_replica_n4 5570a893-eeb3-4a6f-b218-de371c7a71b3369451169187560 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard1 r:core_node13 x:localShardsTestColl_shard1_replica_n4] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 649661 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-11-processing-127.0.0.1:40391_solr localShardsTestColl_shard1_replica_n4 5570a893-eeb3-4a6f-b218-de371c7a71b3369451169187560 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard1 r:core_node13 x:localShardsTestColl_shard1_replica_n4] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 649664 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-11-processing-127.0.0.1:40391_solr localShardsTestColl_shard1_replica_n4 5570a893-eeb3-4a6f-b218-de371c7a71b3369451169187560 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard1 r:core_node13 x:localShardsTestColl_shard1_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'localShardsTestColl_shard1_replica_n4' using configuration from configset conf, trusted=true 2> 649672 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-11-processing-127.0.0.1:40391_solr localShardsTestColl_shard1_replica_n4 5570a893-eeb3-4a6f-b218-de371c7a71b3369451169187560 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard1 r:core_node13 x:localShardsTestColl_shard1_replica_n4] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/localShardsTestColl_shard1_replica_n4], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/localShardsTestColl_shard1_replica_n4/data/] 2> 649746 WARN (parallelCoreAdminAPIBaseExecutor-1179-thread-12-processing-127.0.0.1:40391_solr localShardsTestColl_shard2_replica_n8 5570a893-eeb3-4a6f-b218-de371c7a71b3369451229983812 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard2 r:core_node17 x:localShardsTestColl_shard2_replica_n8] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 649747 WARN (parallelCoreAdminAPIBaseExecutor-1179-thread-12-processing-127.0.0.1:40391_solr localShardsTestColl_shard2_replica_n8 5570a893-eeb3-4a6f-b218-de371c7a71b3369451229983812 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard2 r:core_node17 x:localShardsTestColl_shard2_replica_n8] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 649751 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-12-processing-127.0.0.1:40391_solr localShardsTestColl_shard2_replica_n8 5570a893-eeb3-4a6f-b218-de371c7a71b3369451229983812 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard2 r:core_node17 x:localShardsTestColl_shard2_replica_n8] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 649765 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-12-processing-127.0.0.1:40391_solr localShardsTestColl_shard2_replica_n8 5570a893-eeb3-4a6f-b218-de371c7a71b3369451229983812 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard2 r:core_node17 x:localShardsTestColl_shard2_replica_n8] o.a.s.c.CoreContainer Creating SolrCore 'localShardsTestColl_shard2_replica_n8' using configuration from configset conf, trusted=true 2> 649769 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-12-processing-127.0.0.1:40391_solr localShardsTestColl_shard2_replica_n8 5570a893-eeb3-4a6f-b218-de371c7a71b3369451229983812 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard2 r:core_node17 x:localShardsTestColl_shard2_replica_n8] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/localShardsTestColl_shard2_replica_n8], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node1/localShardsTestColl_shard2_replica_n8/data/] 2> 649771 WARN (parallelCoreAdminAPIBaseExecutor-1178-thread-11-processing-127.0.0.1:44127_solr localShardsTestColl_shard1_replica_n5 5570a893-eeb3-4a6f-b218-de371c7a71b3369451184841301 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard1 r:core_node14 x:localShardsTestColl_shard1_replica_n5] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2) 2> 649772 WARN (parallelCoreAdminAPIBaseExecutor-1178-thread-11-processing-127.0.0.1:44127_solr localShardsTestColl_shard1_replica_n5 5570a893-eeb3-4a6f-b218-de371c7a71b3369451184841301 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard1 r:core_node14 x:localShardsTestColl_shard1_replica_n5] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3) 2> 649773 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-11-processing-127.0.0.1:44127_solr localShardsTestColl_shard1_replica_n5 5570a893-eeb3-4a6f-b218-de371c7a71b3369451184841301 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard1 r:core_node14 x:localShardsTestColl_shard1_replica_n5] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id 2> 649786 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-11-processing-127.0.0.1:44127_solr localShardsTestColl_shard1_replica_n5 5570a893-eeb3-4a6f-b218-de371c7a71b3369451184841301 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard1 r:core_node14 x:localShardsTestColl_shard1_replica_n5] o.a.s.c.CoreContainer Creating SolrCore 'localShardsTestColl_shard1_replica_n5' using configuration from configset conf, trusted=true 2> 649811 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-11-processing-127.0.0.1:44127_solr localShardsTestColl_shard1_replica_n5 5570a893-eeb3-4a6f-b218-de371c7a71b3369451184841301 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard1 r:core_node14 x:localShardsTestColl_shard1_replica_n5] o.a.s.c.SolrCore Opening new SolrCore at [/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node3/localShardsTestColl_shard1_replica_n5], dataDir=[/build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001/tempDir-001/node3/localShardsTestColl_shard1_replica_n5/data/] 2> 655380 INFO (zkCallback-1166-thread-2) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:44127/solr/perReplicaState_testv2_shard1_replica_n5/ 2> 655421 INFO (zkCallback-1166-thread-2) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me 2> 655423 INFO (zkCallback-1166-thread-2) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.c.SyncStrategy http://127.0.0.1:44127/solr/perReplicaState_testv2_shard1_replica_n5/ has no replicas 2> 655423 INFO (zkCallback-1166-thread-2) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/perReplicaState_testv2/leaders/shard1/leader after winning as /collections/perReplicaState_testv2/leader_elect/shard1/election/72081801657778184-core_node6-n_0000000001 2> 655455 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=19 2> 655455 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 2> 655461 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=23 2> 655469 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=39 2> 655480 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 655481 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 655481 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 655481 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 655481 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 655484 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [3]) 2> 655486 INFO (zkCallback-1166-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [3]) 2> 655487 INFO (zkCallback-1164-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [3]) 2> 655504 INFO (zkCallback-1166-thread-2) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:44127/solr/perReplicaState_testv2_shard1_replica_n5/ shard1 2> 655944 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 655945 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 655985 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 655986 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 656081 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 656087 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 656121 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 656123 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736822826827776 2> 656205 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.c.ZkShardTerms Successful update of terms at /collections/localShardsTestColl/terms/shard3 to Terms{values={core_node15=0}, version=0} for registerTerm 2> 656206 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/localShardsTestColl/leaders/shard3 2> 656236 INFO (searcherExecutor-1554-thread-1-processing-localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 core_node15 create 127.0.0.1:39159_solr localShardsTestColl shard3) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 656286 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 656286 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 656289 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard3: total=3 found=1 timeoute in=9990ms 2> 656331 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-13-processing-127.0.0.1:40391_solr localShardsTestColl_shard3_replica_n9 5570a893-eeb3-4a6f-b218-de371c7a71b3369451246322238 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n9] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 656331 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-13-processing-127.0.0.1:40391_solr localShardsTestColl_shard3_replica_n9 5570a893-eeb3-4a6f-b218-de371c7a71b3369451246322238 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n9] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 656344 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 656344 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 656373 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-13-processing-127.0.0.1:40391_solr localShardsTestColl_shard3_replica_n9 5570a893-eeb3-4a6f-b218-de371c7a71b3369451246322238 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n9] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 656374 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-13-processing-127.0.0.1:40391_solr localShardsTestColl_shard3_replica_n9 5570a893-eeb3-4a6f-b218-de371c7a71b3369451246322238 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n9] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 656422 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 656423 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 656430 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 656435 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 656459 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-13-processing-127.0.0.1:40391_solr localShardsTestColl_shard3_replica_n9 5570a893-eeb3-4a6f-b218-de371c7a71b3369451246322238 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n9] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 656464 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-13-processing-127.0.0.1:40391_solr localShardsTestColl_shard3_replica_n9 5570a893-eeb3-4a6f-b218-de371c7a71b3369451246322238 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n9] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 656471 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 656473 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736823193829376 2> 656497 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 656498 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 656498 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 656499 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 656499 INFO (qtp218128001-2528) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 2> 656499 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 656499 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 656499 INFO (qtp218128001-2528) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 656502 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-13-processing-127.0.0.1:40391_solr localShardsTestColl_shard3_replica_n9 5570a893-eeb3-4a6f-b218-de371c7a71b3369451246322238 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n9] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 656504 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 656504 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 656505 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-13-processing-127.0.0.1:40391_solr localShardsTestColl_shard3_replica_n9 5570a893-eeb3-4a6f-b218-de371c7a71b3369451246322238 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n9] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736823226335232 2> 656515 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=30 2> 656579 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/localShardsTestColl/terms/shard2 to Terms{values={core_node10=0}, version=0} for registerTerm 2> 656580 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/localShardsTestColl/leaders/shard2 2> 656630 INFO (searcherExecutor-1558-thread-1-processing-localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 core_node10 create 127.0.0.1:44127_solr localShardsTestColl shard2) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 656636 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-13-processing-127.0.0.1:40391_solr localShardsTestColl_shard3_replica_n9 5570a893-eeb3-4a6f-b218-de371c7a71b3369451246322238 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n9] o.a.s.c.ZkShardTerms Successful update of terms at /collections/localShardsTestColl/terms/shard3 to Terms{values={core_node18=0, core_node15=0}, version=1} for registerTerm 2> 656638 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-13-processing-127.0.0.1:40391_solr localShardsTestColl_shard3_replica_n9 5570a893-eeb3-4a6f-b218-de371c7a71b3369451246322238 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n9] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/localShardsTestColl/leaders/shard3 2> 656643 INFO (searcherExecutor-1556-thread-1-processing-localShardsTestColl_shard3_replica_n9 5570a893-eeb3-4a6f-b218-de371c7a71b3369451246322238 core_node18 create 127.0.0.1:40391_solr localShardsTestColl shard3) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n9] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 656654 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 656663 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 656680 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 656681 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736823411933184 2> 656691 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=3 found=1 timeoute in=9986ms 2> 656695 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-10-processing-127.0.0.1:44127_solr localShardsTestColl_shard3_replica_n3 5570a893-eeb3-4a6f-b218-de371c7a71b3369451156746236 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard3 r:core_node12 x:localShardsTestColl_shard3_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 656695 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-10-processing-127.0.0.1:44127_solr localShardsTestColl_shard3_replica_n3 5570a893-eeb3-4a6f-b218-de371c7a71b3369451156746236 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard3 r:core_node12 x:localShardsTestColl_shard3_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 656748 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-11-processing-127.0.0.1:40391_solr localShardsTestColl_shard1_replica_n4 5570a893-eeb3-4a6f-b218-de371c7a71b3369451169187560 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard1 r:core_node13 x:localShardsTestColl_shard1_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 656748 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-11-processing-127.0.0.1:40391_solr localShardsTestColl_shard1_replica_n4 5570a893-eeb3-4a6f-b218-de371c7a71b3369451169187560 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard1 r:core_node13 x:localShardsTestColl_shard1_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 656752 INFO (searcherExecutor-1560-thread-1-processing-localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 core_node16 create 127.0.0.1:39159_solr localShardsTestColl shard1) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 656757 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-12-processing-127.0.0.1:40391_solr localShardsTestColl_shard2_replica_n8 5570a893-eeb3-4a6f-b218-de371c7a71b3369451229983812 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard2 r:core_node17 x:localShardsTestColl_shard2_replica_n8] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 656757 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-12-processing-127.0.0.1:40391_solr localShardsTestColl_shard2_replica_n8 5570a893-eeb3-4a6f-b218-de371c7a71b3369451229983812 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard2 r:core_node17 x:localShardsTestColl_shard2_replica_n8] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 656777 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-11-processing-127.0.0.1:44127_solr localShardsTestColl_shard1_replica_n5 5570a893-eeb3-4a6f-b218-de371c7a71b3369451184841301 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard1 r:core_node14 x:localShardsTestColl_shard1_replica_n5] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog 2> 656777 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-11-processing-127.0.0.1:44127_solr localShardsTestColl_shard1_replica_n5 5570a893-eeb3-4a6f-b218-de371c7a71b3369451184841301 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard1 r:core_node14 x:localShardsTestColl_shard1_replica_n5] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 2> 656782 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-10-processing-127.0.0.1:44127_solr localShardsTestColl_shard3_replica_n3 5570a893-eeb3-4a6f-b218-de371c7a71b3369451156746236 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard3 r:core_node12 x:localShardsTestColl_shard3_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 656782 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-10-processing-127.0.0.1:44127_solr localShardsTestColl_shard3_replica_n3 5570a893-eeb3-4a6f-b218-de371c7a71b3369451156746236 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard3 r:core_node12 x:localShardsTestColl_shard3_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 656786 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.c.ZkShardTerms Successful update of terms at /collections/localShardsTestColl/terms/shard1 to Terms{values={core_node16=0}, version=0} for registerTerm 2> 656786 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/localShardsTestColl/leaders/shard1 2> 656806 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-11-processing-127.0.0.1:40391_solr localShardsTestColl_shard1_replica_n4 5570a893-eeb3-4a6f-b218-de371c7a71b3369451169187560 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard1 r:core_node13 x:localShardsTestColl_shard1_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 656807 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-11-processing-127.0.0.1:40391_solr localShardsTestColl_shard1_replica_n4 5570a893-eeb3-4a6f-b218-de371c7a71b3369451169187560 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard1 r:core_node13 x:localShardsTestColl_shard1_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 656815 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-12-processing-127.0.0.1:40391_solr localShardsTestColl_shard2_replica_n8 5570a893-eeb3-4a6f-b218-de371c7a71b3369451229983812 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard2 r:core_node17 x:localShardsTestColl_shard2_replica_n8] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 656816 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-12-processing-127.0.0.1:40391_solr localShardsTestColl_shard2_replica_n8 5570a893-eeb3-4a6f-b218-de371c7a71b3369451229983812 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard2 r:core_node17 x:localShardsTestColl_shard2_replica_n8] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 656832 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-11-processing-127.0.0.1:44127_solr localShardsTestColl_shard1_replica_n5 5570a893-eeb3-4a6f-b218-de371c7a71b3369451184841301 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard1 r:core_node14 x:localShardsTestColl_shard1_replica_n5] o.a.s.u.CommitTracker Hard AutoCommit: disabled 2> 656832 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-11-processing-127.0.0.1:44127_solr localShardsTestColl_shard1_replica_n5 5570a893-eeb3-4a6f-b218-de371c7a71b3369451184841301 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard1 r:core_node14 x:localShardsTestColl_shard1_replica_n5] o.a.s.u.CommitTracker Soft AutoCommit: disabled 2> 656892 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-10-processing-127.0.0.1:44127_solr localShardsTestColl_shard3_replica_n3 5570a893-eeb3-4a6f-b218-de371c7a71b3369451156746236 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard3 r:core_node12 x:localShardsTestColl_shard3_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 656898 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=3 found=1 timeoute in=9957ms 2> 656898 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-10-processing-127.0.0.1:44127_solr localShardsTestColl_shard3_replica_n3 5570a893-eeb3-4a6f-b218-de371c7a71b3369451156746236 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard3 r:core_node12 x:localShardsTestColl_shard3_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 656905 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-10-processing-127.0.0.1:44127_solr localShardsTestColl_shard3_replica_n3 5570a893-eeb3-4a6f-b218-de371c7a71b3369451156746236 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard3 r:core_node12 x:localShardsTestColl_shard3_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 656913 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-10-processing-127.0.0.1:44127_solr localShardsTestColl_shard3_replica_n3 5570a893-eeb3-4a6f-b218-de371c7a71b3369451156746236 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard3 r:core_node12 x:localShardsTestColl_shard3_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736823655202816 2> 656914 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-11-processing-127.0.0.1:40391_solr localShardsTestColl_shard1_replica_n4 5570a893-eeb3-4a6f-b218-de371c7a71b3369451169187560 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard1 r:core_node13 x:localShardsTestColl_shard1_replica_n4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 656920 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-11-processing-127.0.0.1:44127_solr localShardsTestColl_shard1_replica_n5 5570a893-eeb3-4a6f-b218-de371c7a71b3369451184841301 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard1 r:core_node14 x:localShardsTestColl_shard1_replica_n5] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 656924 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-12-processing-127.0.0.1:40391_solr localShardsTestColl_shard2_replica_n8 5570a893-eeb3-4a6f-b218-de371c7a71b3369451229983812 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard2 r:core_node17 x:localShardsTestColl_shard2_replica_n8] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf 2> 656928 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-12-processing-127.0.0.1:40391_solr localShardsTestColl_shard2_replica_n8 5570a893-eeb3-4a6f-b218-de371c7a71b3369451229983812 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard2 r:core_node17 x:localShardsTestColl_shard2_replica_n8] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 656937 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-11-processing-127.0.0.1:44127_solr localShardsTestColl_shard1_replica_n5 5570a893-eeb3-4a6f-b218-de371c7a71b3369451184841301 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard1 r:core_node14 x:localShardsTestColl_shard1_replica_n5] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 656940 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-12-processing-127.0.0.1:40391_solr localShardsTestColl_shard2_replica_n8 5570a893-eeb3-4a6f-b218-de371c7a71b3369451229983812 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard2 r:core_node17 x:localShardsTestColl_shard2_replica_n8] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 656942 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-12-processing-127.0.0.1:40391_solr localShardsTestColl_shard2_replica_n8 5570a893-eeb3-4a6f-b218-de371c7a71b3369451229983812 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard2 r:core_node17 x:localShardsTestColl_shard2_replica_n8] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736823685611520 2> 656947 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-11-processing-127.0.0.1:44127_solr localShardsTestColl_shard1_replica_n5 5570a893-eeb3-4a6f-b218-de371c7a71b3369451184841301 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard1 r:core_node14 x:localShardsTestColl_shard1_replica_n5] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 656949 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-11-processing-127.0.0.1:44127_solr localShardsTestColl_shard1_replica_n5 5570a893-eeb3-4a6f-b218-de371c7a71b3369451184841301 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard1 r:core_node14 x:localShardsTestColl_shard1_replica_n5] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736823692951552 2> 656939 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-11-processing-127.0.0.1:40391_solr localShardsTestColl_shard1_replica_n4 5570a893-eeb3-4a6f-b218-de371c7a71b3369451169187560 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard1 r:core_node13 x:localShardsTestColl_shard1_replica_n4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf 2> 656956 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-11-processing-127.0.0.1:40391_solr localShardsTestColl_shard1_replica_n4 5570a893-eeb3-4a6f-b218-de371c7a71b3369451169187560 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard1 r:core_node13 x:localShardsTestColl_shard1_replica_n4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms 2> 656958 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-11-processing-127.0.0.1:40391_solr localShardsTestColl_shard1_replica_n4 5570a893-eeb3-4a6f-b218-de371c7a71b3369451169187560 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard1 r:core_node13 x:localShardsTestColl_shard1_replica_n4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1778736823702388736 2> 656990 INFO (searcherExecutor-1569-thread-1-processing-localShardsTestColl_shard2_replica_n8 5570a893-eeb3-4a6f-b218-de371c7a71b3369451229983812 core_node17 create 127.0.0.1:40391_solr localShardsTestColl shard2) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard2 r:core_node17 x:localShardsTestColl_shard2_replica_n8] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 657002 INFO (searcherExecutor-1564-thread-1-processing-localShardsTestColl_shard3_replica_n3 5570a893-eeb3-4a6f-b218-de371c7a71b3369451156746236 core_node12 create 127.0.0.1:44127_solr localShardsTestColl shard3) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard3 r:core_node12 x:localShardsTestColl_shard3_replica_n3] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 657025 INFO (searcherExecutor-1571-thread-1-processing-localShardsTestColl_shard1_replica_n5 5570a893-eeb3-4a6f-b218-de371c7a71b3369451184841301 core_node14 create 127.0.0.1:44127_solr localShardsTestColl shard1) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard1 r:core_node14 x:localShardsTestColl_shard1_replica_n5] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 657060 INFO (searcherExecutor-1566-thread-1-processing-localShardsTestColl_shard1_replica_n4 5570a893-eeb3-4a6f-b218-de371c7a71b3369451169187560 core_node13 create 127.0.0.1:40391_solr localShardsTestColl shard1) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard1 r:core_node13 x:localShardsTestColl_shard1_replica_n4] o.a.s.c.SolrCore Registered new searcher autowarm time: 0 ms 2> 657077 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-10-processing-127.0.0.1:44127_solr localShardsTestColl_shard3_replica_n3 5570a893-eeb3-4a6f-b218-de371c7a71b3369451156746236 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard3 r:core_node12 x:localShardsTestColl_shard3_replica_n3] o.a.s.c.ZkShardTerms Successful update of terms at /collections/localShardsTestColl/terms/shard3 to Terms{values={core_node18=0, core_node12=0, core_node15=0}, version=2} for registerTerm 2> 657077 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-10-processing-127.0.0.1:44127_solr localShardsTestColl_shard3_replica_n3 5570a893-eeb3-4a6f-b218-de371c7a71b3369451156746236 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard3 r:core_node12 x:localShardsTestColl_shard3_replica_n3] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/localShardsTestColl/leaders/shard3 2> 657105 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-12-processing-127.0.0.1:40391_solr localShardsTestColl_shard2_replica_n8 5570a893-eeb3-4a6f-b218-de371c7a71b3369451229983812 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard2 r:core_node17 x:localShardsTestColl_shard2_replica_n8] o.a.s.c.ZkShardTerms Successful update of terms at /collections/localShardsTestColl/terms/shard2 to Terms{values={core_node10=0, core_node17=0}, version=1} for registerTerm 2> 657106 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-12-processing-127.0.0.1:40391_solr localShardsTestColl_shard2_replica_n8 5570a893-eeb3-4a6f-b218-de371c7a71b3369451229983812 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard2 r:core_node17 x:localShardsTestColl_shard2_replica_n8] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/localShardsTestColl/leaders/shard2 2> 657142 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-11-processing-127.0.0.1:44127_solr localShardsTestColl_shard1_replica_n5 5570a893-eeb3-4a6f-b218-de371c7a71b3369451184841301 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard1 r:core_node14 x:localShardsTestColl_shard1_replica_n5] o.a.s.c.ZkShardTerms Successful update of terms at /collections/localShardsTestColl/terms/shard1 to Terms{values={core_node14=0, core_node16=0}, version=1} for registerTerm 2> 657143 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-11-processing-127.0.0.1:44127_solr localShardsTestColl_shard1_replica_n5 5570a893-eeb3-4a6f-b218-de371c7a71b3369451184841301 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard1 r:core_node14 x:localShardsTestColl_shard1_replica_n5] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/localShardsTestColl/leaders/shard1 2> 657159 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-11-processing-127.0.0.1:40391_solr localShardsTestColl_shard1_replica_n4 5570a893-eeb3-4a6f-b218-de371c7a71b3369451169187560 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard1 r:core_node13 x:localShardsTestColl_shard1_replica_n4] o.a.s.c.ZkShardTerms Successful update of terms at /collections/localShardsTestColl/terms/shard1 to Terms{values={core_node14=0, core_node13=0, core_node16=0}, version=2} for registerTerm 2> 657160 INFO (parallelCoreAdminAPIBaseExecutor-1179-thread-11-processing-127.0.0.1:40391_solr localShardsTestColl_shard1_replica_n4 5570a893-eeb3-4a6f-b218-de371c7a71b3369451169187560 create) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard1 r:core_node13 x:localShardsTestColl_shard1_replica_n4] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/localShardsTestColl/leaders/shard1 2> 657306 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 657306 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 657306 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39159/solr/localShardsTestColl_shard3_replica_n6/ 2> 657311 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.u.PeerSync PeerSync: core=localShardsTestColl_shard3_replica_n6 url=http://127.0.0.1:39159/solr START replicas=[http://127.0.0.1:44127/solr/localShardsTestColl_shard3_replica_n3/, http://127.0.0.1:40391/solr/localShardsTestColl_shard3_replica_n9/] nUpdates=100 2> 657319 INFO (qtp218128001-2525) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard3 r:core_node12 x:localShardsTestColl_shard3_replica_n3] o.a.s.c.S.Request webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=3 2> 657321 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.u.PeerSync PeerSync: core=localShardsTestColl_shard3_replica_n6 url=http://127.0.0.1:39159/solr DONE. We have no versions. sync failed. 2> 657324 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n9] o.a.s.c.S.Request webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=2 2> 657327 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate 2> 657327 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway 2> 657327 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/localShardsTestColl/leaders/shard3/leader after winning as /collections/localShardsTestColl/leader_elect/shard3/election/72081801657778182-core_node15-n_0000000000 2> 657346 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39159/solr/localShardsTestColl_shard3_replica_n6/ shard3 2> 657373 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-13-processing-127.0.0.1:39159_solr localShardsTestColl_shard3_replica_n6 5570a893-eeb3-4a6f-b218-de371c7a71b3369451200587045 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 657400 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. 2> 657401 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 657401 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39159/solr/localShardsTestColl_shard1_replica_n7/ 2> 657407 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.u.PeerSync PeerSync: core=localShardsTestColl_shard1_replica_n7 url=http://127.0.0.1:39159/solr START replicas=[http://127.0.0.1:40391/solr/localShardsTestColl_shard1_replica_n4/, http://127.0.0.1:44127/solr/localShardsTestColl_shard1_replica_n5/] nUpdates=100 2> 657414 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.u.PeerSync PeerSync: core=localShardsTestColl_shard1_replica_n7 url=http://127.0.0.1:39159/solr DONE. We have no versions. sync failed. 2> 657414 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard1 r:core_node13 x:localShardsTestColl_shard1_replica_n4] o.a.s.c.S.Request webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=2 2> 657418 INFO (qtp218128001-2523) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard1 r:core_node14 x:localShardsTestColl_shard1_replica_n5] o.a.s.c.S.Request webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=2 2> 657422 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate 2> 657422 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway 2> 657423 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/localShardsTestColl/leaders/shard1/leader after winning as /collections/localShardsTestColl/leader_elect/shard1/election/72081801657778182-core_node16-n_0000000000 2> 657436 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [3]) 2> 657437 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39159/solr/localShardsTestColl_shard1_replica_n7/ shard1 2> 657442 INFO (parallelCoreAdminAPIBaseExecutor-1177-thread-14-processing-127.0.0.1:39159_solr localShardsTestColl_shard1_replica_n7 5570a893-eeb3-4a6f-b218-de371c7a71b3369451214548409 create) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 657452 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [3]) 2> 657504 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 657507 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 657507 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 657508 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 657508 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 657508 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 657508 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 2> 657509 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 657529 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=9 2> 657723 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [3]) 2> 658178 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [3]) 2> 658178 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [3]) 2> 658204 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [3]) 2> 658205 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [3]) 2> 658237 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [3]) 2> 658238 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [3]) 2> 658248 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [3]) 2> 658512 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 658515 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 658515 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 658515 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 658515 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 658516 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 658516 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 2> 658516 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 658541 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=7 2> 659520 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 659522 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 659522 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 659522 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 659523 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 659523 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 659524 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 2> 659525 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 659555 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=8 2> 660529 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=2 2> 660531 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 660532 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 660532 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 660532 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 660532 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 660535 INFO (qtp218128001-2528) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 2> 660536 INFO (qtp218128001-2528) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=3 2> 660569 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=10 2> 661537 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 661540 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 661540 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 661541 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 661541 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 661541 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 661545 INFO (qtp218128001-2525) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 2> 661545 INFO (qtp218128001-2525) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 661587 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=11 2> 662545 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 662548 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 662548 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 662548 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 662548 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 662548 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 662555 INFO (qtp218128001-2523) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 2> 662556 INFO (qtp218128001-2523) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 662601 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=7 2> 663553 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 663555 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 663556 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 663556 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 663556 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 663556 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 663564 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 2> 663565 INFO (qtp218128001-2524) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=3 2> 663615 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=9 2> 664561 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 664562 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 664563 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 664563 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 664563 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 664563 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 664574 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 2> 664574 INFO (qtp218128001-2526) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 664630 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=9 2> 665568 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 665571 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 665571 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 665571 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 665571 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 665571 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 665582 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 2> 665583 INFO (qtp218128001-2527) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 665642 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=8 2> 666579 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 666581 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 666581 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 666581 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 666581 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 666581 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 666590 INFO (qtp218128001-2528) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 2> 666590 INFO (qtp218128001-2528) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 666653 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=6 2> 666775 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext Was waiting for replicas to come up, but they are taking too long - assuming they won't come back till later 2> 666776 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync 2> 666776 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:44127/solr/localShardsTestColl_shard2_replica_n1/ 2> 666778 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.u.PeerSync PeerSync: core=localShardsTestColl_shard2_replica_n1 url=http://127.0.0.1:44127/solr START replicas=[http://127.0.0.1:39159/solr/localShardsTestColl_shard2_replica_n2/, http://127.0.0.1:40391/solr/localShardsTestColl_shard2_replica_n8/] nUpdates=100 2> 666783 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.u.PeerSync PeerSync: core=localShardsTestColl_shard2_replica_n1 url=http://127.0.0.1:44127/solr DONE. We have no versions. sync failed. 2> 666795 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard2 r:core_node17 x:localShardsTestColl_shard2_replica_n8] o.a.s.c.S.Request webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1 2> 666823 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate 2> 666824 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway 2> 666824 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/localShardsTestColl/leaders/shard2/leader after winning as /collections/localShardsTestColl/leader_elect/shard2/election/72081801657778184-core_node10-n_0000000000 2> 666836 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [3]) 2> 666837 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [3]) 2> 666837 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [3]) 2> 666839 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:44127/solr/localShardsTestColl_shard2_replica_n1/ shard2 2> 666846 INFO (parallelCoreAdminAPIBaseExecutor-1178-thread-9-processing-127.0.0.1:44127_solr localShardsTestColl_shard2_replica_n1 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 create) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary 2> 666861 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [3]) 2> 666861 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [3]) 2> 666865 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [3]) 2> 667192 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [3]) 2> 667192 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [3]) 2> 667197 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [3]) 2> 667585 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 667587 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 667587 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 667588 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 667588 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 667588 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 667596 INFO (qtp218128001-2525) [n:127.0.0.1:44127_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542 2> 667597 INFO (qtp218128001-2525) [n:127.0.0.1:44127_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451130159542&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 667603 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 667603 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 667666 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=9 2> 668594 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 668597 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 668597 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 668597 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 668597 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 668597 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 668610 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 668610 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 668679 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=8 2> 669602 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 669604 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 669605 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 669605 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 669605 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 669606 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 669618 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 669618 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 669692 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=8 2> 670614 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 670616 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 670616 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 670616 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 670616 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 670616 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 670628 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 670628 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 670705 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=8 2> 671621 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 671623 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 671624 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 671624 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 671624 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 671624 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 671635 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 671635 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 671716 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=7 2> 672628 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 672630 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 672630 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 672630 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 672630 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 672630 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 672643 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 672643 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 672729 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=7 2> 673634 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 673636 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 673636 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 673636 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 673636 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 673636 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 673650 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 673650 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 673740 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=7 2> 674641 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 674643 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 674643 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 674644 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 674644 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 674644 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 674658 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 674658 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 674754 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=9 2> 675649 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 675653 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 675653 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 675653 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 675653 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 675653 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 675666 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 675666 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 675768 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=8 2> 676659 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=2 2> 676662 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 676662 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 676662 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 676662 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 676663 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 676674 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 676675 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 676782 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=10 2> 677673 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=2 2> 677677 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 677678 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 677679 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 677679 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 677680 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 677686 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 677686 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=3 2> 677806 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=12 2> 678684 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 678686 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 678686 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 678687 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 678687 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 678687 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 678693 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 678693 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 678808 INFO (TEST-CloudSolrClientTest.queryWithLocalShardsPreferenceRulesTest-seed#[9CAA27B60427D55F]) [] o.a.s.c.MiniSolrCloudCluster waitForActiveCollection: localShardsTestColl 2> 679692 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 679695 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 679695 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 679695 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 679696 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 679696 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 679701 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 679702 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 680699 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 680701 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 680702 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 680702 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 680702 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 680702 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 680708 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 680709 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 681706 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 681708 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 681708 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 681708 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 681708 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 681708 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 681715 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 681715 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 682713 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 682715 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 682715 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 682716 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 682716 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 682716 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 682724 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 682725 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 683728 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=2 2> 683735 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 683736 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 683736 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 683736 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 683737 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 683737 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 683737 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=4 2> 684760 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 684760 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=6 2> 684761 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=7 2> 684763 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 684763 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 684763 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 684764 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 684765 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 685768 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 685769 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 685770 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 685772 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 685772 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 685772 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 685772 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 685772 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 686777 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 686779 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 686780 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 686781 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=5 2> 686781 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 686781 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 686781 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 686782 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 687786 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 687788 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 687788 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 687788 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 687789 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 687789 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 687793 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 687793 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 688793 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 688796 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 688796 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 688796 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 688796 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 688796 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 688800 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 688800 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 689801 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 689803 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 689803 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 689803 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 689803 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 689803 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 689808 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 689808 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 690808 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 690810 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 690810 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 690810 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 690810 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 690810 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 690815 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 690816 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 691815 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 691817 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 691817 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 691817 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 691817 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 691817 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 691823 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 691823 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 692822 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 692824 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 692824 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 692824 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 692824 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 692824 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 692832 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 692833 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=3 2> 693828 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 693830 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 693830 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 693830 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 693831 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 693831 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 693840 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 693840 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 694836 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 694837 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 694837 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 694837 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 694837 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 694838 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 694847 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 694847 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 695842 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 695843 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 695843 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 695844 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 695844 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 695844 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 695854 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 695854 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 696848 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 696850 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 696850 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 696850 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 696850 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 696850 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 696861 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 696861 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 697857 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 697859 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 697859 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 697859 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 697859 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 697859 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 697869 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 697870 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 698864 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 698866 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 698866 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 698866 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 698866 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 698866 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 698877 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 698878 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 699870 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 699872 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 699872 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 699872 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 699873 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 699873 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 699887 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 699888 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 700876 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 700878 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 700878 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 700879 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 700879 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 700879 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 700894 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 700895 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 701884 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 701887 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 701887 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 701887 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 701887 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 701888 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 701902 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 701902 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 702892 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 702894 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 702894 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 702894 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 702895 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 702895 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 702910 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 702911 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2 2> 703900 INFO (qtp1144655750-2518) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 703903 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 703903 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 703903 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 703903 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 703904 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 703918 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 703919 INFO (qtp217923472-2529) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 704909 INFO (qtp1144655750-2530) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=2 2> 704911 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 704912 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 704912 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 704912 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 704912 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 704926 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 704927 INFO (qtp217923472-2519) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 705917 INFO (qtp1144655750-2533) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 705918 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 705919 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 705919 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 705919 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 705919 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 705934 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 705934 INFO (qtp217923472-2537) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 706922 INFO (qtp1144655750-2532) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 706924 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 706925 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 706925 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 706925 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 706925 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 706941 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 706941 INFO (qtp217923472-2536) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 707929 INFO (qtp1144655750-2516) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 707930 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 707930 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 707931 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 707931 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 707931 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 707947 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 707948 INFO (qtp217923472-2538) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 2> 708936 INFO (TEST-CloudSolrClientTest.queryWithLocalShardsPreferenceRulesTest-seed#[9CAA27B60427D55F]) [] o.a.s.SolrTestCaseJ4 ###Ending queryWithLocalShardsPreferenceRulesTest 2> 708936 INFO (qtp1144655750-2521) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.S.Request webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=1 2> 708939 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's generation: 2 2> 708939 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Leader's version: 1696335569443 2> 708939 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's generation: 2 2> 708939 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower's version: 1696335569443 2> 708940 INFO (indexFetcher-1293-thread-1) [n:127.0.0.1:44127_solr] o.a.s.h.IndexFetcher Follower in sync with leader. 2> 708955 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.h.a.CoreAdminOperation Checking request status for : 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 2> 708955 INFO (qtp217923472-2535) [n:127.0.0.1:39159_solr] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1 > java.lang.RuntimeException: Failed while waiting for active collection > Timeout waiting to see state for collection=localShardsTestColl :DocCollection(localShardsTestColl//collections/localShardsTestColl/state.json/24 )={ > "pullReplicas":"0", > "configName":"conf", > "replicationFactor":"3", > "shards":{ > "shard1":{ > "range":"80000000-d554ffff", > "state":"active", > "replicas":{ > "core_node13":{ > "core":"localShardsTestColl_shard1_replica_n4", > "node_name":"127.0.0.1:40391_solr", > "type":"NRT", > "state":"active", > "force_set_state":"false", > "base_url":"http://127.0.0.1:40391/solr"}, > "core_node14":{ > "core":"localShardsTestColl_shard1_replica_n5", > "node_name":"127.0.0.1:44127_solr", > "type":"NRT", > "state":"active", > "force_set_state":"false", > "base_url":"http://127.0.0.1:44127/solr"}, > "core_node16":{ > "core":"localShardsTestColl_shard1_replica_n7", > "node_name":"127.0.0.1:39159_solr", > "type":"NRT", > "state":"active", > "leader":"true", > "force_set_state":"false", > "base_url":"http://127.0.0.1:39159/solr"}}}, > "shard2":{ > "range":"d5550000-2aa9ffff", > "state":"active", > "replicas":{ > "core_node10":{ > "core":"localShardsTestColl_shard2_replica_n1", > "node_name":"127.0.0.1:44127_solr", > "type":"NRT", > "state":"active", > "leader":"true", > "force_set_state":"false", > "base_url":"http://127.0.0.1:44127/solr"}, > "core_node11":{ > "core":"localShardsTestColl_shard2_replica_n2", > "node_name":"127.0.0.1:39159_solr", > "type":"NRT", > "state":"down", > "force_set_state":"false", > "base_url":"http://127.0.0.1:39159/solr"}, > "core_node17":{ > "core":"localShardsTestColl_shard2_replica_n8", > "node_name":"127.0.0.1:40391_solr", > "type":"NRT", > "state":"active", > "force_set_state":"false", > "base_url":"http://127.0.0.1:40391/solr"}}}, > "shard3":{ > "range":"2aaa0000-7fffffff", > "state":"active", > "replicas":{ > "core_node12":{ > "core":"localShardsTestColl_shard3_replica_n3", > "node_name":"127.0.0.1:44127_solr", > "type":"NRT", > "state":"active", > "force_set_state":"false", > "base_url":"http://127.0.0.1:44127/solr"}, > "core_node15":{ > "core":"localShardsTestColl_shard3_replica_n6", > "node_name":"127.0.0.1:39159_solr", > "type":"NRT", > "state":"active", > "leader":"true", > "force_set_state":"false", > "base_url":"http://127.0.0.1:39159/solr"}, > "core_node18":{ > "core":"localShardsTestColl_shard3_replica_n9", > "node_name":"127.0.0.1:40391_solr", > "type":"NRT", > "state":"active", > "force_set_state":"false", > "base_url":"http://127.0.0.1:40391/solr"}}}}, > "router":{"name":"compositeId"}, > "nrtReplicas":"3", > "tlogReplicas":"0"} > Live Nodes: [127.0.0.1:39159_solr, 127.0.0.1:40391_solr, 127.0.0.1:44127_solr] > Last available state: DocCollection(localShardsTestColl//collections/localShardsTestColl/state.json/24 )={ > "pullReplicas":"0", > "configName":"conf", > "replicationFactor":"3", > "shards":{ > "shard1":{ > "range":"80000000-d554ffff", > "state":"active", > "replicas":{ > "core_node13":{ > "core":"localShardsTestColl_shard1_replica_n4", > "node_name":"127.0.0.1:40391_solr", > "type":"NRT", > "state":"active", > "force_set_state":"false", > "base_url":"http://127.0.0.1:40391/solr"}, > "core_node14":{ > "core":"localShardsTestColl_shard1_replica_n5", > "node_name":"127.0.0.1:44127_solr", > "type":"NRT", > "state":"active", > "force_set_state":"false", > "base_url":"http://127.0.0.1:44127/solr"}, > "core_node16":{ > "core":"localShardsTestColl_shard1_replica_n7", > "node_name":"127.0.0.1:39159_solr", > "type":"NRT", > "state":"active", > "leader":"true", > "force_set_state":"false", > "base_url":"http://127.0.0.1:39159/solr"}}}, > "shard2":{ > "range":"d5550000-2aa9ffff", > "state":"active", > "replicas":{ > "core_node10":{ > "core":"localShardsTestColl_shard2_replica_n1", > "node_name":"127.0.0.1:44127_solr", > "type":"NRT", > "state":"active", > "leader":"true", > "force_set_state":"false", > "base_url":"http://127.0.0.1:44127/solr"}, > "core_node11":{ > "core":"localShardsTestColl_shard2_replica_n2", > "node_name":"127.0.0.1:39159_solr", > "type":"NRT", > "state":"down", > "force_set_state":"false", > "base_url":"http://127.0.0.1:39159/solr"}, > "core_node17":{ > "core":"localShardsTestColl_shard2_replica_n8", > "node_name":"127.0.0.1:40391_solr", > "type":"NRT", > "state":"active", > "force_set_state":"false", > "base_url":"http://127.0.0.1:40391/solr"}}}, > "shard3":{ > "range":"2aaa0000-7fffffff", > "state":"active", > "replicas":{ > "core_node12":{ > "core":"localShardsTestColl_shard3_replica_n3", > "node_name":"127.0.0.1:44127_solr", > "type":"NRT", > "state":"active", > "force_set_state":"false", > "base_url":"http://127.0.0.1:44127/solr"}, > "core_node15":{ > "core":"localShardsTestColl_shard3_replica_n6", > "node_name":"127.0.0.1:39159_solr", > "type":"NRT", > "state":"active", > "leader":"true", > "force_set_state":"false", > "base_url":"http://127.0.0.1:39159/solr"}, > "core_node18":{ > "core":"localShardsTestColl_shard3_replica_n9", > "node_name":"127.0.0.1:40391_solr", > "type":"NRT", > "state":"active", > "force_set_state":"false", > "base_url":"http://127.0.0.1:40391/solr"}}}}, > "router":{"name":"compositeId"}, > "nrtReplicas":"3", > "tlogReplicas":"0"} > at __randomizedtesting.SeedInfo.seed([9CAA27B60427D55F:3A81F37DF334062F]:0) > at org.apache.solr.cloud.MiniSolrCloudCluster.waitForActiveCollection(MiniSolrCloudCluster.java:895) > at org.apache.solr.cloud.MiniSolrCloudCluster.waitForActiveCollection(MiniSolrCloudCluster.java:900) > at org.apache.solr.client.solrj.impl.CloudSolrClientTest.queryWithLocalShardsPreferenceRulesTest(CloudSolrClientTest.java:445) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) > at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) > at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) > at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) > at org.junit.rules.RunRules.evaluate(RunRules.java:20) > at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:44) > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) > at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) > at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) > at org.junit.rules.RunRules.evaluate(RunRules.java:20) > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) > at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) > at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) > at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) > at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) > at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) > at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) > at org.junit.rules.RunRules.evaluate(RunRules.java:20) > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) > at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) > at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) > at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) > at org.junit.rules.RunRules.evaluate(RunRules.java:20) > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) > at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) > at java.base/java.lang.Thread.run(Thread.java:829) 2> NOTE: reproduce with: gradlew test --tests CloudSolrClientTest.queryWithLocalShardsPreferenceRulesTest -Dtests.seed=9CAA27B60427D55F -Dtests.badapples=true -Dtests.locale=ff-GN -Dtests.timezone=Africa/Brazzaville -Dtests.asserts=true -Dtests.file.encoding=UTF-8 2> 709154 INFO (jetty-closer-1604-thread-2) [] o.e.j.s.Server Stopped Server@7372ec3a{STOPPING}[10.0.13,sto=0] 2> 709155 INFO (jetty-closer-1604-thread-3) [] o.e.j.s.Server Stopped Server@3a01df69{STOPPING}[10.0.13,sto=0] 2> 709155 INFO (jetty-closer-1604-thread-1) [] o.e.j.s.Server Stopped Server@720ef990{STOPPING}[10.0.13,sto=0] 2> 709164 INFO (jetty-closer-1604-thread-1) [] o.e.j.s.AbstractConnector Stopped ServerConnector@23ad07f{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0} 2> 709169 INFO (jetty-closer-1604-thread-1) [] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@185700aa{/solr,file:///build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-cwd/,STOPPED} 2> 709171 INFO (jetty-closer-1604-thread-2) [] o.e.j.s.AbstractConnector Stopped ServerConnector@3c7a151{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0} 2> 709174 INFO (jetty-closer-1604-thread-2) [] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@f7c6320{/solr,file:///build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-cwd/,STOPPED} 2> 709185 INFO (jetty-closer-1604-thread-3) [] o.e.j.s.AbstractConnector Stopped ServerConnector@4afdf971{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0} 2> 709188 INFO (jetty-closer-1604-thread-3) [] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5c2ab22c{/solr,file:///build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-cwd/,STOPPED} 2> 709206 INFO (jetty-closer-1604-thread-2) [] o.a.s.c.CoreContainer Shutting down CoreContainer instance=114187265 2> 709207 INFO (jetty-closer-1604-thread-3) [] o.a.s.c.CoreContainer Shutting down CoreContainer instance=607367633 2> 709207 INFO (jetty-closer-1604-thread-2) [] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:44127_solr 2> 709208 INFO (jetty-closer-1604-thread-3) [] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:40391_solr 2> 709217 INFO (jetty-closer-1604-thread-2) [] o.a.s.c.ZkController Publish this node as DOWN... 2> 709217 INFO (jetty-closer-1604-thread-2) [] o.a.s.c.ZkController Publish node=127.0.0.1:44127_solr as DOWN 2> 709236 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (1) 2> 709237 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (1) 2> 709237 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (1) 2> 709238 INFO (jetty-closer-1604-thread-3) [] o.a.s.c.ZkController Publish this node as DOWN... 2> 709238 INFO (jetty-closer-1604-thread-3) [] o.a.s.c.ZkController Publish node=127.0.0.1:40391_solr as DOWN 2> 709265 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [1]) 2> 709267 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [1]) 2> 709270 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [1]) 2> 709288 INFO (zkCallback-1166-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [1]) 2> 709289 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [1]) 2> 709290 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [1]) 2> 709292 INFO (zkCallback-1164-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [1]) 2> 709293 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [1]) 2> 709296 INFO (zkCallback-1160-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [1]) 2> 709305 INFO (zkCallback-1166-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [1]) 2> 709310 INFO (zkCallback-1160-thread-7) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [1]) 2> 709328 INFO (zkCallback-1164-thread-5) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [1]) 2> 709360 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [1]) 2> 709362 INFO (zkCallback-1164-thread-5) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [1]) 2> 709368 INFO (zkCallback-1160-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [1]) 2> 709383 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/singleShardPreferenceTestColl/state.json] for collection [singleShardPreferenceTestColl] has occurred - updating... (live nodes size: [1]) 2> 709383 INFO (zkCallback-1160-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/singleShardPreferenceTestColl/state.json] for collection [singleShardPreferenceTestColl] has occurred - updating... (live nodes size: [1]) 2> 709384 INFO (zkCallback-1164-thread-5) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/singleShardPreferenceTestColl/state.json] for collection [singleShardPreferenceTestColl] has occurred - updating... (live nodes size: [1]) 2> 709389 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [1]) 2> 709389 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [1]) 2> 709390 INFO (zkCallback-1164-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [1]) 2> 709390 INFO (zkCallback-1160-thread-7) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [1]) 2> 709390 INFO (zkCallback-1166-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [1]) 2> 709391 INFO (zkCallback-1166-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [1]) 2> 709425 INFO (zkCallback-1160-thread-7) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testVersionsAreReturned/state.json] for collection [testVersionsAreReturned] has occurred - updating... (live nodes size: [1]) 2> 709430 INFO (zkCallback-1164-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testVersionsAreReturned/state.json] for collection [testVersionsAreReturned] has occurred - updating... (live nodes size: [1]) 2> 709444 INFO (zkCallback-1160-thread-7) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/2nd_collection/state.json] for collection [2nd_collection] has occurred - updating... (live nodes size: [1]) 2> 709448 INFO (zkCallback-1164-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/2nd_collection/state.json] for collection [2nd_collection] has occurred - updating... (live nodes size: [1]) 2> 709461 INFO (zkCallback-1160-thread-7) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [1]) 2> 709461 INFO (zkCallback-1166-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [1]) 2> 709464 INFO (zkCallback-1164-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [1]) 2> 709468 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testAliasHandling/state.json] for collection [testAliasHandling] has occurred - updating... (live nodes size: [1]) 2> 709471 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testAliasHandling/state.json] for collection [testAliasHandling] has occurred - updating... (live nodes size: [1]) 2> 709485 INFO (zkCallback-1166-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [1]) 2> 709485 INFO (zkCallback-1166-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [1]) 2> 709489 INFO (zkCallback-1160-thread-7) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [1]) 2> 709489 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [1]) 2> 709495 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [1]) 2> 709495 INFO (zkCallback-1164-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [1]) 2> 709504 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [1]) 2> 709505 INFO (zkCallback-1160-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [1]) 2> 709508 INFO (zkCallback-1164-thread-5) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [1]) 2> 709538 INFO (jetty-closer-1604-thread-1) [] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1230136153 2> 709539 INFO (jetty-closer-1604-thread-1) [] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:39159_solr 2> 709565 INFO (zkCallback-1160-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/singleShardPreferenceTestColl/state.json] for collection [singleShardPreferenceTestColl] has occurred - updating... (live nodes size: [1]) 2> 709567 INFO (zkCallback-1166-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/singleShardPreferenceTestColl/state.json] for collection [singleShardPreferenceTestColl] has occurred - updating... (live nodes size: [1]) 2> 709571 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/singleShardPreferenceTestColl/state.json] for collection [singleShardPreferenceTestColl] has occurred - updating... (live nodes size: [1]) 2> 709572 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) 2> 709574 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) 2> 709574 INFO (jetty-closer-1604-thread-1) [] o.a.s.c.ZkController Publish this node as DOWN... 2> 709575 INFO (jetty-closer-1604-thread-1) [] o.a.s.c.ZkController Publish node=127.0.0.1:39159_solr as DOWN 2> 709582 INFO (zkCallback-1164-thread-4) [] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0) 2> 709600 INFO (zkCallback-1166-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [0]) 2> 709601 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [0]) 2> 709602 INFO (zkCallback-1166-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [0]) 2> 709603 INFO (zkCallback-1166-thread-7) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [0]) 2> 709630 INFO (zkCallback-1160-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [0]) 2> 709630 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [0]) 2> 709631 INFO (zkCallback-1166-thread-7) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [0]) 2> 709635 INFO (zkCallback-1164-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [0]) 2> 709637 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [0]) 2> 709637 INFO (zkCallback-1166-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [0]) 2> 709654 INFO (zkCallback-1166-thread-7) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [0]) 2> 709656 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [0]) 2> 709656 INFO (zkCallback-1164-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [0]) 2> 709656 INFO (zkCallback-1160-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [0]) 2> 709656 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [0]) 2> 709654 INFO (zkCallback-1166-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_test/state.json] for collection [perReplicaState_test] has occurred - updating... (live nodes size: [0]) 2> 709662 INFO (zkCallback-1160-thread-7) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [0]) 2> 709663 INFO (zkCallback-1164-thread-5) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [0]) 2> 709665 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [0]) 2> 709665 INFO (zkCallback-1166-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [0]) 2> 709679 INFO (zkCallback-1160-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [0]) 2> 709691 INFO (zkCallback-1164-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [0]) 2> 709707 INFO (zkCallback-1160-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testVersionsAreReturned/state.json] for collection [testVersionsAreReturned] has occurred - updating... (live nodes size: [0]) 2> 709707 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/multicollection1/state.json] for collection [multicollection1] has occurred - updating... (live nodes size: [0]) 2> 709708 INFO (zkCallback-1160-thread-9) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNonRetryableRequests/state.json] for collection [testNonRetryableRequests] has occurred - updating... (live nodes size: [0]) 2> 709719 INFO (zkCallback-1166-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNonRetryableRequests/state.json] for collection [testNonRetryableRequests] has occurred - updating... (live nodes size: [0]) 2> 709720 INFO (zkCallback-1164-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testVersionsAreReturned/state.json] for collection [testVersionsAreReturned] has occurred - updating... (live nodes size: [0]) 2> 709721 INFO (zkCallback-1164-thread-7) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNonRetryableRequests/state.json] for collection [testNonRetryableRequests] has occurred - updating... (live nodes size: [0]) 2> 709723 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/multicollection1/state.json] for collection [multicollection1] has occurred - updating... (live nodes size: [0]) 2> 709745 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/2nd_collection/state.json] for collection [2nd_collection] has occurred - updating... (live nodes size: [0]) 2> 709749 INFO (zkCallback-1164-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testInitializationWithSolrUrls/state.json] for collection [testInitializationWithSolrUrls] has occurred - updating... (live nodes size: [0]) 2> 709750 INFO (zkCallback-1160-thread-9) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/2nd_collection/state.json] for collection [2nd_collection] has occurred - updating... (live nodes size: [0]) 2> 709753 INFO (zkCallback-1160-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testInitializationWithSolrUrls/state.json] for collection [testInitializationWithSolrUrls] has occurred - updating... (live nodes size: [0]) 2> 709760 INFO (zkCallback-1164-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/multicollection2/state.json] for collection [multicollection2] has occurred - updating... (live nodes size: [0]) 2> 709761 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/multicollection2/state.json] for collection [multicollection2] has occurred - updating... (live nodes size: [0]) 2> 709785 INFO (coreCloseExecutor-1609-thread-1) [n:127.0.0.1:44127_solr c:singleShardPreferenceTestColl s:shard1 r:core_node4 x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@7c25d19e singleShardPreferenceTestColl_shard1_replica_p1 2> 709786 INFO (coreCloseExecutor-1609-thread-1) [n:127.0.0.1:44127_solr c:singleShardPreferenceTestColl s:shard1 r:core_node4 x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.singleShardPreferenceTestColl.shard1.replica_p1 tag=SolrCore@7c25d19e 2> 709788 INFO (zkCallback-1164-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testAliasHandling/state.json] for collection [testAliasHandling] has occurred - updating... (live nodes size: [0]) 2> 709789 INFO (zkCallback-1160-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testAliasHandling/state.json] for collection [testAliasHandling] has occurred - updating... (live nodes size: [0]) 2> 709806 INFO (zkCallback-1160-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testPerReplicaStateCollection/state.json] for collection [testPerReplicaStateCollection] has occurred - updating... (live nodes size: [0]) 2> 709806 INFO (zkCallback-1164-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testPerReplicaStateCollection/state.json] for collection [testPerReplicaStateCollection] has occurred - updating... (live nodes size: [0]) 2> 709814 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 709814 INFO (zkCallback-1166-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 709814 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 709818 INFO (coreCloseExecutor-1609-thread-2) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@5392753e stale_state_test_col_shard1_replica_n3 2> 709831 INFO (coreCloseExecutor-1609-thread-3) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@3cf5692a testNonRetryableRequests_shard1_replica_n1 2> 709836 INFO (coreCloseExecutor-1609-thread-4) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@5a4fc389 perReplicaState_test_shard1_replica_n7 2> 709838 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stateVersionParamTest/state.json] for collection [stateVersionParamTest] has occurred - updating... (live nodes size: [0]) 2> 709839 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stateVersionParamTest/state.json] for collection [stateVersionParamTest] has occurred - updating... (live nodes size: [0]) 2> 709842 INFO (coreCloseExecutor-1609-thread-5) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@4181f1ca perReplicaState_testv2_shard1_replica_n5 2> 709849 INFO (zkCallback-1160-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/singleShardPreferenceTestColl/state.json] for collection [singleShardPreferenceTestColl] has occurred - updating... (live nodes size: [0]) 2> 709849 INFO (zkCallback-1164-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/singleShardPreferenceTestColl/state.json] for collection [singleShardPreferenceTestColl] has occurred - updating... (live nodes size: [0]) 2> 709855 INFO (zkCallback-1166-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/singleShardPreferenceTestColl/state.json] for collection [singleShardPreferenceTestColl] has occurred - updating... (live nodes size: [0]) 2> 709858 INFO (coreCloseExecutor-1609-thread-7) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard3 r:core_node12 x:localShardsTestColl_shard3_replica_n3] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@758620b2 localShardsTestColl_shard3_replica_n3 2> 709859 INFO (coreCloseExecutor-1609-thread-8) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard1 r:core_node14 x:localShardsTestColl_shard1_replica_n5] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@705647a1 localShardsTestColl_shard1_replica_n5 2> 709863 INFO (coreCloseExecutor-1609-thread-6) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@4892b4bc localShardsTestColl_shard2_replica_n1 2> 709899 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routing_collection/state.json] for collection [routing_collection] has occurred - updating... (live nodes size: [0]) 2> 709899 INFO (zkCallback-1160-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routing_collection/state.json] for collection [routing_collection] has occurred - updating... (live nodes size: [0]) 2> 709925 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/ping_test/state.json] for collection [ping_test] has occurred - updating... (live nodes size: [0]) 2> 709926 INFO (zkCallback-1160-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/ping_test/state.json] for collection [ping_test] has occurred - updating... (live nodes size: [0]) 2> 709943 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [0]) 2> 709943 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [0]) 2> 709944 INFO (zkCallback-1166-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [0]) 2> 709949 INFO (zkCallback-1166-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNonRetryableRequests/state.json] for collection [testNonRetryableRequests] has occurred - updating... (live nodes size: [0]) 2> 709951 INFO (zkCallback-1160-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNonRetryableRequests/state.json] for collection [testNonRetryableRequests] has occurred - updating... (live nodes size: [0]) 2> 709954 INFO (zkCallback-1164-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNonRetryableRequests/state.json] for collection [testNonRetryableRequests] has occurred - updating... (live nodes size: [0]) 2> 709981 INFO (zkCallback-1166-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [0]) 2> 709982 INFO (zkCallback-1166-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [0]) 2> 709982 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [0]) 2> 709982 ERROR (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CollectionHandlingUtils Error from shard: http://127.0.0.1:39159/solr 2> => org.apache.solr.client.solrj.SolrServerException: Connection refused 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) 2> org.apache.solr.client.solrj.SolrServerException: Connection refused 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) ~[main/:?] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:192) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:184) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpExchange.notifyFailureComplete(HttpExchange.java:284) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpExchange.abort(HttpExchange.java:256) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpConversation.abort(HttpConversation.java:159) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpRequest.abort(HttpRequest.java:865) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpDestination.abort(HttpDestination.java:530) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpDestination.failed(HttpDestination.java:294) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.AbstractConnectionPool$FutureConnection.failed(AbstractConnectionPool.java:574) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.Promise$Wrapper.failed(Promise.java:169) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1$1.failed(HttpClient.java:628) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HTTPSessionListenerPromise.failConnectionPromise(HTTPSessionListenerPromise.java:136) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HTTPSessionListenerPromise.failed(HTTPSessionListenerPromise.java:51) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.Promise$1.failed(Promise.java:94) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.ClientConnector.connectFailed(ClientConnector.java:539) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.ClientConnector.connect(ClientConnector.java:455) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.HTTP2Client.connect(HTTP2Client.java:406) ~[http2-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2.connect(HttpClientTransportOverHTTP2.java:146) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2.connect(HttpClientTransportOverHTTP2.java:135) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.connect(HttpClient.java:633) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.succeeded(HttpClient.java:610) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.succeeded(HttpClient.java:606) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.SocketAddressResolver$Async.lambda$resolve$1(SocketAddressResolver.java:180) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289) [main/:?] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> Caused by: java.net.ConnectException: Connection refused 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:774) ~[?:?] 2> at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:120) ~[?:?] 2> at org.eclipse.jetty.io.ClientConnector.connect(ClientConnector.java:427) ~[jetty-io-10.0.13.jar:10.0.13] 2> ... 11 more 2> 710009 INFO (zkCallback-1164-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [0]) 2> 710009 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [0]) 2> 710003 INFO (zkCallback-1160-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/perReplicaState_testv2/state.json] for collection [perReplicaState_testv2] has occurred - updating... (live nodes size: [0]) 2> 710029 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testParallelUpdateQTime/state.json] for collection [testParallelUpdateQTime] has occurred - updating... (live nodes size: [0]) 2> 710032 INFO (zkCallback-1164-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testParallelUpdateQTime/state.json] for collection [testParallelUpdateQTime] has occurred - updating... (live nodes size: [0]) 2> 710012 ERROR (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CollectionHandlingUtils Error from shard 127.0.0.1:39159_solr: {STATUS=failed} 2> 710072 INFO (zkCallback-1160-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/multicollection1/state.json] for collection [multicollection1] has occurred - updating... (live nodes size: [0]) 2> 710073 INFO (zkCallback-1164-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/multicollection1/state.json] for collection [multicollection1] has occurred - updating... (live nodes size: [0]) 2> 710090 INFO (coreCloseExecutor-1612-thread-1) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@62cbc8f6 routing_collection_shard1_replica_n1 2> 710092 INFO (coreCloseExecutor-1612-thread-1) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.routing_collection.shard1.replica_n1 tag=SolrCore@62cbc8f6 2> 710096 INFO (coreCloseExecutor-1612-thread-2) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@3fd96993 testVersionsAreReturned_shard1_replica_n2 2> 710101 INFO (zkCallback-1160-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testInitializationWithSolrUrls/state.json] for collection [testInitializationWithSolrUrls] has occurred - updating... (live nodes size: [0]) 2> 710103 INFO (zkCallback-1164-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testInitializationWithSolrUrls/state.json] for collection [testInitializationWithSolrUrls] has occurred - updating... (live nodes size: [0]) 2> 710104 INFO (coreCloseExecutor-1612-thread-3) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@3d962f06 multicollection1_shard1_replica_n2 2> 710127 INFO (coreCloseExecutor-1612-thread-4) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@55619cdd multicollection2_shard1_replica_n2 2> 710133 INFO (coreCloseExecutor-1612-thread-5) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@78a7c0d stateVersionParamTest_shard1_replica_n1 2> 710083 ERROR (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CollectionHandlingUtils Error from shard: http://127.0.0.1:44127/solr 2> => org.apache.solr.client.solrj.SolrServerException: Connection refused 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) 2> org.apache.solr.client.solrj.SolrServerException: Connection refused 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) ~[main/:?] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:192) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:184) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpExchange.notifyFailureComplete(HttpExchange.java:284) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpExchange.abort(HttpExchange.java:256) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpConversation.abort(HttpConversation.java:159) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpRequest.abort(HttpRequest.java:865) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpDestination.abort(HttpDestination.java:530) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpDestination.failed(HttpDestination.java:294) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.AbstractConnectionPool$FutureConnection.failed(AbstractConnectionPool.java:574) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.Promise$Wrapper.failed(Promise.java:169) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1$1.failed(HttpClient.java:628) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HTTPSessionListenerPromise.failConnectionPromise(HTTPSessionListenerPromise.java:136) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HTTPSessionListenerPromise.failed(HTTPSessionListenerPromise.java:51) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.Promise$1.failed(Promise.java:94) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.ClientConnector.connectFailed(ClientConnector.java:539) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.ClientConnector.connect(ClientConnector.java:455) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.HTTP2Client.connect(HTTP2Client.java:406) ~[http2-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2.connect(HttpClientTransportOverHTTP2.java:146) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2.connect(HttpClientTransportOverHTTP2.java:135) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.connect(HttpClient.java:633) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.succeeded(HttpClient.java:610) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.succeeded(HttpClient.java:606) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.SocketAddressResolver$Async.lambda$resolve$1(SocketAddressResolver.java:180) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289) [main/:?] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> Caused by: java.net.ConnectException: Connection refused 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:774) ~[?:?] 2> at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:120) ~[?:?] 2> at org.eclipse.jetty.io.ClientConnector.connect(ClientConnector.java:427) ~[jetty-io-10.0.13.jar:10.0.13] 2> ... 11 more 2> 710142 ERROR (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CollectionHandlingUtils Error from shard 127.0.0.1:44127_solr: {STATUS=failed} 2> 710154 INFO (coreCloseExecutor-1612-thread-7) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@4266fdf2 testParallelUpdateQTime_shard1_replica_n2 2> 710147 INFO (coreCloseExecutor-1612-thread-6) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@26dd0d41 singleShardPreferenceTestColl_shard1_replica_t2 2> 710157 INFO (coreCloseExecutor-1612-thread-8) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@4a78a4c4 testNonRetryableRequests_shard1_replica_n3 2> 710160 INFO (coreCloseExecutor-1612-thread-9) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@452bcaaa testInitializationWithSolrUrls_shard1_replica_n2 2> 710172 INFO (zkCallback-1160-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/multicollection2/state.json] for collection [multicollection2] has occurred - updating... (live nodes size: [0]) 2> 710175 INFO (coreCloseExecutor-1612-thread-12) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@1b1ab821 ping_test_shard1_replica_n2 2> 710177 INFO (coreCloseExecutor-1612-thread-13) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@3570cc7 testPerReplicaStateCollection_shard1_replica_n2 2> 710177 INFO (coreCloseExecutor-1612-thread-10) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@2ec5aae4 testAliasHandling_shard1_replica_n1 2> 710179 INFO (coreCloseExecutor-1612-thread-14) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@1f048edb perReplicaState_test_shard2_replica_n1 2> 710183 INFO (zkCallback-1164-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/multicollection2/state.json] for collection [multicollection2] has occurred - updating... (live nodes size: [0]) 2> 710186 INFO (coreCloseExecutor-1612-thread-16) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n9] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@33d93e36 localShardsTestColl_shard3_replica_n9 2> 710186 INFO (coreCloseExecutor-1612-thread-15) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@d81de9f perReplicaState_testv2_shard2_replica_n1 2> 710188 INFO (coreCloseExecutor-1612-thread-11) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@64cea453 2nd_collection_shard1_replica_n2 2> 710197 INFO (coreCloseExecutor-1612-thread-18) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard1 r:core_node13 x:localShardsTestColl_shard1_replica_n4] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@ed81300 localShardsTestColl_shard1_replica_n4 2> 710197 INFO (coreCloseExecutor-1612-thread-17) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard2 r:core_node17 x:localShardsTestColl_shard2_replica_n8] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@195c93c9 localShardsTestColl_shard2_replica_n8 2> 710208 INFO (zkCallback-1160-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testPerReplicaStateCollection/state.json] for collection [testPerReplicaStateCollection] has occurred - updating... (live nodes size: [0]) 2> 710209 INFO (zkCallback-1164-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testPerReplicaStateCollection/state.json] for collection [testPerReplicaStateCollection] has occurred - updating... (live nodes size: [0]) 2> 710223 INFO (zkCallback-1160-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stateVersionParamTest/state.json] for collection [stateVersionParamTest] has occurred - updating... (live nodes size: [0]) 2> 710222 INFO (zkCallback-1164-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stateVersionParamTest/state.json] for collection [stateVersionParamTest] has occurred - updating... (live nodes size: [0]) 2> 710170 ERROR (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CollectionHandlingUtils Error from shard: http://127.0.0.1:40391/solr 2> => org.apache.solr.client.solrj.SolrServerException: Connection refused 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) 2> org.apache.solr.client.solrj.SolrServerException: Connection refused 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) ~[main/:?] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:192) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:184) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpExchange.notifyFailureComplete(HttpExchange.java:284) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpExchange.abort(HttpExchange.java:256) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpConversation.abort(HttpConversation.java:159) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpRequest.abort(HttpRequest.java:865) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpDestination.abort(HttpDestination.java:530) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpDestination.failed(HttpDestination.java:294) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.AbstractConnectionPool$FutureConnection.failed(AbstractConnectionPool.java:574) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.Promise$Wrapper.failed(Promise.java:169) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1$1.failed(HttpClient.java:628) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HTTPSessionListenerPromise.failConnectionPromise(HTTPSessionListenerPromise.java:136) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HTTPSessionListenerPromise.failed(HTTPSessionListenerPromise.java:51) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.Promise$1.failed(Promise.java:94) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.ClientConnector.connectFailed(ClientConnector.java:539) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.ClientConnector.connect(ClientConnector.java:455) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.HTTP2Client.connect(HTTP2Client.java:406) ~[http2-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2.connect(HttpClientTransportOverHTTP2.java:146) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2.connect(HttpClientTransportOverHTTP2.java:135) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.connect(HttpClient.java:633) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.succeeded(HttpClient.java:610) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.succeeded(HttpClient.java:606) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.SocketAddressResolver$Async.lambda$resolve$1(SocketAddressResolver.java:180) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289) [main/:?] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> Caused by: java.net.ConnectException: Connection refused 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:774) ~[?:?] 2> at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:120) ~[?:?] 2> at org.eclipse.jetty.io.ClientConnector.connect(ClientConnector.java:427) ~[jetty-io-10.0.13.jar:10.0.13] 2> ... 11 more 2> 710254 INFO (zkCallback-1164-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/ping_test/state.json] for collection [ping_test] has occurred - updating... (live nodes size: [0]) 2> 710255 ERROR (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CollectionHandlingUtils Error from shard 127.0.0.1:40391_solr: {STATUS=failed} 2> 710254 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/ping_test/state.json] for collection [ping_test] has occurred - updating... (live nodes size: [0]) 2> 710243 INFO (zkCallback-1164-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routing_collection/state.json] for collection [routing_collection] has occurred - updating... (live nodes size: [0]) 2> 710241 INFO (zkCallback-1160-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routing_collection/state.json] for collection [routing_collection] has occurred - updating... (live nodes size: [0]) 2> 710286 INFO (zkCallback-1166-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNonRetryableRequests/state.json] for collection [testNonRetryableRequests] has occurred - updating... (live nodes size: [0]) 2> 710287 INFO (zkCallback-1164-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNonRetryableRequests/state.json] for collection [testNonRetryableRequests] has occurred - updating... (live nodes size: [0]) 2> 710289 INFO (zkCallback-1160-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNonRetryableRequests/state.json] for collection [testNonRetryableRequests] has occurred - updating... (live nodes size: [0]) 2> 710303 INFO (zkCallback-1160-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/overwrite/state.json] for collection [overwrite] has occurred - updating... (live nodes size: [0]) 2> 710319 INFO (zkCallback-1164-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testParallelUpdateQTime/state.json] for collection [testParallelUpdateQTime] has occurred - updating... (live nodes size: [0]) 2> 710319 INFO (zkCallback-1160-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testParallelUpdateQTime/state.json] for collection [testParallelUpdateQTime] has occurred - updating... (live nodes size: [0]) 2> 710337 INFO (coreCloseExecutor-1614-thread-1) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@4eb487c0 routing_collection_shard2_replica_n2 2> 710338 INFO (coreCloseExecutor-1614-thread-1) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.routing_collection.shard2.replica_n2 tag=SolrCore@4eb487c0 2> 710351 INFO (coreCloseExecutor-1614-thread-2) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@72d15c3b overwrite_shard1_replica_n1 2> 710356 INFO (coreCloseExecutor-1614-thread-3) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@6ffd87a8 testVersionsAreReturned_shard2_replica_n1 2> 710364 INFO (coreCloseExecutor-1614-thread-4) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@58e45191 multicollection1_shard2_replica_n1 2> 710366 INFO (coreCloseExecutor-1614-thread-5) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@738bc8a1 multicollection2_shard2_replica_n1 2> 710372 INFO (coreCloseExecutor-1614-thread-6) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@71c8f084 stateVersionParamTest_shard2_replica_n2 2> 710375 INFO (coreCloseExecutor-1614-thread-7) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@36fd773a singleShardPreferenceTestColl_shard1_replica_n3 2> 710377 INFO (coreCloseExecutor-1614-thread-8) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@23350cdb testParallelUpdateQTime_shard2_replica_n1 2> 710384 INFO (coreCloseExecutor-1614-thread-9) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@7eacec10 testNonRetryableRequests_shard1_replica_n2 2> 710387 INFO (coreCloseExecutor-1614-thread-10) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@2075501c testInitializationWithSolrUrls_shard2_replica_n1 2> 710390 INFO (coreCloseExecutor-1614-thread-12) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@31ffb4b5 2nd_collection_shard2_replica_n1 2> 710390 INFO (coreCloseExecutor-1614-thread-11) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@17f85463 testAliasHandling_shard2_replica_n2 2> 710391 INFO (coreCloseExecutor-1614-thread-13) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@24098180 ping_test_shard2_replica_n1 2> 710392 INFO (coreCloseExecutor-1614-thread-14) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@3c5a93c2 testPerReplicaStateCollection_shard2_replica_n1 2> 710396 INFO (coreCloseExecutor-1614-thread-15) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard2 r:core_node6 x:perReplicaState_test_shard2_replica_n5] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@7350817f perReplicaState_test_shard2_replica_n5 2> 710400 INFO (coreCloseExecutor-1614-thread-16) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@25b5b6f perReplicaState_test_shard1_replica_n3 2> 710404 INFO (coreCloseExecutor-1614-thread-17) [n:127.0.0.1:39159_solr c:perReplicaState_testv2 s:shard2 r:core_node8 x:perReplicaState_testv2_shard2_replica_n7] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@25465559 perReplicaState_testv2_shard2_replica_n7 2> 710407 INFO (coreCloseExecutor-1614-thread-18) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@32fbe23f localShardsTestColl_shard3_replica_n6 2> 710420 INFO (coreCloseExecutor-1614-thread-19) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.c.SolrCore CLOSING SolrCore org.apache.solr.core.SolrCore@6802a10d localShardsTestColl_shard1_replica_n7 2> 710363 ERROR (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CollectionHandlingUtils Error from shard: http://127.0.0.1:44127/solr 2> => org.apache.solr.client.solrj.SolrServerException: Connection refused 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) 2> org.apache.solr.client.solrj.SolrServerException: Connection refused 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) ~[main/:?] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:192) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:184) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpExchange.notifyFailureComplete(HttpExchange.java:284) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpExchange.abort(HttpExchange.java:256) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpConversation.abort(HttpConversation.java:159) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpRequest.abort(HttpRequest.java:865) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpDestination.abort(HttpDestination.java:530) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpDestination.failed(HttpDestination.java:294) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.AbstractConnectionPool$FutureConnection.failed(AbstractConnectionPool.java:574) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.Promise$Wrapper.failed(Promise.java:169) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1$1.failed(HttpClient.java:628) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HTTPSessionListenerPromise.failConnectionPromise(HTTPSessionListenerPromise.java:136) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HTTPSessionListenerPromise.failed(HTTPSessionListenerPromise.java:51) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.Promise$1.failed(Promise.java:94) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.ClientConnector.connectFailed(ClientConnector.java:539) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.ClientConnector.connect(ClientConnector.java:455) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.HTTP2Client.connect(HTTP2Client.java:406) ~[http2-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2.connect(HttpClientTransportOverHTTP2.java:146) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2.connect(HttpClientTransportOverHTTP2.java:135) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.connect(HttpClient.java:633) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.succeeded(HttpClient.java:610) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.succeeded(HttpClient.java:606) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.SocketAddressResolver$Async.lambda$resolve$1(SocketAddressResolver.java:180) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289) [main/:?] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> Caused by: java.net.ConnectException: Connection refused 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:774) ~[?:?] 2> at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:120) ~[?:?] 2> at org.eclipse.jetty.io.ClientConnector.connect(ClientConnector.java:427) ~[jetty-io-10.0.13.jar:10.0.13] 2> ... 11 more 2> 710442 ERROR (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CollectionHandlingUtils Error from shard 127.0.0.1:44127_solr: {STATUS=failed} 2> 710467 ERROR (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CollectionHandlingUtils Error from shard: http://127.0.0.1:39159/solr 2> => org.apache.solr.client.solrj.SolrServerException: Connection refused 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) 2> org.apache.solr.client.solrj.SolrServerException: Connection refused 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) ~[main/:?] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:192) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:184) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpExchange.notifyFailureComplete(HttpExchange.java:284) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpExchange.abort(HttpExchange.java:256) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpConversation.abort(HttpConversation.java:159) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpRequest.abort(HttpRequest.java:865) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpDestination.abort(HttpDestination.java:530) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpDestination.failed(HttpDestination.java:294) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.AbstractConnectionPool$FutureConnection.failed(AbstractConnectionPool.java:574) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.Promise$Wrapper.failed(Promise.java:169) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1$1.failed(HttpClient.java:628) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HTTPSessionListenerPromise.failConnectionPromise(HTTPSessionListenerPromise.java:136) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HTTPSessionListenerPromise.failed(HTTPSessionListenerPromise.java:51) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.Promise$1.failed(Promise.java:94) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.ClientConnector.connectFailed(ClientConnector.java:539) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.ClientConnector.connect(ClientConnector.java:455) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.HTTP2Client.connect(HTTP2Client.java:406) ~[http2-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2.connect(HttpClientTransportOverHTTP2.java:146) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2.connect(HttpClientTransportOverHTTP2.java:135) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.connect(HttpClient.java:633) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.succeeded(HttpClient.java:610) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.succeeded(HttpClient.java:606) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.SocketAddressResolver$Async.lambda$resolve$1(SocketAddressResolver.java:180) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289) [main/:?] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> Caused by: java.net.ConnectException: Connection refused 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:774) ~[?:?] 2> at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:120) ~[?:?] 2> at org.eclipse.jetty.io.ClientConnector.connect(ClientConnector.java:427) ~[jetty-io-10.0.13.jar:10.0.13] 2> ... 11 more 2> 710503 ERROR (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CollectionHandlingUtils Error from shard 127.0.0.1:39159_solr: {STATUS=failed} 2> 710534 ERROR (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CollectionHandlingUtils Error from shard: http://127.0.0.1:39159/solr 2> => org.apache.solr.client.solrj.SolrServerException: Connection refused 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) 2> org.apache.solr.client.solrj.SolrServerException: Connection refused 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) ~[main/:?] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:192) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:184) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpExchange.notifyFailureComplete(HttpExchange.java:284) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpExchange.abort(HttpExchange.java:256) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpConversation.abort(HttpConversation.java:159) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpRequest.abort(HttpRequest.java:865) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpDestination.abort(HttpDestination.java:530) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpDestination.failed(HttpDestination.java:294) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.AbstractConnectionPool$FutureConnection.failed(AbstractConnectionPool.java:574) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.Promise$Wrapper.failed(Promise.java:169) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1$1.failed(HttpClient.java:628) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HTTPSessionListenerPromise.failConnectionPromise(HTTPSessionListenerPromise.java:136) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HTTPSessionListenerPromise.failed(HTTPSessionListenerPromise.java:51) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.Promise$1.failed(Promise.java:94) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.ClientConnector.connectFailed(ClientConnector.java:539) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.ClientConnector.connect(ClientConnector.java:455) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.HTTP2Client.connect(HTTP2Client.java:406) ~[http2-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2.connect(HttpClientTransportOverHTTP2.java:146) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2.connect(HttpClientTransportOverHTTP2.java:135) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.connect(HttpClient.java:633) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.succeeded(HttpClient.java:610) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.succeeded(HttpClient.java:606) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.SocketAddressResolver$Async.lambda$resolve$1(SocketAddressResolver.java:180) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289) [main/:?] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> Caused by: java.net.ConnectException: Connection refused 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:774) ~[?:?] 2> at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:120) ~[?:?] 2> at org.eclipse.jetty.io.ClientConnector.connect(ClientConnector.java:427) ~[jetty-io-10.0.13.jar:10.0.13] 2> ... 11 more 2> 710575 ERROR (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CollectionHandlingUtils Error from shard 127.0.0.1:39159_solr: {STATUS=failed} 2> 710593 ERROR (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CollectionHandlingUtils Error from shard: http://127.0.0.1:40391/solr 2> => org.apache.solr.client.solrj.SolrServerException: Connection refused 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) 2> org.apache.solr.client.solrj.SolrServerException: Connection refused 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) ~[main/:?] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:192) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:184) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpExchange.notifyFailureComplete(HttpExchange.java:284) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpExchange.abort(HttpExchange.java:256) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpConversation.abort(HttpConversation.java:159) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpRequest.abort(HttpRequest.java:865) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpDestination.abort(HttpDestination.java:530) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpDestination.failed(HttpDestination.java:294) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.AbstractConnectionPool$FutureConnection.failed(AbstractConnectionPool.java:574) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.Promise$Wrapper.failed(Promise.java:169) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1$1.failed(HttpClient.java:628) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HTTPSessionListenerPromise.failConnectionPromise(HTTPSessionListenerPromise.java:136) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HTTPSessionListenerPromise.failed(HTTPSessionListenerPromise.java:51) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.Promise$1.failed(Promise.java:94) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.ClientConnector.connectFailed(ClientConnector.java:539) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.ClientConnector.connect(ClientConnector.java:455) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.HTTP2Client.connect(HTTP2Client.java:406) ~[http2-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2.connect(HttpClientTransportOverHTTP2.java:146) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2.connect(HttpClientTransportOverHTTP2.java:135) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.connect(HttpClient.java:633) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.succeeded(HttpClient.java:610) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.succeeded(HttpClient.java:606) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.SocketAddressResolver$Async.lambda$resolve$1(SocketAddressResolver.java:180) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289) [main/:?] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> Caused by: java.net.ConnectException: Connection refused 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:774) ~[?:?] 2> at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:120) ~[?:?] 2> at org.eclipse.jetty.io.ClientConnector.connect(ClientConnector.java:427) ~[jetty-io-10.0.13.jar:10.0.13] 2> ... 11 more 2> 710637 ERROR (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CollectionHandlingUtils Error from shard 127.0.0.1:40391_solr: {STATUS=failed} 2> 710663 ERROR (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CollectionHandlingUtils Error from shard: http://127.0.0.1:40391/solr 2> => org.apache.solr.client.solrj.SolrServerException: Connection refused 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) 2> org.apache.solr.client.solrj.SolrServerException: Connection refused 2> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.onFailure(Http2SolrClient.java:486) ~[main/:?] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:192) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.ResponseNotifier.notifyFailure(ResponseNotifier.java:184) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpExchange.notifyFailureComplete(HttpExchange.java:284) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpExchange.abort(HttpExchange.java:256) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpConversation.abort(HttpConversation.java:159) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpRequest.abort(HttpRequest.java:865) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpDestination.abort(HttpDestination.java:530) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpDestination.failed(HttpDestination.java:294) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.AbstractConnectionPool$FutureConnection.failed(AbstractConnectionPool.java:574) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.Promise$Wrapper.failed(Promise.java:169) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1$1.failed(HttpClient.java:628) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HTTPSessionListenerPromise.failConnectionPromise(HTTPSessionListenerPromise.java:136) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HTTPSessionListenerPromise.failed(HTTPSessionListenerPromise.java:51) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.Promise$1.failed(Promise.java:94) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.ClientConnector.connectFailed(ClientConnector.java:539) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.io.ClientConnector.connect(ClientConnector.java:455) ~[jetty-io-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.HTTP2Client.connect(HTTP2Client.java:406) ~[http2-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2.connect(HttpClientTransportOverHTTP2.java:146) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2.connect(HttpClientTransportOverHTTP2.java:135) ~[http2-http-client-transport-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.connect(HttpClient.java:633) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.succeeded(HttpClient.java:610) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.client.HttpClient$1.succeeded(HttpClient.java:606) ~[jetty-client-10.0.13.jar:10.0.13] 2> at org.eclipse.jetty.util.SocketAddressResolver$Async.lambda$resolve$1(SocketAddressResolver.java:180) ~[jetty-util-10.0.13.jar:10.0.13] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289) [main/:?] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> Caused by: java.net.ConnectException: Connection refused 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:774) ~[?:?] 2> at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:120) ~[?:?] 2> at org.eclipse.jetty.io.ClientConnector.connect(ClientConnector.java:427) ~[jetty-io-10.0.13.jar:10.0.13] 2> ... 11 more 2> 710689 ERROR (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CollectionHandlingUtils Error from shard 127.0.0.1:40391_solr: {STATUS=failed} 2> 710690 ERROR (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CollectionHandlingUtils Cleaning up collection [localShardsTestColl]. 2> 710703 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CollectionHandlingUtils Executing Collection Cmd=action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true, asyncId=null 2> 710719 INFO (zkCallback-1160-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710719 INFO (zkCallback-1164-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710720 INFO (zkCallback-1160-thread-9) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710720 INFO (zkCallback-1164-thread-1) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710720 INFO (zkCallback-1160-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710720 INFO (zkCallback-1164-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710720 INFO (zkCallback-1164-thread-7) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710720 INFO (zkCallback-1160-thread-7) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710720 INFO (zkCallback-1164-thread-5) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710721 INFO (zkCallback-1166-thread-7) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710721 INFO (zkCallback-1166-thread-6) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710719 INFO (zkCallback-1160-thread-3) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710721 INFO (zkCallback-1166-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710722 INFO (zkCallback-1166-thread-2) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710722 INFO (zkCallback-1166-thread-4) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710734 INFO (zkCallback-1164-thread-8) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710740 INFO (zkCallback-1164-thread-9) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710746 INFO (zkCallback-1164-thread-10) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710780 INFO (zkCallback-1164-thread-11) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710781 INFO (zkCallback-1164-thread-12) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710785 INFO (zkCallback-1164-thread-13) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710785 INFO (zkCallback-1166-thread-9) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710793 INFO (zkCallback-1166-thread-10) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710793 INFO (zkCallback-1160-thread-10) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710796 INFO (zkCallback-1160-thread-11) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710798 INFO (zkCallback-1160-thread-12) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710812 INFO (zkCallback-1160-thread-13) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710812 INFO (zkCallback-1166-thread-11) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710816 INFO (zkCallback-1166-thread-12) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710818 INFO (zkCallback-1166-thread-13) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710821 INFO (zkCallback-1166-thread-14) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/localShardsTestColl/state.json] for collection [localShardsTestColl] has occurred - updating... (live nodes size: [0]) 2> 710825 INFO (watches-1168-thread-2) [] o.a.s.c.ZkController Replica localShardsTestColl_shard1_replica_n7 removed from clusterstate, remove it. 2> 710857 INFO (watches-1170-thread-2) [] o.a.s.c.ZkController Replica localShardsTestColl_shard1_replica_n4 removed from clusterstate, remove it. 2> 710867 INFO (watches-1172-thread-4) [] o.a.s.c.ZkController Replica localShardsTestColl_shard1_replica_n5 removed from clusterstate, remove it. 2> 710878 INFO (watches-1168-thread-2) [] o.a.s.c.ZkController Replica localShardsTestColl_shard3_replica_n6 removed from clusterstate, remove it. 2> 710879 INFO (watches-1170-thread-2) [] o.a.s.c.ZkController Replica localShardsTestColl_shard2_replica_n8 removed from clusterstate, remove it. 2> 710938 INFO (watches-1170-thread-3) [] o.a.s.c.ZkController Replica localShardsTestColl_shard3_replica_n9 removed from clusterstate, remove it. 2> 710961 INFO (watches-1172-thread-3) [] o.a.s.c.ZkController Replica localShardsTestColl_shard2_replica_n1 removed from clusterstate, remove it. 2> 711353 INFO (watches-1172-thread-3) [] o.a.s.c.ZkController Replica localShardsTestColl_shard3_replica_n3 removed from clusterstate, remove it. 2> 711562 WARN (zkCallback-1166-thread-9) [] o.a.s.c.LeaderElector Our node is no longer in line to be leader 2> 711564 WARN (zkCallback-1164-thread-11) [] o.a.s.c.LeaderElector Our node is no longer in line to be leader 2> 711590 INFO (coreCloseExecutor-1609-thread-1) [n:127.0.0.1:44127_solr c:singleShardPreferenceTestColl s:shard1 r:core_node4 x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.singleShardPreferenceTestColl.shard1.leader tag=SolrCore@7c25d19e 2> 711590 INFO (coreCloseExecutor-1609-thread-2) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.stale_state_test_col.shard1.replica_n3 tag=SolrCore@5392753e 2> 711617 INFO (coreCloseExecutor-1609-thread-1) [n:127.0.0.1:44127_solr c:singleShardPreferenceTestColl s:shard1 r:core_node4 x:singleShardPreferenceTestColl_shard1_replica_p1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 711678 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.CreateCollectionCmd Cleaned up artifacts for failed create collection for [localShardsTestColl] 2> 711682 ERROR (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.a.c.OverseerCollectionMessageHandler Collection: localShardsTestColl operation: create failed 2> => org.apache.solr.common.SolrException: Underlying core creation failed while creating collection: localShardsTestColl 2> at org.apache.solr.cloud.api.collections.CreateCollectionCmd.call(CreateCollectionCmd.java:449) 2> org.apache.solr.common.SolrException: Underlying core creation failed while creating collection: localShardsTestColl 2> at org.apache.solr.cloud.api.collections.CreateCollectionCmd.call(CreateCollectionCmd.java:449) ~[solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:126) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:566) [solr-core-9.2.1.jar:9.2.1 (not a git checkout) - builduser] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289) [main/:?] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] 2> at java.lang.Thread.run(Thread.java:829) [?:?] 2> 711754 INFO (OverseerThreadFactory-1176-thread-5) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000043 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 711773 INFO (coreCloseExecutor-1612-thread-2) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testVersionsAreReturned.shard1.replica_n2 tag=SolrCore@3fd96993 2> 711924 INFO (coreCloseExecutor-1614-thread-1) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.routing_collection.shard2.leader tag=SolrCore@4eb487c0 2> 711924 INFO (coreCloseExecutor-1614-thread-2) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.overwrite.shard1.replica_n1 tag=SolrCore@72d15c3b 2> 711941 INFO (coreCloseExecutor-1614-thread-1) [n:127.0.0.1:39159_solr c:routing_collection s:shard2 r:core_node4 x:routing_collection_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 712310 INFO (coreCloseExecutor-1612-thread-3) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.multicollection1.shard1.replica_n2 tag=SolrCore@3d962f06 2> 712338 INFO (coreCloseExecutor-1609-thread-3) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testNonRetryableRequests.shard1.replica_n1 tag=SolrCore@3cf5692a 2> 712521 INFO (coreCloseExecutor-1614-thread-3) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testVersionsAreReturned.shard2.replica_n1 tag=SolrCore@6ffd87a8 2> 712944 INFO (coreCloseExecutor-1612-thread-3) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.multicollection1.shard1.leader tag=SolrCore@3d962f06 2> 712945 INFO (coreCloseExecutor-1612-thread-4) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.multicollection2.shard1.replica_n2 tag=SolrCore@55619cdd 2> 712958 INFO (coreCloseExecutor-1609-thread-3) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testNonRetryableRequests.shard1.leader tag=SolrCore@3cf5692a 2> 712961 INFO (coreCloseExecutor-1609-thread-4) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.perReplicaState_test.shard1.replica_n7 tag=SolrCore@5a4fc389 2> 712978 INFO (coreCloseExecutor-1612-thread-3) [n:127.0.0.1:40391_solr c:multicollection1 s:shard1 r:core_node4 x:multicollection1_shard1_replica_n2] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 712994 INFO (coreCloseExecutor-1609-thread-3) [n:127.0.0.1:44127_solr c:testNonRetryableRequests s:shard1 r:core_node4 x:testNonRetryableRequests_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 713163 INFO (coreCloseExecutor-1614-thread-3) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testVersionsAreReturned.shard2.leader tag=SolrCore@6ffd87a8 2> 713164 INFO (coreCloseExecutor-1614-thread-4) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.multicollection1.shard2.replica_n1 tag=SolrCore@58e45191 2> 713176 INFO (coreCloseExecutor-1614-thread-3) [n:127.0.0.1:39159_solr c:testVersionsAreReturned s:shard2 r:core_node3 x:testVersionsAreReturned_shard2_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 713603 INFO (OverseerCollectionConfigSetProcessor-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000043 doesn't exist. Requestor may have disconnected from ZooKeeper 2> 713611 INFO (coreCloseExecutor-1609-thread-6) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.localShardsTestColl.shard2.replica_n1 tag=SolrCore@4892b4bc 2> 713636 INFO (coreCloseExecutor-1612-thread-4) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.multicollection2.shard1.leader tag=SolrCore@55619cdd 2> 713638 INFO (coreCloseExecutor-1612-thread-5) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.stateVersionParamTest.shard1.replica_n1 tag=SolrCore@78a7c0d 2> 713669 INFO (coreCloseExecutor-1612-thread-4) [n:127.0.0.1:40391_solr c:multicollection2 s:shard1 r:core_node4 x:multicollection2_shard1_replica_n2] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 713827 INFO (coreCloseExecutor-1614-thread-4) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.multicollection1.shard2.leader tag=SolrCore@58e45191 2> 713828 INFO (coreCloseExecutor-1614-thread-5) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.multicollection2.shard2.replica_n1 tag=SolrCore@738bc8a1 2> 713884 INFO (coreCloseExecutor-1614-thread-4) [n:127.0.0.1:39159_solr c:multicollection1 s:shard2 r:core_node3 x:multicollection1_shard2_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 714256 INFO (coreCloseExecutor-1609-thread-7) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard3 r:core_node12 x:localShardsTestColl_shard3_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.localShardsTestColl.shard3.replica_n3 tag=SolrCore@758620b2 2> 714267 INFO (coreCloseExecutor-1612-thread-5) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.stateVersionParamTest.shard1.leader tag=SolrCore@78a7c0d 2> 714267 INFO (coreCloseExecutor-1612-thread-7) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testParallelUpdateQTime.shard1.replica_n2 tag=SolrCore@4266fdf2 2> 714290 INFO (coreCloseExecutor-1612-thread-5) [n:127.0.0.1:40391_solr c:stateVersionParamTest s:shard1 r:core_node3 x:stateVersionParamTest_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 714444 INFO (coreCloseExecutor-1614-thread-5) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.multicollection2.shard2.leader tag=SolrCore@738bc8a1 2> 714446 INFO (coreCloseExecutor-1614-thread-6) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.stateVersionParamTest.shard2.replica_n2 tag=SolrCore@71c8f084 2> 714459 INFO (coreCloseExecutor-1614-thread-5) [n:127.0.0.1:39159_solr c:multicollection2 s:shard2 r:core_node3 x:multicollection2_shard2_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 714875 INFO (coreCloseExecutor-1612-thread-7) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testParallelUpdateQTime.shard1.leader tag=SolrCore@4266fdf2 2> 714876 INFO (coreCloseExecutor-1612-thread-6) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.singleShardPreferenceTestColl.shard1.replica_t2 tag=SolrCore@26dd0d41 2> 714897 INFO (coreCloseExecutor-1612-thread-7) [n:127.0.0.1:40391_solr c:testParallelUpdateQTime s:shard1 r:core_node4 x:testParallelUpdateQTime_shard1_replica_n2] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() . 2> 714984 INFO (coreCloseExecutor-1609-thread-7) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard3 r:core_node12 x:localShardsTestColl_shard3_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard3.leader tag=SolrCore@758620b2 2> 714985 INFO (coreCloseExecutor-1609-thread-8) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard1 r:core_node14 x:localShardsTestColl_shard1_replica_n5] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.localShardsTestColl.shard1.replica_n5 tag=SolrCore@705647a1 2> 715021 INFO (coreCloseExecutor-1609-thread-7) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard3 r:core_node12 x:localShardsTestColl_shard3_replica_n3] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 715129 INFO (coreCloseExecutor-1614-thread-6) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.stateVersionParamTest.shard2.leader tag=SolrCore@71c8f084 2> 715130 INFO (coreCloseExecutor-1614-thread-7) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.singleShardPreferenceTestColl.shard1.replica_n3 tag=SolrCore@36fd773a 2> 715144 INFO (coreCloseExecutor-1614-thread-6) [n:127.0.0.1:39159_solr c:stateVersionParamTest s:shard2 r:core_node4 x:stateVersionParamTest_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 715513 INFO (coreCloseExecutor-1612-thread-6) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.singleShardPreferenceTestColl.shard1.leader tag=SolrCore@26dd0d41 2> 715514 INFO (coreCloseExecutor-1612-thread-8) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testNonRetryableRequests.shard1.replica_n3 tag=SolrCore@4a78a4c4 2> 715534 INFO (coreCloseExecutor-1612-thread-6) [n:127.0.0.1:40391_solr c:singleShardPreferenceTestColl s:shard1 r:core_node5 x:singleShardPreferenceTestColl_shard1_replica_t2] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 715650 INFO (coreCloseExecutor-1609-thread-5) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.perReplicaState_testv2.shard1.replica_n5 tag=SolrCore@4181f1ca 2> 715700 INFO (coreCloseExecutor-1614-thread-7) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.singleShardPreferenceTestColl.shard1.leader tag=SolrCore@36fd773a 2> 715701 INFO (coreCloseExecutor-1614-thread-8) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testParallelUpdateQTime.shard2.replica_n1 tag=SolrCore@23350cdb 2> 715712 INFO (coreCloseExecutor-1614-thread-7) [n:127.0.0.1:39159_solr c:singleShardPreferenceTestColl s:shard1 r:core_node6 x:singleShardPreferenceTestColl_shard1_replica_n3] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 716148 INFO (coreCloseExecutor-1612-thread-8) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testNonRetryableRequests.shard1.leader tag=SolrCore@4a78a4c4 2> 716149 INFO (coreCloseExecutor-1612-thread-9) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testInitializationWithSolrUrls.shard1.replica_n2 tag=SolrCore@452bcaaa 2> 716161 INFO (coreCloseExecutor-1612-thread-8) [n:127.0.0.1:40391_solr c:testNonRetryableRequests s:shard1 r:core_node6 x:testNonRetryableRequests_shard1_replica_n3] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 716250 INFO (coreCloseExecutor-1609-thread-5) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.perReplicaState_testv2.shard1.leader tag=SolrCore@4181f1ca 2> 716250 INFO (coreCloseExecutor-1609-thread-2) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.stale_state_test_col.shard1.leader tag=SolrCore@5392753e 2> 716251 INFO (coreCloseExecutor-1609-thread-4) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.perReplicaState_test.shard1.leader tag=SolrCore@5a4fc389 2> 716254 INFO (coreCloseExecutor-1609-thread-6) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard2.leader tag=SolrCore@4892b4bc 2> 716254 INFO (coreCloseExecutor-1609-thread-8) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard1 r:core_node14 x:localShardsTestColl_shard1_replica_n5] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard1.leader tag=SolrCore@705647a1 2> 716260 INFO (coreCloseExecutor-1609-thread-2) [n:127.0.0.1:44127_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 716261 INFO (coreCloseExecutor-1609-thread-4) [n:127.0.0.1:44127_solr c:perReplicaState_test s:shard1 r:core_node8 x:perReplicaState_test_shard1_replica_n7] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 716282 INFO (coreCloseExecutor-1609-thread-8) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard1 r:core_node14 x:localShardsTestColl_shard1_replica_n5] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 716289 INFO (coreCloseExecutor-1609-thread-6) [n:127.0.0.1:44127_solr c:localShardsTestColl s:shard2 r:core_node10 x:localShardsTestColl_shard2_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 716320 INFO (coreCloseExecutor-1609-thread-5) [n:127.0.0.1:44127_solr c:perReplicaState_testv2 s:shard1 r:core_node6 x:perReplicaState_testv2_shard1_replica_n5] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 716332 INFO (jetty-closer-1604-thread-2) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null 2> 716359 INFO (coreCloseExecutor-1614-thread-8) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testParallelUpdateQTime.shard2.leader tag=SolrCore@23350cdb 2> 716360 INFO (coreCloseExecutor-1614-thread-9) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testNonRetryableRequests.shard1.replica_n2 tag=SolrCore@7eacec10 2> 716373 INFO (coreCloseExecutor-1614-thread-8) [n:127.0.0.1:39159_solr c:testParallelUpdateQTime s:shard2 r:core_node3 x:testParallelUpdateQTime_shard2_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() . 2> 716663 INFO (jetty-closer-1604-thread-2) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null 2> 716692 INFO (coreCloseExecutor-1612-thread-9) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testInitializationWithSolrUrls.shard1.leader tag=SolrCore@452bcaaa 2> 716693 INFO (coreCloseExecutor-1612-thread-12) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.ping_test.shard1.replica_n2 tag=SolrCore@1b1ab821 2> 716708 INFO (coreCloseExecutor-1612-thread-9) [n:127.0.0.1:40391_solr c:testInitializationWithSolrUrls s:shard1 r:core_node4 x:testInitializationWithSolrUrls_shard1_replica_n2] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 716809 INFO (jetty-closer-1604-thread-2) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null 2> 716810 INFO (jetty-closer-1604-thread-2) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null 2> 716903 INFO (coreCloseExecutor-1614-thread-9) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testNonRetryableRequests.shard1.leader tag=SolrCore@7eacec10 2> 716904 INFO (coreCloseExecutor-1614-thread-10) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testInitializationWithSolrUrls.shard2.replica_n1 tag=SolrCore@2075501c 2> 716917 INFO (coreCloseExecutor-1614-thread-9) [n:127.0.0.1:39159_solr c:testNonRetryableRequests s:shard1 r:core_node5 x:testNonRetryableRequests_shard1_replica_n2] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 717126 INFO (coreCloseExecutor-1612-thread-10) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testAliasHandling.shard1.replica_n1 tag=SolrCore@2ec5aae4 2> 717500 INFO (coreCloseExecutor-1614-thread-12) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.2nd_collection.shard2.replica_n1 tag=SolrCore@31ffb4b5 2> 717760 INFO (coreCloseExecutor-1612-thread-13) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testPerReplicaStateCollection.shard1.replica_n2 tag=SolrCore@3570cc7 2> 717865 INFO (coreCloseExecutor-1614-thread-12) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.2nd_collection.shard2.leader tag=SolrCore@31ffb4b5 2> 717865 INFO (coreCloseExecutor-1614-thread-13) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.ping_test.shard2.replica_n1 tag=SolrCore@24098180 2> 717877 INFO (coreCloseExecutor-1614-thread-12) [n:127.0.0.1:39159_solr c:2nd_collection s:shard2 r:core_node3 x:2nd_collection_shard2_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 718107 INFO (coreCloseExecutor-1612-thread-13) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testPerReplicaStateCollection.shard1.leader tag=SolrCore@3570cc7 2> 718108 INFO (coreCloseExecutor-1612-thread-14) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.perReplicaState_test.shard2.replica_n1 tag=SolrCore@1f048edb 2> 718121 INFO (coreCloseExecutor-1612-thread-13) [n:127.0.0.1:40391_solr c:testPerReplicaStateCollection s:shard1 r:core_node4 x:testPerReplicaStateCollection_shard1_replica_n2] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 718254 INFO (coreCloseExecutor-1614-thread-13) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.ping_test.shard2.leader tag=SolrCore@24098180 2> 718255 INFO (coreCloseExecutor-1614-thread-11) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testAliasHandling.shard2.replica_n2 tag=SolrCore@17f85463 2> 718266 INFO (coreCloseExecutor-1614-thread-13) [n:127.0.0.1:39159_solr c:ping_test s:shard2 r:core_node3 x:ping_test_shard2_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 718602 INFO (coreCloseExecutor-1612-thread-14) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.perReplicaState_test.shard2.leader tag=SolrCore@1f048edb 2> 718602 INFO (coreCloseExecutor-1612-thread-15) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.perReplicaState_testv2.shard2.replica_n1 tag=SolrCore@d81de9f 2> 718624 INFO (coreCloseExecutor-1612-thread-14) [n:127.0.0.1:40391_solr c:perReplicaState_test s:shard2 r:core_node2 x:perReplicaState_test_shard2_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 718725 INFO (coreCloseExecutor-1614-thread-14) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testPerReplicaStateCollection.shard2.replica_n1 tag=SolrCore@3c5a93c2 2> 719083 INFO (coreCloseExecutor-1612-thread-15) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.perReplicaState_testv2.shard2.leader tag=SolrCore@d81de9f 2> 719084 INFO (coreCloseExecutor-1612-thread-16) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n9] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.localShardsTestColl.shard3.replica_n9 tag=SolrCore@33d93e36 2> 719099 INFO (coreCloseExecutor-1612-thread-15) [n:127.0.0.1:40391_solr c:perReplicaState_testv2 s:shard2 r:core_node2 x:perReplicaState_testv2_shard2_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 719211 INFO (coreCloseExecutor-1614-thread-14) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testPerReplicaStateCollection.shard2.leader tag=SolrCore@3c5a93c2 2> 719211 INFO (coreCloseExecutor-1614-thread-15) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard2 r:core_node6 x:perReplicaState_test_shard2_replica_n5] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.perReplicaState_test.shard2.replica_n5 tag=SolrCore@7350817f 2> 719230 INFO (coreCloseExecutor-1614-thread-14) [n:127.0.0.1:39159_solr c:testPerReplicaStateCollection s:shard2 r:core_node3 x:testPerReplicaStateCollection_shard2_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 719534 INFO (coreCloseExecutor-1612-thread-18) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard1 r:core_node13 x:localShardsTestColl_shard1_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.localShardsTestColl.shard1.replica_n4 tag=SolrCore@ed81300 2> 719674 INFO (coreCloseExecutor-1614-thread-15) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard2 r:core_node6 x:perReplicaState_test_shard2_replica_n5] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.perReplicaState_test.shard2.leader tag=SolrCore@7350817f 2> 719674 INFO (coreCloseExecutor-1614-thread-16) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.perReplicaState_test.shard1.replica_n3 tag=SolrCore@25b5b6f 2> 719696 INFO (coreCloseExecutor-1614-thread-15) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard2 r:core_node6 x:perReplicaState_test_shard2_replica_n5] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 719996 INFO (coreCloseExecutor-1612-thread-17) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard2 r:core_node17 x:localShardsTestColl_shard2_replica_n8] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.localShardsTestColl.shard2.replica_n8 tag=SolrCore@195c93c9 2> 720167 INFO (coreCloseExecutor-1614-thread-16) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.perReplicaState_test.shard1.leader tag=SolrCore@25b5b6f 2> 720168 INFO (coreCloseExecutor-1614-thread-17) [n:127.0.0.1:39159_solr c:perReplicaState_testv2 s:shard2 r:core_node8 x:perReplicaState_testv2_shard2_replica_n7] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.perReplicaState_testv2.shard2.replica_n7 tag=SolrCore@25465559 2> 720190 INFO (coreCloseExecutor-1614-thread-16) [n:127.0.0.1:39159_solr c:perReplicaState_test s:shard1 r:core_node4 x:perReplicaState_test_shard1_replica_n3] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 720199 WARN (coreCloseExecutor-1612-thread-11) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.m.SolrMetricManager Could not obtain lock to modify reporters registry: solr.core.2nd_collection.shard1.replica_n2 2> 720408 WARN (coreCloseExecutor-1614-thread-18) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.m.SolrMetricManager Could not obtain lock to modify reporters registry: solr.core.localShardsTestColl.shard3.replica_n6 2> 720425 WARN (coreCloseExecutor-1614-thread-19) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.m.SolrMetricManager Could not obtain lock to modify reporters registry: solr.core.localShardsTestColl.shard1.replica_n7 2> 720426 INFO (coreCloseExecutor-1612-thread-1) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.routing_collection.shard1.leader tag=SolrCore@62cbc8f6 2> 720431 INFO (coreCloseExecutor-1612-thread-2) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testVersionsAreReturned.shard1.leader tag=SolrCore@3fd96993 2> 720438 INFO (coreCloseExecutor-1612-thread-1) [n:127.0.0.1:40391_solr c:routing_collection s:shard1 r:core_node3 x:routing_collection_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 720439 INFO (coreCloseExecutor-1612-thread-12) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.ping_test.shard1.leader tag=SolrCore@1b1ab821 2> 720439 INFO (coreCloseExecutor-1612-thread-10) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testAliasHandling.shard1.leader tag=SolrCore@2ec5aae4 2> 720441 INFO (coreCloseExecutor-1612-thread-16) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n9] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard3.leader tag=SolrCore@33d93e36 2> 720447 INFO (coreCloseExecutor-1612-thread-18) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard1 r:core_node13 x:localShardsTestColl_shard1_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard1.leader tag=SolrCore@ed81300 2> 720447 INFO (coreCloseExecutor-1612-thread-11) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.2nd_collection.shard1.leader tag=SolrCore@64cea453 2> 720466 INFO (coreCloseExecutor-1612-thread-10) [n:127.0.0.1:40391_solr c:testAliasHandling s:shard1 r:core_node3 x:testAliasHandling_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 720485 INFO (coreCloseExecutor-1612-thread-16) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n9] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 720495 INFO (coreCloseExecutor-1612-thread-17) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard2 r:core_node17 x:localShardsTestColl_shard2_replica_n8] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard2.leader tag=SolrCore@195c93c9 2> 720502 INFO (coreCloseExecutor-1612-thread-2) [n:127.0.0.1:40391_solr c:testVersionsAreReturned s:shard1 r:core_node4 x:testVersionsAreReturned_shard1_replica_n2] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 720508 INFO (coreCloseExecutor-1612-thread-12) [n:127.0.0.1:40391_solr c:ping_test s:shard1 r:core_node4 x:ping_test_shard1_replica_n2] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 720520 INFO (coreCloseExecutor-1612-thread-18) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard1 r:core_node13 x:localShardsTestColl_shard1_replica_n4] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 720554 INFO (coreCloseExecutor-1612-thread-17) [n:127.0.0.1:40391_solr c:localShardsTestColl s:shard2 r:core_node17 x:localShardsTestColl_shard2_replica_n8] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 720664 INFO (coreCloseExecutor-1614-thread-17) [n:127.0.0.1:39159_solr c:perReplicaState_testv2 s:shard2 r:core_node8 x:perReplicaState_testv2_shard2_replica_n7] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.perReplicaState_testv2.shard2.leader tag=SolrCore@25465559 2> 720664 INFO (coreCloseExecutor-1614-thread-2) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.overwrite.shard1.leader tag=SolrCore@72d15c3b 2> 720665 INFO (coreCloseExecutor-1614-thread-10) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testInitializationWithSolrUrls.shard2.leader tag=SolrCore@2075501c 2> 720665 INFO (coreCloseExecutor-1614-thread-11) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testAliasHandling.shard2.leader tag=SolrCore@17f85463 2> 720666 INFO (coreCloseExecutor-1614-thread-18) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard3.leader tag=SolrCore@32fbe23f 2> 720672 INFO (coreCloseExecutor-1614-thread-2) [n:127.0.0.1:39159_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 720673 INFO (coreCloseExecutor-1614-thread-11) [n:127.0.0.1:39159_solr c:testAliasHandling s:shard2 r:core_node4 x:testAliasHandling_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 720675 INFO (coreCloseExecutor-1614-thread-19) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard1.leader tag=SolrCore@6802a10d 2> 720675 INFO (coreCloseExecutor-1614-thread-10) [n:127.0.0.1:39159_solr c:testInitializationWithSolrUrls s:shard2 r:core_node3 x:testInitializationWithSolrUrls_shard2_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 720712 INFO (coreCloseExecutor-1614-thread-17) [n:127.0.0.1:39159_solr c:perReplicaState_testv2 s:shard2 r:core_node8 x:perReplicaState_testv2_shard2_replica_n7] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 720730 INFO (coreCloseExecutor-1612-thread-11) [n:127.0.0.1:40391_solr c:2nd_collection s:shard1 r:core_node4 x:2nd_collection_shard1_replica_n2] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 720739 INFO (jetty-closer-1604-thread-3) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null 2> 720908 INFO (coreCloseExecutor-1614-thread-18) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n6] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 720923 INFO (coreCloseExecutor-1614-thread-19) [n:127.0.0.1:39159_solr c:localShardsTestColl s:shard1 r:core_node16 x:localShardsTestColl_shard1_replica_n7] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary). 2> 720944 INFO (jetty-closer-1604-thread-1) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null 2> 721105 INFO (jetty-closer-1604-thread-3) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null 2> 721218 INFO (jetty-closer-1604-thread-3) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null 2> 721219 INFO (jetty-closer-1604-thread-3) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null 2> 721232 INFO (closeThreadPool-1616-thread-1) [] o.a.s.c.Overseer Overseer (id=72081801657778183-127.0.0.1:40391_solr-n_0000000000) closing 2> 721234 INFO (OverseerStateUpdate-72081801657778183-127.0.0.1:40391_solr-n_0000000000) [n:127.0.0.1:40391_solr] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:40391_solr 2> 721251 INFO (jetty-closer-1604-thread-1) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null 2> 721263 INFO (closeThreadPool-1616-thread-2) [] o.a.s.c.Overseer Overseer (id=72081801657778183-127.0.0.1:40391_solr-n_0000000000) closing 2> 721355 INFO (jetty-closer-1604-thread-1) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null 2> 721356 INFO (jetty-closer-1604-thread-1) [] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null 2> 721391 INFO (jetty-closer-1604-thread-3) [] o.a.s.c.Overseer Overseer (id=72081801657778183-127.0.0.1:40391_solr-n_0000000000) closing 2> 781511 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.c.ZkTestServer Shutting down ZkTestServer. 2> 781748 WARN (ZkTestServer Run Thread) [] o.a.s.c.ZkTestServer Watch limit violations: 2> Maximum concurrent create/delete watches above limit: 2> 2> 31 /solr/collections/localShardsTestColl/state.json 2> 15 /solr/collections/replicaTypesTestColl/terms/shard1 2> 13 /solr/collections/replicaTypesTestColl/terms/shard2 2> 10 /solr/collections/replicaTypesTestColl/terms/shard3 2> 9 /solr/aliases.json 2> 9 /solr/collections/singleShardPreferenceTestColl/terms/shard1 2> 9 /solr/clusterprops.json 2> 9 /solr/collections/localShardsTestColl/terms/shard1 2> 9 /solr/collections/localShardsTestColl/terms/shard3 2> 9 /solr/collections/perReplicaState_test/terms/shard1 2> 9 /solr/collections/testNonRetryableRequests/terms/shard1 2> 8 /solr/collections/stale_state_test_col/terms/shard1 2> 5 /solr/collections/perReplicaState_testv2/terms/shard1 2> 5 /solr/collections/perReplicaState_testv2/terms/shard2 2> 5 /solr/collections/localShardsTestColl/terms/shard2 2> 5 /solr/collections/perReplicaState_test/terms/shard2 2> 4 /solr/configs/conf 2> 4 /solr/packages.json 2> 4 /solr/collections/perReplicaState_test/collectionprops.json 2> 4 /solr/security.json 2> 3 /solr/collections/testInitializationWithSolrUrls/terms/shard1 2> 3 /solr/collections/multicollection1/terms/shard2 2> 3 /solr/collections/multicollection1/terms/shard1 2> 3 /solr/collections/2nd_collection/terms/shard2 2> 3 /solr/collections/multicollection2/terms/shard2 2> 3 /solr/collections/multicollection2/terms/shard1 2> 3 /solr/collections/testVersionsAreReturned/terms/shard1 2> 3 /solr/collections/routing_collection/terms/shard2 2> 3 /solr/collections/testVersionsAreReturned/terms/shard2 2> 3 /solr/collections/routing_collection/terms/shard1 2> 3 /solr/collections/overwrite/terms/shard1 2> 3 /solr/collections/testParallelUpdateQTime/terms/shard2 2> 3 /solr/collections/testParallelUpdateQTime/terms/shard1 2> 3 /solr/collections/testAliasHandling/terms/shard1 2> 2 /solr/collections/testAliasHandling/terms/shard2 2> 2 /solr/collections/ping_test/terms/shard2 2> 2 /solr/collections/ping_test/terms/shard1 2> 2 /solr/collections/testInitializationWithSolrUrls/terms/shard2 2> 2 /solr/collections/multicollection1/collectionprops.json 2> 2 /solr/collections/stale_state_test_col/collectionprops.json 2> 2 /solr/collections/testInitializationWithSolrUrls/collectionprops.json 2> 2 /solr/collections/2nd_collection/collectionprops.json 2> 2 /solr/collections/replicaTypesTestColl/collectionprops.json 2> 2 /solr/collections/2nd_collection/terms/shard1 2> 2 /solr/collections/multicollection2/collectionprops.json 2> 2 /solr/collections/testVersionsAreReturned/collectionprops.json 2> 2 /solr/collections/stateVersionParamTest/terms/shard1 2> 2 /solr/collections/stateVersionParamTest/terms/shard2 2> 2 /solr/collections/singleShardPreferenceTestColl/collectionprops.json 2> 2 /solr/collections/testAliasHandling/collectionprops.json 2> 2 /solr/collections/testPerReplicaStateCollection/terms/shard1 2> 2 /solr/collections/testPerReplicaStateCollection/terms/shard2 2> 2 /solr/collections/testParallelUpdateQTime/collectionprops.json 2> 2 /solr/collections/routing_collection/collectionprops.json 2> 2> Maximum concurrent data watches above limit: 2> 2> 68 /solr/collections/replicaTypesTestColl/state.json 2> 62 /solr/collections/localShardsTestColl/state.json 2> 57 /solr/collections/perReplicaState_test/state.json 2> 53 /solr/collections/perReplicaState_testv2/state.json 2> 47 /solr/collections/stale_state_test_col/state.json 2> 31 /solr/collections/singleShardPreferenceTestColl/state.json 2> 28 /solr/collections/testNonRetryableRequests/state.json 2> 18 /solr/collections/testParallelUpdateQTime/state.json 2> 18 /solr/collections/multicollection1/state.json 2> 18 /solr/collections/testInitializationWithSolrUrls/state.json 2> 17 /solr/collections/2nd_collection/state.json 2> 17 /solr/collections/routing_collection/state.json 2> 17 /solr/collections/multicollection2/state.json 2> 17 /solr/collections/testVersionsAreReturned/state.json 2> 17 /solr/collections/testAliasHandling/state.json 2> 16 /solr/collections/ping_test/state.json 2> 16 /solr/collections/stateVersionParamTest/state.json 2> 15 /solr/collections/testPerReplicaStateCollection/state.json 2> 12 /solr/aliases.json 2> 8 /solr/collections/overwrite/state.json 2> 2 /solr/collections/localShardsTestColl/leader_elect/shard3/election/72081801657778182-core_node15-n_0000000000 2> 2 /solr/overseer_elect/election/72081801657778183-127.0.0.1:40391_solr-n_0000000000 2> 2> Maximum concurrent children watches above limit: 2> 2> 96 /solr/collections 2> 81 /solr/overseer/collection-queue-work 2> 61 /solr/collections/perReplicaState_test/state.json 2> 43 /solr/collections/perReplicaState_testv2/state.json 2> 36 /solr/collections/replicaTypesTestColl/state.json 2> 32 /solr/collections/localShardsTestColl/state.json 2> 31 /solr/live_nodes 2> 18 /solr/collections/singleShardPreferenceTestColl/state.json 2> 17 /solr/collections/testNonRetryableRequests/state.json 2> 17 /solr/collections/stale_state_test_col/state.json 2> 12 /solr/collections/2nd_collection/state.json 2> 12 /solr/collections/testParallelUpdateQTime/state.json 2> 12 /solr/collections/ping_test/state.json 2> 12 /solr/collections/multicollection1/state.json 2> 12 /solr/collections/routing_collection/state.json 2> 12 /solr/collections/multicollection2/state.json 2> 12 /solr/collections/testVersionsAreReturned/state.json 2> 12 /solr/collections/testInitializationWithSolrUrls/state.json 2> 12 /solr/collections/testAliasHandling/state.json 2> 12 /solr/collections/stateVersionParamTest/state.json 2> 11 /solr/collections/testPerReplicaStateCollection/state.json 2> 7 /solr/collections/overwrite/state.json 2> 2> 781841 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.u.ErrorLogMuter Closing ErrorLogMuter-regex-33 after mutting 0 log messages 2> 781841 INFO (SUITE-CloudSolrClientTest-seed#[9CAA27B60427D55F]-worker) [] o.a.s.u.ErrorLogMuter Creating ErrorLogMuter-regex-34 for ERROR logs matching regex: ignore_exception 2> NOTE: leaving temporary files on disk at: /build/solr/src/solr-9.2.1/solr/solrj/build/tmp/tests-tmp/solr.client.solrj.impl.CloudSolrClientTest_9CAA27B60427D55F-001 2> Oct 03, 2023 12:22:29 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks 2> WARNING: Will linger awaiting termination of 1 leaked thread(s). 2> Oct 03, 2023 12:22:30 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.client.solrj.impl.CloudSolrClientTest: 2> 1) Thread[id=2926, name=parallelCoreAdminAPIBaseExecutor-1177-thread-12-processing-127.0.0.1:39159_solr localShardsTestColl_shard2_replica_n2 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 create, state=WAITING, group=TGRP-CloudSolrClientTest] 2> at java.base@11.0.20/jdk.internal.misc.Unsafe.park(Native Method) 2> at java.base@11.0.20/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) 2> at java.base@11.0.20/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) 2> at java.base@11.0.20/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:917) 2> at java.base@11.0.20/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1240) 2> at java.base@11.0.20/java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:959) 2> at java.management@11.0.20/com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:415) 2> at java.management@11.0.20/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1855) 2> at java.management@11.0.20/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:955) 2> at java.management@11.0.20/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:890) 2> at java.management@11.0.20/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:320) 2> at java.management@11.0.20/com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) 2> at app//org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.registerMBean(JmxMetricsReporter.java:546) 2> at app//org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.onCounterAdded(JmxMetricsReporter.java:615) 2> at app//com.codahale.metrics.MetricRegistry.notifyListenerOfAddedMetric(MetricRegistry.java:554) 2> at app//com.codahale.metrics.MetricRegistry.onMetricAdded(MetricRegistry.java:546) 2> at app//com.codahale.metrics.MetricRegistry.register(MetricRegistry.java:166) 2> at app//com.codahale.metrics.MetricRegistry.getOrAdd(MetricRegistry.java:521) 2> at app//com.codahale.metrics.MetricRegistry.counter(MetricRegistry.java:204) 2> at app//org.apache.solr.metrics.SolrMetricManager.counter(SolrMetricManager.java:686) 2> at app//org.apache.solr.metrics.SolrMetricsContext.counter(SolrMetricsContext.java:143) 2> at app//org.apache.solr.handler.RequestHandlerBase$HandlerMetrics.(RequestHandlerBase.java:192) 2> at app//org.apache.solr.handler.RequestHandlerBase.initializeMetrics(RequestHandlerBase.java:162) 2> at app//org.apache.solr.metrics.SolrCoreMetricManager.registerMetricProducer(SolrCoreMetricManager.java:139) 2> at app//org.apache.solr.core.SolrCore.registerInfoBean(SolrCore.java:3454) 2> at app//org.apache.solr.core.PluginBag.registerMBean(PluginBag.java:367) 2> at app//org.apache.solr.core.PluginBag.put(PluginBag.java:284) 2> at app//org.apache.solr.core.PluginBag.init(PluginBag.java:333) 2> at app//org.apache.solr.core.RequestHandlers.initHandlersFromConfig(RequestHandlers.java:119) 2> at app//org.apache.solr.core.SolrCore.(SolrCore.java:1138) 2> at app//org.apache.solr.core.SolrCore.(SolrCore.java:1051) 2> at app//org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1666) 2> at app//org.apache.solr.core.CoreContainer.create(CoreContainer.java:1532) 2> at app//org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:111) 2> at app//org.apache.solr.handler.admin.CoreAdminOperation$$Lambda$373/0x000000010056f840.execute(Unknown Source) 2> at app//org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:398) 2> at app//org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:354) 2> at app//org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:225) 2> at app//org.apache.solr.handler.admin.CoreAdminHandler$$Lambda$1248/0x0000000100c3ec40.call(Unknown Source) 2> at app//org.apache.solr.handler.admin.CoreAdminHandler$CoreAdminAsyncTracker.lambda$submitAsyncTask$0(CoreAdminHandler.java:445) 2> at app//org.apache.solr.handler.admin.CoreAdminHandler$CoreAdminAsyncTracker$$Lambda$1249/0x0000000100c3f040.run(Unknown Source) 2> at app//com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:212) 2> at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289) 2> at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$354/0x000000010052f040.run(Unknown Source) 2> at java.base@11.0.20/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) 2> at java.base@11.0.20/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) 2> at java.base@11.0.20/java.lang.Thread.run(Thread.java:829) 2> Oct 03, 2023 12:22:30 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll 2> INFO: Starting to interrupt leaked threads: 2> 1) Thread[id=2926, name=parallelCoreAdminAPIBaseExecutor-1177-thread-12-processing-127.0.0.1:39159_solr localShardsTestColl_shard2_replica_n2 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 create, state=WAITING, group=TGRP-CloudSolrClientTest] 2> Oct 03, 2023 12:22:32 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll 2> SEVERE: There are still zombie threads that couldn't be terminated: 2> 1) Thread[id=2926, name=parallelCoreAdminAPIBaseExecutor-1177-thread-12-processing-127.0.0.1:39159_solr localShardsTestColl_shard2_replica_n2 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 create, state=WAITING, group=TGRP-CloudSolrClientTest] 2> at java.base@11.0.20/jdk.internal.misc.Unsafe.park(Native Method) 2> at java.base@11.0.20/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) 2> at java.base@11.0.20/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) 2> at java.base@11.0.20/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:917) 2> at java.base@11.0.20/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1240) 2> at java.base@11.0.20/java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:959) 2> at java.management@11.0.20/com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:415) 2> at java.management@11.0.20/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1855) 2> at java.management@11.0.20/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:955) 2> at java.management@11.0.20/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:890) 2> at java.management@11.0.20/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:320) 2> at java.management@11.0.20/com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) 2> at app//org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.registerMBean(JmxMetricsReporter.java:546) 2> at app//org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.onCounterAdded(JmxMetricsReporter.java:615) 2> at app//com.codahale.metrics.MetricRegistry.notifyListenerOfAddedMetric(MetricRegistry.java:554) 2> at app//com.codahale.metrics.MetricRegistry.onMetricAdded(MetricRegistry.java:546) 2> at app//com.codahale.metrics.MetricRegistry.register(MetricRegistry.java:166) 2> at app//com.codahale.metrics.MetricRegistry.getOrAdd(MetricRegistry.java:521) 2> at app//com.codahale.metrics.MetricRegistry.counter(MetricRegistry.java:204) 2> at app//org.apache.solr.metrics.SolrMetricManager.counter(SolrMetricManager.java:686) 2> at app//org.apache.solr.metrics.SolrMetricsContext.counter(SolrMetricsContext.java:143) 2> at app//org.apache.solr.handler.RequestHandlerBase$HandlerMetrics.(RequestHandlerBase.java:192) 2> at app//org.apache.solr.handler.RequestHandlerBase.initializeMetrics(RequestHandlerBase.java:162) 2> at app//org.apache.solr.metrics.SolrCoreMetricManager.registerMetricProducer(SolrCoreMetricManager.java:139) 2> at app//org.apache.solr.core.SolrCore.registerInfoBean(SolrCore.java:3454) 2> at app//org.apache.solr.core.PluginBag.registerMBean(PluginBag.java:367) 2> at app//org.apache.solr.core.PluginBag.put(PluginBag.java:284) 2> at app//org.apache.solr.core.PluginBag.init(PluginBag.java:333) 2> at app//org.apache.solr.core.RequestHandlers.initHandlersFromConfig(RequestHandlers.java:119) 2> at app//org.apache.solr.core.SolrCore.(SolrCore.java:1138) 2> at app//org.apache.solr.core.SolrCore.(SolrCore.java:1051) 2> at app//org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1666) 2> at app//org.apache.solr.core.CoreContainer.create(CoreContainer.java:1532) 2> at app//org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:111) 2> at app//org.apache.solr.handler.admin.CoreAdminOperation$$Lambda$373/0x000000010056f840.execute(Unknown Source) 2> at app//org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:398) 2> at app//org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:354) 2> at app//org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:225) 2> at app//org.apache.solr.handler.admin.CoreAdminHandler$$Lambda$1248/0x0000000100c3ec40.call(Unknown Source) 2> at app//org.apache.solr.handler.admin.CoreAdminHandler$CoreAdminAsyncTracker.lambda$submitAsyncTask$0(CoreAdminHandler.java:445) 2> at app//org.apache.solr.handler.admin.CoreAdminHandler$CoreAdminAsyncTracker$$Lambda$1249/0x0000000100c3f040.run(Unknown Source) 2> at app//com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:212) 2> at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289) 2> at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$354/0x000000010052f040.run(Unknown Source) 2> at java.base@11.0.20/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) 2> at java.base@11.0.20/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) 2> at java.base@11.0.20/java.lang.Thread.run(Thread.java:829) > com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.client.solrj.impl.CloudSolrClientTest: > 1) Thread[id=2926, name=parallelCoreAdminAPIBaseExecutor-1177-thread-12-processing-127.0.0.1:39159_solr localShardsTestColl_shard2_replica_n2 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 create, state=WAITING, group=TGRP-CloudSolrClientTest] > at java.base@11.0.20/jdk.internal.misc.Unsafe.park(Native Method) > at java.base@11.0.20/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) > at java.base@11.0.20/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) > at java.base@11.0.20/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:917) > at java.base@11.0.20/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1240) > at java.base@11.0.20/java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:959) > at java.management@11.0.20/com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:415) > at java.management@11.0.20/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1855) > at java.management@11.0.20/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:955) > at java.management@11.0.20/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:890) > at java.management@11.0.20/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:320) > at java.management@11.0.20/com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) > at app//org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.registerMBean(JmxMetricsReporter.java:546) > at app//org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.onCounterAdded(JmxMetricsReporter.java:615) > at app//com.codahale.metrics.MetricRegistry.notifyListenerOfAddedMetric(MetricRegistry.java:554) > at app//com.codahale.metrics.MetricRegistry.onMetricAdded(MetricRegistry.java:546) > at app//com.codahale.metrics.MetricRegistry.register(MetricRegistry.java:166) > at app//com.codahale.metrics.MetricRegistry.getOrAdd(MetricRegistry.java:521) > at app//com.codahale.metrics.MetricRegistry.counter(MetricRegistry.java:204) > at app//org.apache.solr.metrics.SolrMetricManager.counter(SolrMetricManager.java:686) > at app//org.apache.solr.metrics.SolrMetricsContext.counter(SolrMetricsContext.java:143) > at app//org.apache.solr.handler.RequestHandlerBase$HandlerMetrics.(RequestHandlerBase.java:192) > at app//org.apache.solr.handler.RequestHandlerBase.initializeMetrics(RequestHandlerBase.java:162) > at app//org.apache.solr.metrics.SolrCoreMetricManager.registerMetricProducer(SolrCoreMetricManager.java:139) > at app//org.apache.solr.core.SolrCore.registerInfoBean(SolrCore.java:3454) > at app//org.apache.solr.core.PluginBag.registerMBean(PluginBag.java:367) > at app//org.apache.solr.core.PluginBag.put(PluginBag.java:284) > at app//org.apache.solr.core.PluginBag.init(PluginBag.java:333) > at app//org.apache.solr.core.RequestHandlers.initHandlersFromConfig(RequestHandlers.java:119) > at app//org.apache.solr.core.SolrCore.(SolrCore.java:1138) > at app//org.apache.solr.core.SolrCore.(SolrCore.java:1051) > at app//org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1666) > at app//org.apache.solr.core.CoreContainer.create(CoreContainer.java:1532) > at app//org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:111) > at app//org.apache.solr.handler.admin.CoreAdminOperation$$Lambda$373/0x000000010056f840.execute(Unknown Source) > at app//org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:398) > at app//org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:354) > at app//org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:225) > at app//org.apache.solr.handler.admin.CoreAdminHandler$$Lambda$1248/0x0000000100c3ec40.call(Unknown Source) > at app//org.apache.solr.handler.admin.CoreAdminHandler$CoreAdminAsyncTracker.lambda$submitAsyncTask$0(CoreAdminHandler.java:445) > at app//org.apache.solr.handler.admin.CoreAdminHandler$CoreAdminAsyncTracker$$Lambda$1249/0x0000000100c3f040.run(Unknown Source) > at app//com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:212) > at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289) > at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$354/0x000000010052f040.run(Unknown Source) > at java.base@11.0.20/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at java.base@11.0.20/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > at java.base@11.0.20/java.lang.Thread.run(Thread.java:829) > at __randomizedtesting.SeedInfo.seed([9CAA27B60427D55F]:0) > com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated: > 1) Thread[id=2926, name=parallelCoreAdminAPIBaseExecutor-1177-thread-12-processing-127.0.0.1:39159_solr localShardsTestColl_shard2_replica_n2 5570a893-eeb3-4a6f-b218-de371c7a71b3369451142966876 create, state=WAITING, group=TGRP-CloudSolrClientTest] > at java.base@11.0.20/jdk.internal.misc.Unsafe.park(Native Method) > at java.base@11.0.20/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) > at java.base@11.0.20/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) > at java.base@11.0.20/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:917) > at java.base@11.0.20/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1240) > at java.base@11.0.20/java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:959) > at java.management@11.0.20/com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:415) > at java.management@11.0.20/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1855) > at java.management@11.0.20/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:955) > at java.management@11.0.20/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:890) > at java.management@11.0.20/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:320) > at java.management@11.0.20/com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) > at app//org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.registerMBean(JmxMetricsReporter.java:546) > at app//org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.onCounterAdded(JmxMetricsReporter.java:615) > at app//com.codahale.metrics.MetricRegistry.notifyListenerOfAddedMetric(MetricRegistry.java:554) > at app//com.codahale.metrics.MetricRegistry.onMetricAdded(MetricRegistry.java:546) > at app//com.codahale.metrics.MetricRegistry.register(MetricRegistry.java:166) > at app//com.codahale.metrics.MetricRegistry.getOrAdd(MetricRegistry.java:521) > at app//com.codahale.metrics.MetricRegistry.counter(MetricRegistry.java:204) > at app//org.apache.solr.metrics.SolrMetricManager.counter(SolrMetricManager.java:686) > at app//org.apache.solr.metrics.SolrMetricsContext.counter(SolrMetricsContext.java:143) > at app//org.apache.solr.handler.RequestHandlerBase$HandlerMetrics.(RequestHandlerBase.java:192) > at app//org.apache.solr.handler.RequestHandlerBase.initializeMetrics(RequestHandlerBase.java:162) > at app//org.apache.solr.metrics.SolrCoreMetricManager.registerMetricProducer(SolrCoreMetricManager.java:139) > at app//org.apache.solr.core.SolrCore.registerInfoBean(SolrCore.java:3454) > at app//org.apache.solr.core.PluginBag.registerMBean(PluginBag.java:367) > at app//org.apache.solr.core.PluginBag.put(PluginBag.java:284) > at app//org.apache.solr.core.PluginBag.init(PluginBag.java:333) > at app//org.apache.solr.core.RequestHandlers.initHandlersFromConfig(RequestHandlers.java:119) > at app//org.apache.solr.core.SolrCore.(SolrCore.java:1138) > at app//org.apache.solr.core.SolrCore.(SolrCore.java:1051) > at app//org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1666) > at app//org.apache.solr.core.CoreContainer.create(CoreContainer.java:1532) > at app//org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:111) > at app//org.apache.solr.handler.admin.CoreAdminOperation$$Lambda$373/0x000000010056f840.execute(Unknown Source) > at app//org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:398) > at app//org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:354) > at app//org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:225) > at app//org.apache.solr.handler.admin.CoreAdminHandler$$Lambda$1248/0x0000000100c3ec40.call(Unknown Source) > at app//org.apache.solr.handler.admin.CoreAdminHandler$CoreAdminAsyncTracker.lambda$submitAsyncTask$0(CoreAdminHandler.java:445) > at app//org.apache.solr.handler.admin.CoreAdminHandler$CoreAdminAsyncTracker$$Lambda$1249/0x0000000100c3f040.run(Unknown Source) > at app//com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:212) > at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289) > at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$354/0x000000010052f040.run(Unknown Source) > at java.base@11.0.20/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at java.base@11.0.20/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > at java.base@11.0.20/java.lang.Thread.run(Thread.java:829) > at __randomizedtesting.SeedInfo.seed([9CAA27B60427D55F]:0) 2> NOTE: test params are: codec=Asserting(Lucene94): {multiDefault=PostingsFormat(name=Direct), a_t=Lucene90, title_s=PostingsFormat(name=Direct), id=BlockTreeOrds(blocksize=128), text=PostingsFormat(name=Asserting)}, docValues:{_version_=DocValuesFormat(name=Lucene90), multiDefault=DocValuesFormat(name=Lucene90), title_s=DocValuesFormat(name=Lucene90), intDefault=DocValuesFormat(name=Lucene90), id=DocValuesFormat(name=Lucene90), timestamp=DocValuesFormat(name=Lucene90)}, maxPointsInLeafNode=1368, maxMBSortInHeap=6.967333616392303, sim=Asserting(RandomSimilarity(queryNorm=false): {}), locale=ff-GN, timezone=Africa/Brazzaville 2> NOTE: Linux 5.10.113-g387b6863253c-dirty riscv64/Oracle Corporation 11.0.20 (64-bit)/cpus=1,threads=2,free=177765008,total=448790528 2> NOTE: All tests run in this JVM: [CollectionAdminRequestRequiredParamsTest, SolrExampleBinaryHttp2Test, SolrExampleXMLTest, SolrQueryTest, TestBatchUpdate, TestLBHttpSolrClient, TestDocumentObjectBinder, LargeVolumeBinaryJettyTest, LargeVolumeJettyTest, SolrExampleEmbeddedTest, SolrExampleStreamingBinaryHttp2Test, SolrExampleStreamingHttp2Test, SolrExampleXMLHttp2Test, TestSolrProperties, CloudHttp2SolrClientBadInputTest, CloudHttp2SolrClientMultiConstructorTest, CloudHttp2SolrClientTest, CloudSolrClientBuilderTest, CloudSolrClientMultiConstructorTest, CloudSolrClientTest] 2> NOTE: reproduce with: gradlew test --tests CloudSolrClientTest -Dtests.seed=9CAA27B60427D55F -Dtests.badapples=true -Dtests.locale=ff-GN -Dtests.timezone=Africa/Brazzaville -Dtests.asserts=true -Dtests.file.encoding=UTF-8 > Task :solr:core:test :solr:core:test (FAILURE): 4651 test(s), 2 failure(s), 202 skipped 4651 tests completed, 2 failed, 202 skipped > Task :solr:core:test FAILED > Task :solr:core:wipeTaskTemp > Task :solr:solrj:test :solr:solrj:test (FAILURE): 1090 test(s), 3 failure(s), 225 skipped 1090 tests completed, 3 failed, 225 skipped > Task :solr:solrj:test FAILED > Task :solr:solrj:wipeTaskTemp ERROR: The following test(s) have failed: - org.apache.solr.client.solrj.impl.CloudSolrClientTest.queryWithLocalShardsPreferenceRulesTest (:solr:solrj) Test output: /build/solr/src/solr-9.2.1/solr/solrj/build/test-results/test/outputs/OUTPUT-org.apache.solr.client.solrj.impl.CloudSolrClientTest.txt Reproduce with: gradlew :solr:solrj:test --tests "org.apache.solr.client.solrj.impl.CloudSolrClientTest.queryWithLocalShardsPreferenceRulesTest" -Ptests.jvms=2 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=9CAA27B60427D55F -Ptests.file.encoding=ISO-8859-1 - org.apache.solr.client.solrj.impl.CloudSolrClientTest.classMethod (:solr:solrj) Test output: /build/solr/src/solr-9.2.1/solr/solrj/build/test-results/test/outputs/OUTPUT-org.apache.solr.client.solrj.impl.CloudSolrClientTest.txt Reproduce with: gradlew :solr:solrj:test --tests "org.apache.solr.client.solrj.impl.CloudSolrClientTest.classMethod" -Ptests.jvms=2 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=9CAA27B60427D55F -Ptests.file.encoding=ISO-8859-1 - org.apache.solr.client.solrj.impl.CloudSolrClientTest.classMethod (:solr:solrj) Test output: /build/solr/src/solr-9.2.1/solr/solrj/build/test-results/test/outputs/OUTPUT-org.apache.solr.client.solrj.impl.CloudSolrClientTest.txt Reproduce with: gradlew :solr:solrj:test --tests "org.apache.solr.client.solrj.impl.CloudSolrClientTest.classMethod" -Ptests.jvms=2 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=9CAA27B60427D55F -Ptests.file.encoding=ISO-8859-1 - org.apache.solr.cloud.HttpPartitionOnCommitTest.test (:solr:core) Test output: /build/solr/src/solr-9.2.1/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.HttpPartitionOnCommitTest.txt Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.cloud.HttpPartitionOnCommitTest.test" -Ptests.jvms=2 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=9CAA27B60427D55F -Ptests.file.encoding=ISO-8859-1 - org.apache.solr.cloud.ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget (:solr:core) Test output: /build/solr/src/solr-9.2.1/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.ReplaceNodeTest.txt Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.cloud.ReplaceNodeTest.testGoodSpreadDuringAssignWithNoTarget" -Ptests.jvms=2 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=9CAA27B60427D55F -Ptests.file.encoding=ISO-8859-1 FAILURE: Build completed with 2 failures. 1: Task failed with an exception. ----------- * What went wrong: Execution failed for task ':solr:core:test'. > There were failing tests. See the results at: file:///build/solr/src/solr-9.2.1/solr/core/build/test-results/test/ * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. > Run with --scan to get full insights. ============================================================================== 2: Task failed with an exception. ----------- * What went wrong: Execution failed for task ':solr:solrj:test'. > There were failing tests. See the results at: file:///build/solr/src/solr-9.2.1/solr/solrj/build/test-results/test/ * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. > Run with --scan to get full insights. ============================================================================== * Get more help at https://help.gradle.org BUILD FAILED in 4h 49m 24s 44 actionable tasks: 26 executed, 18 up-to-date ==> ERROR: A failure occurred in check().  Aborting... ==> ERROR: Build failed, check /var/lib/archbuild/extra-riscv64/root2/build receiving incremental file list solr-9.2.1-1-riscv64-build.log solr-9.2.1-1-riscv64-check.log solr-9.2.1-1-riscv64-prepare.log sent 81 bytes received 174,969 bytes 350,100.00 bytes/sec total size is 2,232,018 speedup is 12.75