==>(B Building on scovillain(B ==>(B Checking for remote environment...(B ==>(B Syncing package to remote host...(B sending incremental file list ./ .SRCINFO 1,465B 64% 0B/s 0:00:00 2,284B 100% 819KB/s 0:00:00 (xfr#1, to-chk=7/9) .nvchecker.toml 54B 100% 54.0KB/s 0:00:00 54B 100% 54.0KB/s 0:00:00 (xfr#2, to-chk=6/9) LICENSE 646B 100% 646KB/s 0:00:00 646B 100% 646KB/s 0:00:00 (xfr#3, to-chk=5/9) PKGBUILD 3,183B 54% 3.18MB/s 0:00:00 5,893B 100% 5.89MB/s 0:00:00 (xfr#4, to-chk=4/9) REUSE.toml 375B 100% 375KB/s 0:00:00 375B 100% 375KB/s 0:00:00 (xfr#5, to-chk=3/9) python-langsmith-0.14.2-1.log 709B 100% 709KB/s 0:00:00 709B 100% 709KB/s 0:00:00 (xfr#6, to-chk=2/9) LICENSES/ sent 3,120 bytes received 240 bytes 2,240.00 bytes/sec total size is 9,471 speedup is 2.82 ==>(B Applying RISC-V patches...(B sending incremental file list ./ evaluation-timing.patch 4,054B 100% 0B/s 0:00:00 4,054B 100% 0B/s 0:00:00 (xfr#1, to-chk=1/3) riscv64.patch 635B 100% 635KB/s 0:00:00 635B 100% 635KB/s 0:00:00 (xfr#2, to-chk=0/3) sent 1,708 bytes received 69 bytes 710.80 bytes/sec total size is 4,689 speedup is 2.64 patching file PKGBUILD Hunk #1 succeeded at 68 (offset 2 lines). Hunk #2 succeeded at 136 (offset 9 lines). ==>(B Running pkgctl build --arch riscv64 on remote host...(B ==> WARNING:(B invalid architecture: riscv64(B ==>(B Updating pacman database cache(B [?25l:: Synchronizing package databases... core downloading... extra downloading... multilib downloading... [?25h==>(B Building python-langsmith(B  ->(B repo: extra(B  ->(B arch: riscv64(B  ->(B worker: felix-0(B ==>(B Building python-langsmith for [extra] (riscv64)(B ]3008;start=98ed96fae0ce481c83d2cad28e9524ff;user=felix;hostname=scovillain.felixc.at;machineid=5bc7e45048834d3a82e1c93a2f026cc1;bootid=1e2f86132edf45eaa12af2fe0200ba58;pid=512945;comm=sudo;targetuser=root;type=session\Note: in a future version of systemd-nspawn the default set of permitted socket address families will be restricted to AF_INET, AF_INET6 and AF_UNIX. Use --restrict-address-families= to configure the set of permitted socket address families, or set RestrictAddressFamilies= in a .nspawn file. ]3008;start=6c966b611d524dea891fe6fdde9453d4;user=root;hostname=scovillain.felixc.at;machineid=5bc7e45048834d3a82e1c93a2f026cc1;bootid=1e2f86132edf45eaa12af2fe0200ba58;pid=512989;comm=systemd-nspawn;container=arch-nspawn-512989;type=container\]11;?\]2;🔵 Container arch-nspawn-512989 on scovillain.felixc.at\[?25l:: Synchronizing package databases... core downloading... extra downloading... :: Starting full system upgrade... there is nothing to do [?25h[!p]104\[?7h]3008;end=6c966b611d524dea891fe6fdde9453d4\==>(B Building in chroot for [extra] (riscv64)...(B ==>(B Synchronizing chroot copy [/var/lib/archbuild/extra-riscv64/root] -> [felix-0]...(Bdone(B ]3008;start=6a1d95bd8be7472194fad5c09e46b85e;user=root;hostname=scovillain.felixc.at;machineid=5bc7e45048834d3a82e1c93a2f026cc1;bootid=1e2f86132edf45eaa12af2fe0200ba58;pid=513348;comm=sudo;targetuser=felix;type=session\==>(B Making package: python-langsmith 0.14.2-1 (Fri Nov 6 19:08:54 2026)(B ==>(B Retrieving sources...(B  ->(B Updating langsmith-sdk git repo...(B  ->(B Found evaluation-timing.patch(B ==>(B Validating source files with sha512sums...(B langsmith-sdk ... Passed evaluation-timing.patch ... Passed ]3008;end=6a1d95bd8be7472194fad5c09e46b85e\Note: in a future version of systemd-nspawn the default set of permitted socket address families will be restricted to AF_INET, AF_INET6 and AF_UNIX. Use --restrict-address-families= to configure the set of permitted socket address families, or set RestrictAddressFamilies= in a .nspawn file. ]3008;start=00167b547e374cd5b489025de5bd8b46;user=root;hostname=scovillain.felixc.at;machineid=5bc7e45048834d3a82e1c93a2f026cc1;bootid=1e2f86132edf45eaa12af2fe0200ba58;pid=514347;comm=systemd-nspawn;container=makechrootpkg-python-langsmith.build.512966;type=container\]11;?\]2;🔵 Container makechrootpkg-python-langsmith.build.512966 on scovillain.felixc.at\]3008;start=885b7a9e427f4bb49d36148c8ac7222e;user=root;hostname=makechrootpkg-python-langsmith.build.512966;machineid=f7ddc4a20a754cfcac5123f2ba16e0f3;bootid=320523153aed4f278bb53f8dfc391dd3;pid=14;comm=sudo;targetuser=builduser;type=session\==>(B Making package: python-langsmith 0.14.2-1 (Fri Nov 6 19:09:27 2026)(B ==>(B Checking runtime dependencies...(B ==>(B Installing missing dependencies...(B ]3008;start=c78095a8904d475b97a394a61b9111f6;user=builduser;hostname=makechrootpkg-python-langsmith.build.512966;machineid=f7ddc4a20a754cfcac5123f2ba16e0f3;bootid=320523153aed4f278bb53f8dfc391dd3;pid=888;comm=sudo;targetuser=root;type=session\[?25lresolving dependencies... looking for conflicting packages... Package (24) New Version Net Change extra/python-annotated-types 0.8.0-1 0.12 MiB extra/python-certifi 2026.10.23-1 0.02 MiB extra/python-charset-normalizer 3.5.1-1 0.63 MiB extra/python-h11 0.16.0-2 0.29 MiB extra/python-httpcore2 2.13.1-1 1.09 MiB extra/python-idna 3.20-1 0.68 MiB extra/python-pydantic-core 3:2.50.2-1 5.45 MiB extra/python-truststore 0.10.5-1 0.20 MiB extra/python-typing-inspection 0.4.4-1 0.13 MiB extra/python-urllib3 2.8.0-1 1.48 MiB extra/python-anyio 4.16.0-1 2.29 MiB extra/python-distro 1.9.0-4 0.20 MiB extra/python-httpx2 2.13.1-1 1.45 MiB extra/python-orjson 3.13.0-1 0.65 MiB extra/python-packaging 26.3-1 1.58 MiB extra/python-pydantic 2.14.2-1 6.22 MiB extra/python-requests 2.34.2-1 0.76 MiB extra/python-requests-toolbelt 1.0.0-6 0.44 MiB extra/python-sniffio 1.3.1-5 0.04 MiB extra/python-typing_extensions 4.16.0-1 0.53 MiB extra/python-uuid-utils 1.2.0-1 0.67 MiB extra/python-websockets 17.2-1 2.50 MiB extra/python-xxhash 4.0.2-1 0.08 MiB extra/python-zstandard 0.25.0-3 1.45 MiB Total Installed Size: 28.93 MiB :: Proceed with installation? [Y/n] checking keyring... checking package integrity... loading package files... checking for file conflicts... :: Processing package changes... installing python-idna... installing python-typing_extensions... installing python-anyio... Optional dependencies for python-anyio python-outcome: trio backend python-pytest: pytest plugin python-trio: trio backend python-uvloop: use uvloop for asyncio backend installing python-distro... installing python-h11... installing python-truststore... Optional dependencies for python-truststore python-urllib3: when injecting into SSLContext via urllib3 [pending] installing python-httpcore2... Optional dependencies for python-httpcore2 python-anyio: asyncio support [installed] python-h2: HTTP/2 support python-socksio: SOCKS proxy support python-trio: trio async support installing python-httpx2... Optional dependencies for python-httpx2 python-brotli: for brotli response decompression python-brotlicffi: for brotli response decompression python-click: command line client support python-h2: HTTP/2 support python-pygments: command line client support python-rich: command line client support python-socksio: SOCKS proxy support python-trio: alternative async library python-wsproto: WebSocket support installing python-orjson... installing python-packaging... installing python-annotated-types... installing python-typing-inspection... installing python-pydantic-core... installing python-pydantic... Optional dependencies for python-pydantic mypy: for type validation with mypy python-dotenv: for .env file support python-email-validator: for email validation python-hypothesis: for hypothesis plugin when using legacy v1 installing python-charset-normalizer... installing python-urllib3... Optional dependencies for python-urllib3 python-brotli: Brotli support python-brotlicffi: Brotli support python-h2: HTTP/2 support python-pysocks: SOCKS support installing python-certifi... installing python-requests... Optional dependencies for python-requests python-chardet: alternative character encoding library python-pysocks: SOCKS proxy support installing python-requests-toolbelt... installing python-sniffio... installing python-uuid-utils... installing python-websockets... Optional dependencies for python-websockets python-trio: trio backend support python-werkzeug: routing support installing python-xxhash... installing python-zstandard... Optional dependencies for python-zstandard python-cffi :: Running post-transaction hooks... (1/1) Arming ConditionNeedsUpdate... [?25h]3008;end=c78095a8904d475b97a394a61b9111f6\==>(B Checking buildtime dependencies...(B ==>(B Installing missing dependencies...(B ]3008;start=0c5e07d965da4ba7b71a355631703c1b;user=builduser;hostname=makechrootpkg-python-langsmith.build.512966;machineid=f7ddc4a20a754cfcac5123f2ba16e0f3;bootid=320523153aed4f278bb53f8dfc391dd3;pid=1018;comm=sudo;targetuser=root;type=session\[?25lresolving dependencies... looking for conflicting packages... warning: insufficient columns available for table display Packages (63) abseil-cpp-20260817.0-2 gtest-1.18.0-1 libyaml-0.2.5-3 perl-error-0.17031-1 perl-mailtools-2.22-3 perl-timedate-2.35-1 protobuf-36.1-1 python-cffi-2.1.1-1 python-editables-0.6-1 python-googleapis-common-protos-1.76.0-1 python-httpcore-1.0.9-3 python-iniconfig-2.3.0-1 python-jsonpatch-1.41-1 python-jsonpointer-3.8-1 python-langchain-protocol-0.0.21-1 python-langsmith-0.14.1-1 python-markdown-it-py-4.2.0-1 python-marshmallow-4.4.0-1 python-mdurl-0.1.2-9 python-mypy_extensions-1.1.0-2 python-opentelemetry-exporter-http-transport-1.45.2-1 python-opentelemetry-exporter-otlp-common-1.45.2-1 python-opentelemetry-exporter-otlp-proto-common-1.45.2-1 python-opentelemetry-proto-1.45.2-1 python-opentelemetry-semantic-conventions-1.45.2-1 python-pathspec-1.1.1-1 python-pluggy-1.7.0-1 python-protobuf-36.1-1 python-pycparser-3.12-1 python-pygments-2.21.0-1 python-pyproject-hooks-1.3.3-1 python-tenacity-9.2.1-1 python-tomlkit-0.15.2-1 python-trove-classifiers-2026.9.21.13-1 python-typing_inspect-0.9.0-7 python-wrapt-2.5.2-1 python-yaml-6.0.3-2 zlib-ng-2.3.3-1 git-2.56.0-1 python-attrs-26.1.0-1 python-build-1.6.1-1 python-cachetools-7.2.8-1 python-cryptography-50.0.2-1 python-dataclasses-json-0.6.7-1 python-hatchling-1.32.4-1 python-httpx-0.28.1-7 python-installer-1.1.0-1 python-langchain-core-1.6.9-1 python-multipart-2.0.1-1 python-opentelemetry-api-1.45.2-1 python-opentelemetry-exporter-otlp-proto-http-1.45.2-1 python-opentelemetry-sdk-1.45.2-1 python-pyjwt-2.15.1-1 python-pytest-1:9.2.0-1 python-pytest-asyncio-1.4.1-1 python-pytest-httpx-0.36.2-1 python-pytest-rerunfailures-16.7-1 python-pytest-retry-1.7.0-1 python-pytest-socket-0.8.1-1 python-pytest-timeout-2.5.0-1 python-rich-15.0.0-1 python-vcrpy-9.3.2-1 python-wheel-0.48.0-1 Total Download Size: 1.15 MiB Total Installed Size: 141.55 MiB :: Proceed with installation? [Y/n] :: Retrieving packages... python-langchain-core-1.6.9-1-any downloading... python-vcrpy-9.3.2-1-any downloading... checking keyring... checking package integrity... loading package files... checking for file conflicts... :: Processing package changes... installing perl-error... installing perl-timedate... installing perl-mailtools... installing zlib-ng... installing git... Optional dependencies for git git-zsh-completion: upstream zsh completion tk: gitk and git gui openssh: ssh transport and crypto man: show help with `git command --help` perl-libwww: git svn perl-term-readkey: git svn and interactive.singlekey setting perl-io-socket-ssl: git send-email TLS support perl-authen-sasl: git send-email TLS support perl-cgi: gitweb (web interface) support python: git svn & git p4 [installed] subversion: git svn org.freedesktop.secrets: keyring credential helper libsecret: libsecret credential helper [installed] less: the default pager for git installing python-pyproject-hooks... installing python-build... Optional dependencies for python-build python-pip: to use as the Python package installer (default) python-uv: to use as the Python package installer python-virtualenv: to use virtualenv for build isolation installing python-editables... installing python-pathspec... Optional dependencies for python-pathspec python-google-re2: re2 backend installing python-pluggy... installing python-tomlkit... installing python-trove-classifiers... installing python-hatchling... installing python-installer... installing python-wheel... Optional dependencies for python-wheel python-keyring: for wheel.signatures python-xdg: for wheel.signatures python-setuptools: for legacy bdist_wheel subcommand installing python-attrs... installing python-cachetools... installing python-pycparser... installing python-cffi... Optional dependencies for python-cffi python-setuptools: "limited api" version checking in cffi.setuptools_ext installing python-cryptography... installing python-marshmallow... installing python-mypy_extensions... installing python-typing_inspect... installing python-dataclasses-json... installing python-httpcore... Optional dependencies for python-httpcore python-h2: for HTTP/2 support python-socksio: for SOCKS support python-anyio: for asyncio backend [installed] python-trio: for trio backend python-sniffio: for async support [installed] installing python-httpx... Optional dependencies for python-httpx python-brotli: for brotli response decompression python-brotlicffi: for brotli response decompression python-zstandard: for zstd response decompression [installed] python-h2: HTTP/2 support python-socksio: SOCKS proxy support python-click: command line client support python-rich: command line client support [pending] python-pygments: command line client support [pending] python-trio: alternative async library installing python-jsonpointer... installing python-jsonpatch... installing python-langchain-protocol... installing python-langsmith... Optional dependencies for python-langsmith python-langchain-core: LangChain runnable integration support [pending] python-cachetools: LiveKit and Pipecat tracing support [installed] python-pytest: LangSmith pytest plugin support [pending] python-rich: enhanced pytest output support [pending] python-vcrpy: VCR support [pending] python-opentelemetry-api: OpenTelemetry tracing support [pending] python-opentelemetry-sdk: OpenTelemetry tracing support [pending] python-opentelemetry-exporter-otlp-proto-http: OpenTelemetry export support [pending] installing python-tenacity... Optional dependencies for python-tenacity python-tornado: for Tornado coroutines support installing libyaml... installing python-yaml... installing python-langchain-core... installing python-multipart... installing python-opentelemetry-api... installing gtest... Optional dependencies for gtest python: gmock generator [installed] installing abseil-cpp... installing protobuf... installing python-protobuf... installing python-googleapis-common-protos... Optional dependencies for python-googleapis-common-protos python-grpcio: for grpc support installing python-opentelemetry-exporter-http-transport... Optional dependencies for python-opentelemetry-exporter-http-transport python-requests: requests HTTP transport [installed] python-urllib3: urllib3 HTTP transport [installed] installing python-opentelemetry-semantic-conventions... installing python-opentelemetry-sdk... installing python-opentelemetry-exporter-otlp-common... Optional dependencies for python-opentelemetry-exporter-otlp-common python-opentelemetry-exporter-http-transport: OTLP HTTP client support [installed] installing python-opentelemetry-proto... installing python-opentelemetry-exporter-otlp-proto-common... installing python-opentelemetry-exporter-otlp-proto-http... installing python-pyjwt... installing python-iniconfig... installing python-pygments... installing python-pytest... installing python-pytest-asyncio... installing python-pytest-httpx... installing python-pytest-rerunfailures... Optional dependencies for python-pytest-rerunfailures python-pytest-xdist: for recovering from crashes installing python-pytest-retry... installing python-pytest-socket... installing python-pytest-timeout... installing python-mdurl... installing python-markdown-it-py... Optional dependencies for python-markdown-it-py python-mdit_py_plugins: core plugins python-linkify-it-py: linkify extension installing python-rich... installing python-wrapt... installing python-vcrpy... :: Running post-transaction hooks... (1/4) Creating system user accounts... Creating group 'git' with GID 968. Creating user 'git' (git daemon user) with UID 968 and GID 968. (2/4) Reloading system manager configuration... Skipped: Current root is not booted. (3/4) Arming ConditionNeedsUpdate... (4/4) Checking for old perl modules... WARNING: '/usr/lib/perl5/5.42' contains data from at least 1 packages which will NOT be used by the installed perl interpreter. -> Run the following command to get a list of affected packages: pacman -Qqo '/usr/lib/perl5/5.42' [?25h]3008;end=0c5e07d965da4ba7b71a355631703c1b\==>(B Retrieving sources...(B  ->(B Found evaluation-timing.patch(B ==> WARNING:(B Skipping all source file integrity checks.(B ==>(B Extracting sources...(B  ->(B Creating working copy of langsmith-sdk git repo...(B Cloning into 'langsmith-sdk'... done. Updating files: 84% (902/1073) Updating files: 85% (913/1073) Updating files: 86% (923/1073) Updating files: 87% (934/1073) Updating files: 88% (945/1073) Updating files: 89% (955/1073) Updating files: 90% (966/1073) Updating files: 91% (977/1073) Updating files: 92% (988/1073) Updating files: 93% (998/1073) Updating files: 94% (1009/1073) Updating files: 95% (1020/1073) Updating files: 96% (1031/1073) Updating files: 97% (1041/1073) Updating files: 98% (1052/1073) Updating files: 99% (1063/1073) Updating files: 100% (1073/1073) Updating files: 100% (1073/1073), done. Switched to a new branch 'makepkg' ==>(B Starting prepare()...(B patching file tests/unit_tests/evaluation/test_runner.py ==>(B Starting build()...(B * Getting build dependencies for wheel... * Building wheel... Successfully built langsmith-0.14.2-py3-none-any.whl ==>(B Starting check()...(B /usr/lib/python3.14/site-packages/pytest_asyncio/plugin.py:300: PytestDeprecationWarning: The configuration option "asyncio_default_fixture_loop_scope" is unset. The event loop scope for asynchronous fixtures will default to the "fixture" caching scope. Future versions of pytest-asyncio will default the loop scope for asynchronous fixtures to "function" scope. Set the default fixture loop scope explicitly in order to avoid unexpected behavior in the future. Valid fixture loop scopes are: "function", "class", "module", "package", "session" warnings.warn(PytestDeprecationWarning(_DEFAULT_FIXTURE_LOOP_SCOPE_UNSET)) ============================= test session starts ============================== platform linux -- Python 3.14.9, pytest-9.2.0, pluggy-1.7.0 rootdir: /build/python-langsmith/src/langsmith-sdk/python configfile: pyproject.toml plugins: socket-0.8.1, rerunfailures-16.7, httpx-0.36.2, langsmith-0.14.1, retry-1.7.0, anyio-4.16.0, asyncio-1.4.1, timeout-2.4.0 asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function timeout: 120.0s timeout method: thread timeout func_only: False collected 2609 items / 8 deselected / 2 skipped / 2601 selected tests/unit_tests/evaluation/test_async_experiment_results.py ....... [ 0%] tests/unit_tests/evaluation/test_evaluator.py ....................... [ 1%] tests/unit_tests/evaluation/test_key_extraction.py ........... [ 1%] tests/unit_tests/evaluation/test_runner.py .....................F....... [ 2%] ........................................................................ [ 5%] ....... [ 5%] tests/unit_tests/evaluation/test_sync_experiment_results.py ........ [ 6%] tests/unit_tests/sandbox/test_access_delegation.py .......... [ 6%] tests/unit_tests/sandbox/test_async_client.py .......................... [ 7%] ....................................... [ 8%] tests/unit_tests/sandbox/test_async_sandbox.py ......................... [ 9%] ........ [ 10%] tests/unit_tests/sandbox/test_async_ws_execute.py ...................... [ 11%] ........................ [ 11%] tests/unit_tests/sandbox/test_aws_role_auth.py ......................... [ 12%] ...... [ 13%] tests/unit_tests/sandbox/test_client.py ................................ [ 14%] ................................................................ [ 16%] tests/unit_tests/sandbox/test_command_handle.py .... [ 16%] tests/unit_tests/sandbox/test_command_id_retry.py ............ [ 17%] tests/unit_tests/sandbox/test_exceptions.py ...................... [ 18%] tests/unit_tests/sandbox/test_helpers.py ..... [ 18%] tests/unit_tests/sandbox/test_models.py .............................. [ 19%] tests/unit_tests/sandbox/test_proxy_config.py .......................... [ 20%] ........................ [ 21%] tests/unit_tests/sandbox/test_run_config_and_files.py .................. [ 22%] ..................... [ 23%] tests/unit_tests/sandbox/test_sandbox.py ............................... [ 24%] .. [ 24%] tests/unit_tests/sandbox/test_service_access.py ....... [ 24%] tests/unit_tests/sandbox/test_sse_execute.py ........................... [ 25%] ......... [ 25%] tests/unit_tests/sandbox/test_sync_async_conversion.py ......... [ 26%] tests/unit_tests/sandbox/test_transport.py ............................. [ 27%] ................. [ 28%] tests/unit_tests/sandbox/test_tunnel.py ................................ [ 29%] . [ 29%] tests/unit_tests/sandbox/test_verify.py ................................ [ 30%] ....... [ 30%] tests/unit_tests/sandbox/test_ws_execute.py ............................ [ 31%] ......................................... [ 33%] tests/unit_tests/sandbox/test_yamux.py ....................... [ 34%] tests/unit_tests/secret/test_secret_integration.py ........... [ 34%] tests/unit_tests/secret/test_secret_serde.py ........................... [ 35%] ............................... [ 37%] tests/unit_tests/secret/test_secret_type.py ............................ [ 38%] .................. [ 38%] tests/unit_tests/test_address.py ....................................... [ 40%] .......................... [ 41%] tests/unit_tests/test_anonymizer.py .................................... [ 42%] ............. [ 43%] tests/unit_tests/test_async_client.py .......................... [ 44%] tests/unit_tests/test_background_thread.py .. [ 44%] tests/unit_tests/test_beta_evals.py . [ 44%] tests/unit_tests/test_client.py ........................................ [ 45%] ........................................................................ [ 48%] ........................................................................ [ 51%] ........................................................................ [ 54%] .................................................................sss.... [ 56%] .............. [ 57%] tests/unit_tests/test_custom_headers.py ...... [ 57%] tests/unit_tests/test_env.py ................. [ 58%] tests/unit_tests/test_expect.py . [ 58%] tests/unit_tests/test_experiment_metadata.py .......... [ 58%] tests/unit_tests/test_httpx_compat.py ... [ 58%] tests/unit_tests/test_hub_methods.py ................................... [ 60%] ........... [ 60%] tests/unit_tests/test_insights_report.py .... [ 60%] tests/unit_tests/test_ls_agent_type.py ....... [ 61%] tests/unit_tests/test_multipart_attachment_reencode.py ...... [ 61%] tests/unit_tests/test_multipart_retry_matrix.py ........................ [ 62%] .....................x [ 63%] tests/unit_tests/test_multiple_endpoints.py .......... [ 63%] tests/unit_tests/test_oauth_refresh_lock.py ......... [ 63%] tests/unit_tests/test_operations.py ....... [ 64%] tests/unit_tests/test_otel_exporter.py ........................ [ 65%] tests/unit_tests/test_otel_url.py ................. [ 65%] tests/unit_tests/test_profiles_oauth_discovery.py ........... [ 66%] tests/unit_tests/test_prompt_cache.py .................................. [ 67%] ................................... [ 68%] tests/unit_tests/test_pytest_plugin.py .... [ 68%] tests/unit_tests/test_replica_compression.py ........................... [ 69%] ............ [ 70%] tests/unit_tests/test_replica_endpoints.py ............................. [ 71%] ............................. [ 72%] tests/unit_tests/test_replica_run_ids.py .... [ 72%] tests/unit_tests/test_replica_serialization.py . [ 72%] tests/unit_tests/test_run_helpers.py ................................... [ 74%] .ss..................................................................... [ 76%] ................... [ 77%] tests/unit_tests/test_run_tree_gc_leak.py ..... [ 77%] tests/unit_tests/test_run_trees.py ..................................... [ 79%] .. [ 79%] tests/unit_tests/test_runtime_overrides.py ............ [ 79%] tests/unit_tests/test_span_utils.py .. [ 79%] tests/unit_tests/test_testing.py .... [ 80%] tests/unit_tests/test_utils.py ..................... [ 80%] tests/unit_tests/test_uuid_v7.py ......... [ 81%] tests/unit_tests/test_v2_migration_utils.py ............................ [ 82%] ........ [ 82%] tests/unit_tests/wrappers/test_anthropic_processing.py ................. [ 83%] ...... [ 83%] tests/unit_tests/wrappers/test_claude_agent_sdk_hooks.py ............... [ 84%] ............................. [ 85%] tests/unit_tests/wrappers/test_gemini_live.py ............... [ 85%] tests/unit_tests/wrappers/test_gemini_processing.py .................... [ 86%] ..................... [ 87%] tests/unit_tests/wrappers/test_google_adk_live.py ...................... [ 88%] ............................... [ 89%] tests/unit_tests/wrappers/test_livekit.py .............................. [ 90%] ........................................................................ [ 93%] ........................ [ 94%] tests/unit_tests/wrappers/test_openai_processing.py .................... [ 94%] .. [ 95%] tests/unit_tests/wrappers/test_openai_realtime.py ............. [ 95%] tests/unit_tests/wrappers/test_openai_realtime_agents.py ............... [ 96%] ...... [ 96%] tests/unit_tests/wrappers/test_pipecat.py .............................. [ 97%] ...................................... [ 98%] tests/unit_tests/wrappers/test_voice_helpers.py ........ [ 99%] tests/unit_tests/wrappers/test_voice_session.py ................... [100%] =================================== FAILURES =================================== ___________________ test_aevaluate_results[True-True-False] ____________________ self = verb = 'POST', endpoint = 'http://localhost:1984/runs/query', args = () kwargs = {'stream': False, 'timeout': (10.0, 60.0), 'data': b'{"session":["853b71e0-2e4c-41a3-8933-1648f205eb72"],"is_root":tru...': 'langsmith-py/0.14.2-httpx2', 'Accept': 'application/json', 'x-api-key': '123', 'Content-Type': 'application/json'}} def request(self, verb: str, endpoint: str, *args, **kwargs): if verb == "GET": if endpoint == "http://localhost:1984/datasets": res = MagicMock() res.json.return_value = { "id": self.ds_id, "created_at": "2021-09-01T00:00:00Z", "name": self.ds_name, } return res elif endpoint == "http://localhost:1984/examples": res = MagicMock() res.json.return_value = [ e.dict() if not isinstance(e, dict) else e for e in self.ds_examples ] return res elif endpoint == "http://localhost:1984/sessions": res = {} # type: ignore if kwargs["params"]["name"] == self.created_session["name"]: # type: ignore res = self.created_session # type: ignore response = MagicMock() response.json.return_value = res return response elif ( endpoint == f"http://localhost:1984/sessions/{self.created_session['id']}" ): # type: ignore res = self.created_session # type: ignore response = MagicMock() response.json.return_value = res return response else: self.should_fail = True raise ValueError(f"Unknown endpoint: {endpoint}") elif verb == "POST": if endpoint == "http://localhost:1984/sessions": self.created_session = json.loads(kwargs["data"]) | { "tenant_id": self.tenant_id } response = MagicMock() response.json.return_value = self.created_session return response elif ( endpoint == "http://localhost:1984/runs/batch" or endpoint == "http://localhost:1984/runs/multipart" ): loaded_runs = parse_request_data(kwargs["data"]) posted = loaded_runs.get("post", []) patched = loaded_runs.get("patch", []) for p in posted: self.runs[p["id"]] = p for p in patched: self.runs[p["id"]].update(p) response = MagicMock() return response elif endpoint == "http://localhost:1984/runs/query": res = MagicMock() res.json.return_value = { "runs": [ r > for r in self.runs.values() ^^^^^^^^^^^^^^^^^^ if r["trace_id"] == r["id"] and r.get("reference_example_id") ] } E RuntimeError: dictionary changed size during iteration tests/unit_tests/evaluation/test_runner.py:118: RuntimeError The above exception was the direct cause of the following exception: blocking = False, as_runnable = True, upload_results = True @pytest.mark.skipif(sys.version_info < (3, 9), reason="requires python3.9 or higher") @pytest.mark.parametrize("blocking", [False, True]) @pytest.mark.parametrize("as_runnable", [False, True]) @pytest.mark.parametrize("upload_results", [False, True]) async def test_aevaluate_results( blocking: bool, as_runnable: bool, upload_results: bool ) -> None: session = mock.Mock() ds_name = "my-dataset" ds_id = "00886375-eb2a-4038-9032-efff60309896" SPLIT_SIZE = 3 NUM_REPETITIONS = 4 ds_example_responses = [_create_example(i) for i in range(10)] ds_examples = [e[0] for e in ds_example_responses] dev_split = random.sample(ds_examples, SPLIT_SIZE) tenant_id = str(uuid.uuid4()) fake_request = FakeRequest( ds_id, ds_name, [e[1] for e in ds_example_responses], tenant_id ) session.request = fake_request.request client = Client( api_url="http://localhost:1984", api_key="123", session=session, info=ls_schemas.LangSmithInfo( batch_ingest_config=ls_schemas.BatchIngestConfig( size_limit_bytes=None, # Note this field is not used here size_limit=100, scale_up_nthreads_limit=16, scale_up_qsize_trigger=1000, scale_down_nempty_trigger=4, ) ), ) client._tenant_id = uuid.UUID(tenant_id) ordering_of_stuff: List[str] = [] locked = False lock = asyncio.Lock() slow_index = None async def predict(inputs: dict) -> dict: nonlocal locked nonlocal slow_index async with lock: delay = len(ordering_of_stuff) > 2 and not locked if delay: locked = True else: ordering_of_stuff.append("predict") if delay: await asyncio.sleep(3) # Yield until the other evaluations finish, with a bounded wait. deadline = time.monotonic() + 30 while len(ordering_of_stuff) != ( SPLIT_SIZE * NUM_REPETITIONS - 1 ) * (len(evaluators) + 1): if time.monotonic() >= deadline: raise TimeoutError("Other evaluations did not finish") await asyncio.sleep(0.1) slow_index = len(ordering_of_stuff) ordering_of_stuff.append("predict") return {"output": inputs["in"] + 1} if as_runnable: try: from langchain_core.runnables import RunnableLambda except ImportError: pytest.skip("langchain-core not installed.") return else: predict = RunnableLambda(predict) async def score_value_first(run, example): ordering_of_stuff.append("evaluate") return {"score": 0.3} async def score_unpacked_inputs_outputs(inputs, outputs): ordering_of_stuff.append("evaluate") return {"score": outputs["output"]} async def score_unpacked_inputs_outputs_reference( inputs, outputs, reference_outputs ): ordering_of_stuff.append("evaluate") return {"score": reference_outputs["answer"]} async def score_unpacked_inputs_outputs_attachments(inputs, outputs, attachments): ordering_of_stuff.append("evaluate") return {"score": outputs["output"]} async def score_unpacked_outputs(outputs): ordering_of_stuff.append("evaluate") return {"score": outputs["output"]} async def eval_float(run, example): ordering_of_stuff.append("evaluate") return 0.2 async def eval_str(run, example): ordering_of_stuff.append("evaluate") return "good" async def eval_list(run, example): ordering_of_stuff.append("evaluate") return [ {"score": True, "key": "list_eval_bool"}, {"score": 1, "key": "list_eval_int"}, ] def summary_eval_runs_examples(runs_, examples_): return {"score": len(runs_[0].dotted_order)} def summary_eval_inputs_outputs(inputs, outputs): return {"score": len([x["in"] for x in inputs])} def summary_eval_outputs_reference(outputs, reference_outputs): return {"score": len([x["answer"] for x in reference_outputs])} evaluators = [ score_value_first, score_unpacked_inputs_outputs, score_unpacked_inputs_outputs_reference, score_unpacked_inputs_outputs_attachments, score_unpacked_outputs, eval_float, eval_str, eval_list, ] summary_evaluators = [ summary_eval_runs_examples, summary_eval_inputs_outputs, summary_eval_outputs_reference, ] results = await aevaluate( predict, client=client, data=dev_split, evaluators=evaluators, summary_evaluators=summary_evaluators, num_repetitions=NUM_REPETITIONS, blocking=blocking, upload_results=upload_results, max_concurrency=None, ) if not blocking: deltas: list = [] start = time.time() last = start now = None async for _ in results: now = time.time() deltas.append((now - last)) last = now total = now - start # type: ignore # Include the bounded wait for other evaluations on slower builders. assert 40 > total > 1.5 # Essentially we want to check that most calls were very fast. assert len(deltas) == SPLIT_SIZE * NUM_REPETITIONS total_quick = sum([d < 0.5 for d in deltas]) total_slow = sum([d > 0.5 for d in deltas]) tolerance = 3 assert total_slow < tolerance assert total_quick > (SPLIT_SIZE * NUM_REPETITIONS - 1) - tolerance assert max(deltas) > (total / 3) async for r in results: assert r["run"].outputs["output"] == r["example"].inputs["in"] + 1 # type: ignore assert set(r["run"].outputs.keys()) == {"output"} # type: ignore assert all( er.score is not None or er.value is not None for er in r["evaluation_results"]["results"] ) assert len(results._summary_results["results"]) == len(summary_evaluators) N_PREDS = SPLIT_SIZE * NUM_REPETITIONS if upload_results: assert fake_request.created_session _wait_until(lambda: fake_request.runs) _wait_until(lambda: len(ordering_of_stuff) == N_PREDS * (len(evaluators) + 1)) _wait_until(lambda: slow_index is not None) # Want it to be interleaved assert ordering_of_stuff[:N_PREDS] != ["predict"] * N_PREDS assert slow_index is not None # The delayed prediction follows all other predictions and their evaluators. assert slow_index == (N_PREDS - 1) * (len(evaluators) + 1) assert fake_request.created_session["name"] else: assert not fake_request.created_session async def score_value(run, example): return {"score": 0.7} if upload_results: > ex_results = await aevaluate( fake_request.created_session["name"], evaluators=[score_value], client=client, blocking=blocking, ) tests/unit_tests/evaluation/test_runner.py:895: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ langsmith/evaluation/_arunner.py:299: in aevaluate return await aevaluate_existing( langsmith/evaluation/_arunner.py:464: in aevaluate_existing runs = await aitertools.aio_to_thread( langsmith/_internal/_aiter.py:363: in aio_to_thread return await _default_aio_to_thread(ctx, func, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ langsmith/_internal/_aiter.py:376: in _default_aio_to_thread return await loop.run_in_executor(None, func_call) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ /usr/lib/python3.14/concurrent/futures/thread.py:86: in run result = ctx.run(self.task) ^^^^^^^^^^^^^^^^^^ /usr/lib/python3.14/concurrent/futures/thread.py:73: in run return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ langsmith/evaluation/_runner.py:1213: in _load_traces_for_experiment return list(runs) ^^^^^^^^^^ langsmith/client.py:4631: in list_runs for i, run in enumerate( langsmith/client.py:2265: in _get_cursor_paginated_list response = self.request_with_retries( _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = Client (API URL: http://localhost:1984), method = 'POST' pathname = '/runs/query' request_kwargs = {'timeout': (10.0, 60.0), 'data': b'{"session":["853b71e0-2e4c-41a3-8933-1648f205eb72"],"is_root":true,"select":["app_...': 'langsmith-py/0.14.2-httpx2', 'Accept': 'application/json', 'x-api-key': '123', 'Content-Type': 'application/json'}} stop_after_attempt = 1, retry_on = None, to_ignore = None handle_response = None, _context = '', kwargs = {} def request_with_retries( self, /, method: Literal["GET", "POST", "PUT", "PATCH", "DELETE"], pathname: str, *, request_kwargs: Optional[Mapping] = None, stop_after_attempt: int = 1, retry_on: Optional[Sequence[type[BaseException]]] = None, to_ignore: Optional[Sequence[type[BaseException]]] = None, handle_response: Optional[Callable[[requests.Response, int], Any]] = None, _context: str = "", **kwargs: Any, ) -> requests.Response: """Send a request with retries. Args: method: The HTTP request method. pathname: The pathname of the request URL. Will be appended to the API URL. request_kwargs: Additional request parameters. stop_after_attempt: The number of attempts to make. retry_on: The exceptions to retry on. In addition to: `[LangSmithConnectionError, LangSmithAPIError]`. to_ignore: The exceptions to ignore / pass on. handle_response: A function to handle the response and return whether to continue retrying. _context: The context of the request. **kwargs: Additional keyword arguments to pass to the request. Returns: The response object. Raises: LangSmithAPIError: If a server error occurs. LangSmithUserError: If the request fails. LangSmithConnectionError: If a connection error occurs. LangSmithError: If the request fails. """ self._ensure_profile_auth() request_kwargs = request_kwargs or {} headers = { **self._headers, **request_kwargs.get("headers", {}), **kwargs.get("headers", {}), } if self._profile_auth is not None: headers = self._profile_auth.prepare_request_headers(headers) request_kwargs = { "timeout": self._timeout, **request_kwargs, **kwargs, "headers": headers, } if ( method != "GET" and "data" in request_kwargs and "files" not in request_kwargs and not request_kwargs["headers"].get("Content-Type") ): request_kwargs["headers"]["Content-Type"] = "application/json" logging_filters = [ ls_utils.FilterLangSmithRetry(), ls_utils.FilterPoolFullWarning(host=str(self._host)), ] retry_on_: tuple[type[BaseException], ...] = ( *(retry_on or ()), *( ls_utils.LangSmithConnectionError, ls_utils.LangSmithRequestTimeout, # 408 ls_utils.LangSmithAPIError, # 500 ), ) to_ignore_: tuple[type[BaseException], ...] = (*(to_ignore or ()),) response = None for idx in range(stop_after_attempt): try: try: with ls_utils.filter_logs(_urllib3_logger, logging_filters): response = self.session.request( method, _construct_url(self.api_url, pathname), stream=False, **request_kwargs, ) ls_utils.raise_for_status_with_text(response) return response except requests.exceptions.ReadTimeout as e: logger.debug("Passing on exception %s", e) if idx + 1 == stop_after_attempt: raise sleep_time = 2**idx + (random.random() * 0.5) time.sleep(sleep_time) continue except requests.HTTPError as e: if response is not None: if handle_response is not None: if idx + 1 < stop_after_attempt: should_continue = handle_response(response, idx + 1) if should_continue: continue if response.status_code == 500: raise ls_utils.LangSmithAPIError( f"Server error ({response.status_code}) caused failure to {method}" f" {pathname} in" f" LangSmith API. {repr(e)}" f"{_context}" ) elif response.status_code == 408: raise ls_utils.LangSmithRequestTimeout( f"Client took too long to send request to {method}" f"{pathname} {_context}" ) elif response.status_code == 429: raise ls_utils.LangSmithRateLimitError( f"Rate limit exceeded for {pathname}. {repr(e)}" f"{_context}" ) elif response.status_code == 401: raise ls_utils.LangSmithAuthError( f"Authentication failed for {pathname}. {repr(e)}" f"{_context}" ) elif response.status_code == 404: raise ls_utils.LangSmithNotFoundError( f"Resource not found for {pathname}. {repr(e)}" f"{_context}" ) elif response.status_code == 409: raise ls_utils.LangSmithConflictError( f"Conflict for {pathname}. {repr(e)}{_context}" ) elif response.status_code == 403: try: error_data = response.json() error_code = error_data.get("error", "") if error_code == "org_scoped_key_requires_workspace": raise ls_utils.LangSmithUserError( "This API key is org-scoped and requires workspace specification. " "Please provide 'workspace_id' parameter, " "or set LANGSMITH_WORKSPACE_ID environment variable." ) except (ValueError, KeyError): pass raise ls_utils.LangSmithError( f"Failed to {method} {pathname} in LangSmith" f" API. {repr(e)}" ) else: raise ls_utils.LangSmithError( f"Failed to {method} {pathname} in LangSmith" f" API. {repr(e)}" ) else: raise ls_utils.LangSmithUserError( f"Failed to {method} {pathname} in LangSmith API. {repr(e)}" ) except requests.ConnectionError as e: recommendation = ( "Please confirm your LANGCHAIN_ENDPOINT." if self.api_url != "https://api.smith.langchain.com" else "Please confirm your internet connection." ) try: content_length = int( str(e.request.headers.get("Content-Length")) if e.request else "" ) size_rec = self._content_above_size(content_length) if size_rec: recommendation = size_rec except ValueError: content_length = None api_key = ( e.request.headers.get("x-api-key") or "" if e.request else "" ) prefix, suffix = api_key[:5], api_key[-2:] filler = "*" * (max(0, len(api_key) - 7)) masked_api_key = f"{prefix}{filler}{suffix}" raise ls_utils.LangSmithConnectionError( f"Connection error caused failure to {method} {pathname}" f" in LangSmith API. {recommendation}" f" {repr(e)}" f"\nContent-Length: {content_length}" f"\nAPI Key: {masked_api_key}" f"{_context}" ) from e except Exception as e: args = list(e.args) msg = args[1] if len(args) > 1 else "" msg = msg.replace("session", "session (project)") if args: emsg = "\n".join( [str(args[0])] + [msg] + [str(arg) for arg in (args[2:] if len(args) > 2 else [])] ) else: emsg = msg > raise ls_utils.LangSmithError( f"Failed to {method} {pathname} in LangSmith API. {emsg}" f"{_context}" ) from e E langsmith.utils.LangSmithError: Failed to POST /runs/query in LangSmith API. dictionary changed size during iteration langsmith/client.py:2176: LangSmithError ----------------------------- Captured stdout call ----------------------------- View the evaluation results for experiment: 'healthy-stretch-51' at: http://localhost/o/c929b1c3-99e7-45aa-a81e-e78130a2c957/datasets/00886375-eb2a-4038-9032-efff60309896/compare?selectedSessions=853b71e0-2e4c-41a3-8933-1648f205eb72 ------------------------------ Captured log call ------------------------------- WARNING langsmith.client:_background_thread.py:575 Run compression is not enabled. Please update to the latest version of LangSmith. Falling back to regular multipart ingestion. =============================== warnings summary =============================== ../../../../../usr/lib/python3.14/site-packages/_pytest/config/__init__.py:1507 /usr/lib/python3.14/site-packages/_pytest/config/__init__.py:1507: PytestAssertRewriteWarning: Module already imported so cannot be rewritten; anyio self._mark_plugins_for_rewrite(hook, disable_autoload) ../../../../../usr/lib/python3.14/site-packages/_pytest/assertion/rewrite.py:185 /usr/lib/python3.14/site-packages/_pytest/assertion/rewrite.py:185: DeprecationWarning: langsmith.wrappers._openai_agents is deprecated and has been moved to langsmith.integrations.openai_agents_sdk. Please update your imports. exec(co, module.__dict__) tests/unit_tests/evaluation/test_runner.py: 51 warnings /build/python-langsmith/src/langsmith-sdk/python/langsmith/evaluation/_arunner.py:508: DeprecationWarning: 'asyncio.iscoroutinefunction' is deprecated and slated for removal in Python 3.16; use inspect.iscoroutinefunction() instead asyncio.iscoroutinefunction(target) tests/unit_tests/evaluation/test_runner.py: 26 warnings /build/python-langsmith/src/langsmith-sdk/python/langsmith/evaluation/_arunner.py:1384: DeprecationWarning: 'asyncio.iscoroutinefunction' is deprecated and slated for removal in Python 3.16; use inspect.iscoroutinefunction() instead if not asyncio.iscoroutinefunction(target) and not _is_langchain_runnable(target): tests/unit_tests/evaluation/test_runner.py: 28 warnings /build/python-langsmith/src/langsmith-sdk/python/langsmith/client.py:8456: DeprecationWarning: The following arguments are no longer used in the create_feedback endpoint: ['metadata'] warnings.warn( tests/unit_tests/evaluation/test_runner.py::test_evaluate_results[False-False-False] /build/python-langsmith/src/langsmith-sdk/python/langsmith/evaluation/_runner.py:408: LangSmithBetaWarning: 'upload_results' parameter is in beta. _warn_once("'upload_results' parameter is in beta.") tests/unit_tests/test_address.py::TestPrecedence::test_tracing_context_address_beats_a_decorator_project tests/unit_tests/test_client.py::test_agent_addressed_run_sends_no_project /build/python-langsmith/src/langsmith-sdk/python/langsmith/_internal/_agent_addressing.py:76: LangSmithBetaWarning: Addressing runs with `langsmith.address` is in beta and is enabled per workspace. A workspace without it rejects the run, so the trace is lost rather than falling back to a project. The behavior may change without notice. _warn_once( tests/unit_tests/test_async_client.py::test_async_create_feedback_includes_trace_id_and_feedback_id /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_async_client.py:432: LangSmithWarning: Creating feedback for a run without session_id is deprecated and will stop working in a future release. See https://docs.langchain.com/langsmith/smithdb-sdk-migration-feedback#feedback-create feedback = await client.create_feedback( tests/unit_tests/test_async_client.py::test_list_runs_child_run_ids_deprecation_warning /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_async_client.py:704: DeprecationWarning: list_runs() is deprecated and will be removed after Jan 31, 2027. Use client.runs.query() instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration-query-runs#runs-query for the migration guide. async for _ in client.list_runs( tests/unit_tests/test_beta_evals.py::test_compute_test_metrics_deprecation_warning /build/python-langsmith/src/langsmith-sdk/python/langsmith/_internal/_beta_decorator.py:85: LangSmithBetaWarning: Function compute_test_metrics is in beta. return func(*args, **kwargs) tests/unit_tests/test_client.py: 123 warnings /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_client.py:926: DeprecationWarning: 'asyncio.iscoroutinefunction' is deprecated and slated for removal in Python 3.16; use inspect.iscoroutinefunction() instead and not asyncio.iscoroutinefunction(getattr(Client, method)) tests/unit_tests/test_client.py: 123 warnings /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_client.py:933: DeprecationWarning: 'asyncio.iscoroutinefunction' is deprecated and slated for removal in Python 3.16; use inspect.iscoroutinefunction() instead and asyncio.iscoroutinefunction(getattr(Client, method)) tests/unit_tests/test_client.py::test_create_feedback_string_source_type[api] tests/unit_tests/test_client.py::test_create_feedback_string_source_type[model] /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_client.py:2180: LangSmithWarning: Creating feedback for a run without session_id is deprecated and will stop working in a future release. See https://docs.langchain.com/langsmith/smithdb-sdk-migration-feedback#feedback-create client.create_feedback( tests/unit_tests/test_client.py::test_create_feedback_can_opt_out_of_extending_trace_retention /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_client.py:2201: LangSmithWarning: Creating feedback for a run without session_id is deprecated and will stop working in a future release. See https://docs.langchain.com/langsmith/smithdb-sdk-migration-feedback#feedback-create client.create_feedback( tests/unit_tests/test_client.py::test_create_feedback_opt_out_uses_direct_post_when_batching_available /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_client.py:2239: LangSmithWarning: Creating feedback for a run without session_id is deprecated and will stop working in a future release. See https://docs.langchain.com/langsmith/smithdb-sdk-migration-feedback#feedback-create client.create_feedback( tests/unit_tests/test_client.py::test_feedback_sampling_follows_run_id /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_client.py:3118: LangSmithWarning: Creating feedback for a run without session_id is deprecated and will stop working in a future release. See https://docs.langchain.com/langsmith/smithdb-sdk-migration-feedback#feedback-create sampled_feedback = client.create_feedback( tests/unit_tests/test_client.py::test_feedback_sampling_follows_run_id /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_client.py:3127: LangSmithWarning: Creating feedback for a run without session_id is deprecated and will stop working in a future release. See https://docs.langchain.com/langsmith/smithdb-sdk-migration-feedback#feedback-create filtered_feedback = client.create_feedback( tests/unit_tests/test_client.py::test_feedback_sampling_follows_trace_id_not_run_id /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_client.py:3199: LangSmithWarning: Creating feedback for a run without session_id is deprecated and will stop working in a future release. See https://docs.langchain.com/langsmith/smithdb-sdk-migration-feedback#feedback-create client.create_feedback( tests/unit_tests/test_client.py::test_feedback_sampling_follows_trace_id_not_run_id /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_client.py:3209: LangSmithWarning: Creating feedback for a run without session_id is deprecated and will stop working in a future release. See https://docs.langchain.com/langsmith/smithdb-sdk-migration-feedback#feedback-create client.create_feedback( tests/unit_tests/test_client.py::test_select_eval_results /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_client.py:3270: PydanticDeprecatedSince20: The `dict` method is deprecated; use `model_dump` instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.14/migration/ (1, expected.dict()), tests/unit_tests/test_client.py::test_select_eval_results /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_client.py:3272: PydanticDeprecatedSince20: The `dict` method is deprecated; use `model_dump` instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.14/migration/ (1, {"results": [expected.dict()]}), tests/unit_tests/test_client.py::test_select_eval_results tests/unit_tests/test_client.py::test_select_eval_results /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_client.py:3273: PydanticDeprecatedSince20: The `dict` method is deprecated; use `model_dump` instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.14/migration/ (2, {"results": [expected.dict(), expected.dict()]}), tests/unit_tests/test_client.py::test_select_eval_results /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_client.py:3289: PydanticDeprecatedSince20: The `dict` method is deprecated; use `model_dump` instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.14/migration/ **expected2.dict(exclude={"comment"}), tests/unit_tests/test_client.py::test_create_feedback_with_zstd_compression /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_client.py:4851: LangSmithWarning: Creating feedback for a run without session_id is deprecated and will stop working in a future release. See https://docs.langchain.com/langsmith/smithdb-sdk-migration-feedback#feedback-create client.create_feedback( tests/unit_tests/test_client.py::test_list_runs_child_run_ids_deprecation_warning /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_client.py:6668: DeprecationWarning: list_runs() is deprecated and will be removed after Jan 31, 2027. Use client.runs.query() instead. See https://docs.langchain.com/langsmith/smithdb-sdk-migration-query-runs#runs-query for the migration guide. client.list_runs( tests/unit_tests/test_client.py::TestAConfiguredProjectTravelsWithTheAgent::test_a_per_call_project_still_wins /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_client.py:8214: LangSmithWarning: LANGSMITH_AGENT_ID ('my-agent') and a configured project ('my-proj') are both set in the environment, so calls that name no destination in code are not traced. Unset one of them. return Client( tests/unit_tests/test_client.py::TestFeedbackAgentAddressing::test_create_feedback_never_rejects_addressing[kwargs0] tests/unit_tests/test_client.py::TestFeedbackAgentAddressing::test_create_feedback_never_rejects_addressing[kwargs1] /build/python-langsmith/src/langsmith-sdk/python/langsmith/client.py:8456: DeprecationWarning: The following arguments are no longer used in the create_feedback endpoint: ['agent_id'] warnings.warn( tests/unit_tests/test_client.py::TestFeedbackAgentAddressing::test_create_feedback_never_rejects_addressing[kwargs2] /build/python-langsmith/src/langsmith-sdk/python/langsmith/client.py:8456: DeprecationWarning: The following arguments are no longer used in the create_feedback endpoint: ['agent_environment'] warnings.warn( tests/unit_tests/test_insights_report.py::test_get_insights_report_basic_metadata /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_insights_report.py:93: LangSmithBetaWarning: Function get_insights_report is in beta. result = client.get_insights_report( tests/unit_tests/test_replica_compression.py::test_feedback_is_kept_out_of_a_replica_owned_frame /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_replica_compression.py:265: LangSmithWarning: Creating feedback for a run without session_id is deprecated and will stop working in a future release. See https://docs.langchain.com/langsmith/smithdb-sdk-migration-feedback#feedback-create h.client.create_feedback(run.id, key="score", score=1.0, trace_id=run.trace_id) tests/unit_tests/test_replica_compression.py::test_feedback_shares_the_frame_when_no_replica_owns_it /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_replica_compression.py:276: LangSmithWarning: Creating feedback for a run without session_id is deprecated and will stop working in a future release. See https://docs.langchain.com/langsmith/smithdb-sdk-migration-feedback#feedback-create h.client.create_feedback(run.id, key="score", score=1.0, trace_id=run.trace_id) tests/unit_tests/test_run_trees.py::test_json_serializable /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_run_trees.py:110: PydanticDeprecatedSince20: The `dict` method is deprecated; use `model_dump` instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.14/migration/ d = run_tree.dict() tests/unit_tests/test_run_trees.py::test_json_serializable /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_run_trees.py:113: PydanticDeprecatedSince20: The `dict` method is deprecated; use `model_dump` instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.14/migration/ d = run_tree.dict() tests/unit_tests/test_run_trees.py::test_json_serializable /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_run_trees.py:115: PydanticDeprecatedSince20: The `json` method is deprecated; use `model_dump_json` instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.14/migration/ d = json.loads(run_tree.json()) tests/unit_tests/test_run_trees.py::test_json_serializable /build/python-langsmith/src/langsmith-sdk/python/langsmith/client.py:758: LangSmithMissingAPIKeyWarning: API key must be provided when using hosted LangSmith API warnings.warn( tests/unit_tests/test_run_trees.py::test_json_serializable /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_run_trees.py:118: PydanticDeprecatedSince20: The `dict` method is deprecated; use `model_dump` instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.14/migration/ d = run_tree.dict() tests/unit_tests/test_run_trees.py::test_json_serializable /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_run_trees.py:120: PydanticDeprecatedSince20: The `json` method is deprecated; use `model_dump_json` instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.14/migration/ d = json.loads(run_tree.json()) tests/unit_tests/test_run_trees.py::test_json_serializable /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_run_trees.py:123: PydanticDeprecatedSince20: The `dict` method is deprecated; use `model_dump` instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.14/migration/ d = run_tree.dict() tests/unit_tests/test_run_trees.py::test_json_serializable /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/test_run_trees.py:125: PydanticDeprecatedSince20: The `json` method is deprecated; use `model_dump_json` instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.14/migration/ d = json.loads(run_tree.json()) tests/unit_tests/wrappers/test_livekit.py::TestAutomaticSessionReport::test_configured_processor_installs_the_same_automatic_hook /build/python-langsmith/src/langsmith-sdk/python/tests/unit_tests/wrappers/test_livekit.py:737: LangSmithBetaWarning: Function configure_livekit is in beta. proc = configure_livekit(downstream_processor=MagicMock()) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html =========================== short test summary info ============================ FAILED tests/unit_tests/evaluation/test_runner.py::test_aevaluate_results[True-True-False] = 1 failed, 2594 passed, 7 skipped, 8 deselected, 1 xfailed, 390 warnings in 435.92s (0:07:15) = ==> ERROR:(B A failure occurred in check().(B  Aborting...(B ]3008;end=885b7a9e427f4bb49d36148c8ac7222e\[!p]104\[?7h]3008;end=00167b547e374cd5b489025de5bd8b46\==> ERROR:(B Build failed, check /var/lib/archbuild/extra-riscv64/felix-0/build(B [?12l[?25h[?12l[?25h]3008;end=98ed96fae0ce481c83d2cad28e9524ff\[?12l[?25hreceiving incremental file list python-langsmith-0.14.2-1-riscv64-build.log python-langsmith-0.14.2-1-riscv64-check.log python-langsmith-0.14.2-1-riscv64-prepare.log sent 81 bytes received 10,434 bytes 7,010.00 bytes/sec total size is 51,339 speedup is 4.88