Zi`}%SrSSKJr SSKrSSKrSSKrSSKJr SSK r SSK r SSK r SSK r SSK r SSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKJrJrJrJrJ r SSK J!r!J"r" SSK#J$r$ SSK%J&r& SSK'J(r(J)r) SS KJ*r*J+r+J,r,J-r-J.r.J/r/J0r0J1r1J2r2J3r3 SS K4J5r6 SSK7r7SSK8r8SS K9J:r: SS K8J;r< SS K=J>r? SSK@JArAJBrB SSKCJDrD SSKEJFrF SSKGrGSSKGJHrI SSKGJJrK SSKGJLrM SSKNJOrO SSKPJQrQ SSKPJRrS SSKTJUrU SSKVJWrW SSKXJYrYJZrZJ[r[J\r\J]r] SSK^J_r_J`r`JaraJbrbJcrc SSKdJereJfrfJgrg SSKhJiriJjrjJkrkJlrlJmrmJnrnJoroJprp SSKqJrrs SSKtJuru SSKvJwrwJxrx SS KyJzrzJ{r{ \R"\}5r~S!rSqS"q\GR"5r\.S#r\"\3"\55rS$\S%'StS&jrSuS'jrS(rS)rS*rSS+.SvS,jjrS-rSS.KJr SS1KJr \*(aASSKrSS4KJr SSKGJJrJ SS5KJr SS6KJr SS7KJr SS8KJrJrJr SS9KJrJrJrJrJrJrJrJr \R"\}5r~\R"S:5rS;rS'\"5rS?\$"\D5GRd;rS@r\"5rSA\SB'\!"SCSD9SwSEj5rSSFSGSH.SxSIjjrSySJjrSzSKjrS{S|SLjjrS}SMjr\1\GRz\4r\.SNr\ RB"SCSD9S~SOj5rSSPjrSQSR.SSSjjrSSTjrSSSUjjrSSVjrSSSWjjr\RSSXj5r\RSYSZ.SS[jj5rS\SZ.SS]jjr\ RB"SCSD9S^5r"S_S`\ deprecated ``otel_enabled`` argument > ``LANGSMITH_TRACING_MODE`` env var > legacy ``OTEL_ENABLED`` / ``OTEL_ONLY`` env vars > default ``"langsmith"``. TRACING_MODE OTEL_ENABLED OTEL_ONLYzInvalid tracing_mode=z. Must be one of: , zThe 'otel_enabled' parameter is deprecated and will be removed in the next minor version. Use 'tracing_mode' instead, e.g. Client(tracing_mode="hybrid") or Client(tracing_mode="otel").r\ stacklevelrGrHrFzInvalid LANGSMITH_TRACING_MODE=zBoth LANGSMITH_z and the legacy LANGSMITH_z / LANGSMITH_z env vars are set. LANGSMITH_z takes precedence.) ls_utils get_env_varlowerrILangSmithUserErrorjoinsortedwarningswarn FutureWarningis_env_var_truish) tracing_moderamode_envvar_nameotel_enabled_envvar_nameotel_only_envvar_nameenv_mode legacy_otel legacy_onlys rV_resolve_tracing_moderzs&-'##$45H#))+ 3 3--' '78##'99V4H-I#J"KM   L    ))*?@@>># / /--1(>##'99V4H-I#J"KM 001IJ 001FG + MM!"2!34567234-..@B  !!"788!!.11 rZcnSSKJn SSKJn SSKJn SSKJn XX#4$![a [S5ef=f)z,Dynamically import OTEL modules when needed.rtrace)set_span_in_context)get_otlp_tracer_provider OTELExporterzKTo use OTEL tracing, you must install it with `pip install langsmith[otel]`) opentelemetryr}opentelemetry.tracer~%langsmith._internal.otel._otel_clientr'langsmith._internal.otel._otel_exporterr ImportError) otel_tracer~rrs rV _import_otelrsC  5;  I0HVV   Y   s4)ZoneInfoc\rSrSrSrSrg)rzIntroduced in python 3.9.rYN__name__ __module__ __qualname____firstlineno____doc____static_attributes__rYrZrVrrs'rZr)TracerProviderc\rSrSrSrSrg)rzUsed for optional OTEL tracing.rYNrrYrZrVrrs-rZr)Runnabler|r evaluator) AEVALUATOR_T ATARGET_TAsyncExperimentResults)COMPARATIVE_EVALUATOR_TDATA_T EVALUATOR_T EXPERIMENT_TSUMMARY_EVALUATOR_TTARGET_TComparativeExperimentResultsExperimentResultszurllib3.connectionpool x-api-keyrYztuple[dict, ...] EMPTY_SEQ key_blocksizez:How are people using my agent? What are they asking about?zset[str]_fallback_dirs_createdrKmaxsizecSnSSKJn [RR U5nU[RR S5:=(a" U[RR S5:=(d" U[RR S5:nU$![ [ [4a!n[RSU5 SnAU$SnAff=f) zCheck if the installed `langchain_core.load.load` supports the `allowed_objects` parameter. Returns `True` if `langchain-core >= 0.3.81` and `< 1.0`, or `>= 1.2.5`. Fr) __version__z0.3.81z1.0.0z1.2.5zbFailed to determine langchain-core version for allowed_objects support, defaulting to disabled: %sN) langchain_corer packagingversionrr ValueError TypeErrorrQdebug)allowed_objects_supportedr lc_versionexcs rV&_lc_load_allowed_objects_arg_supportedr/s !& .&&,,[9  )++11(; ; >Y..44W==%>I--33G<< " %$ Y /   2   %$ sBBC4CCr]2depth max_depth max_widthc^^^TS:a [S5eTS:a [S5eTT:ag[U[5(a[U51Sk:Xa USS:XagTS-T:Xag[U[5(a:[ UUU4S j[ R "UR5T555$[U[[45(a[ UUU4S jUS T55$g) zidlctypersecretTc3B># UHn[UTS-TTS9v M g7frKrN_manifest_has_secrets.0xrrrs rV (_manifest_has_secrets..\s0 D "i9 Dc3B># UHn[UTS-TTS9v M g7frrrs rVrrcs. * "i9 *rN) r isinstancedictsetany itertoolsislicevaluestuplelist)manifestrrrs ```rVrrIs1}6771}677 8T"" M1 1 V  ( i  Hd # # %%hoo&7C    Hudm , , jy)    rZcj[R"U5un nUS:waU(d [S5egg)N-zPulling a public prompt by owner/name is disabled by default because prompts may contain untrusted serialized LangChain objects. If you trust this prompt, set `dangerously_pull_public_prompt=True` to acknowledge the risk.)riparse_prompt_identifierr)prompt_identifierdangerously_pull_public_promptowner_s rV_validate_public_prompt_pullrms>223DEKE1a |: $  ;|rZc  SSKJn SSKJn SSKJn SSKJn SSKJ n SSK J n J n SS KJn 0n [%5(aU(aS OS U S 'U "5 U"UR&4UUS.U D6n SSS5 [/W U5(d+[/X5(a[/U R0U5(a[/X5(aU O8[/X5(a'[/U R0U5(a U R0OSnUc[2R4"S5eUR6c0UlUR6R9UR:UR<UR>S.5 U(Gay[/X5(Gah[/U R0U5(GaL[AU RB5S:XGa2[/U RDU5(Gd[/U RDU 5(a[/U RDRFU5(a[IXR0U RDRF-5n[AURB5S:XaUURBSnU "U R0URJ"S00U RDRLED6URD5n U $Un U $[/U RDU5(aU R0U RD-n U $U $![a [S5ef=f![a [ R"S 5n GNf=f![(a=n[+UR&5(aU(dU(d[-SU35eUeSnAff=f!,(df  GN =f)a@Process a prompt manifest into a LangChain prompt object. This is the common logic shared between `Client.pull_prompt()` and `AsyncClient.pull_prompt()`. Args: prompt_object: The prompt commit object containing the manifest. include_model: Whether to include model information. secrets: Map of secrets to use when loading. secrets_from_env: Whether to load secrets from environment variables. Returns: The processed prompt object. Raises: ImportError: If `langchain-core` is not installed. ValueError: If secrets are required but not provided. r)BaseLanguageModel)load)BaseOutputParser)BasePromptTemplateStructuredPromptRunnableBindingRunnableSequencezuThe client.pull_prompt function requires the langchain-core package to run. Install with `pip install langchain-core`)suppress_langchain_beta_warningc3# Sv g7fNrYrYrZrVrA_process_prompt_manifest..suppress_langchain_beta_warnings sallcoreallowed_objects) secrets_mapsecrets_from_envaFailed to load prompt. The prompt manifest contains secrets (like API keys or access tokens) but no secrets were provided. This is due to a security patch in langsmith 0.5.1 that disabled reading secrets from environment variables by default. To resolve this: - Recommended: Pass secrets directly via `secrets={'KEY_NAME': 'value'}` - If this is a trusted prompt: Set `secrets_from_env=True` to read secrets from environment variables Underlying error: Nz-Prompt object is not a valid prompt template.) lc_hub_owner lc_hub_repolc_hub_commit_hashr\rKrY)'#langchain_core.language_models.baserlangchain_core.load.loadrlangchain_core.output_parsersrlangchain_core.promptsr!langchain_core.prompts.structuredrlangchain_core.runnables.baserrrlangchain_core._apir contextlibcontextmanagerrr ExceptionrrrfirstriLangSmithErrormetadataupdaterrepo commit_hashlenstepslastboundrbindkwargs) prompt_object include_modelsecretsrrrrrrrrr load_kwargsprompteprompt_templateseq rebound_llms rV_process_prompt_manifestrzs'2  I1B=FS GK-//2?V %& ( * &&#!1 F +8 6-.. f / / v||%7 8 8&55 f77v||-?@@    "))?   # # +')O $  '' - 3 3,11&3&?&?    v 0 0 v||%5 6 6  ! #Jv{{DT,U,U fkk? 3 3 KK  09 9 ' 8I8I)IJC399~"!iil )LL$$>'=&++*<*<'=>HH M M  %6 7 7'-||fkk'AF M  MM   I      " "  # $ %m&<&<==( *+,.  G#  + *sF&K*LM4L**L L'&L'* M148M,,M11M44 Nc[U[R5(d[R"U5(a U[U54$[ R "[U55nURRS5n[U5U:aXR*n[USS9 O[R"SUSU35eURS:XaSnX4$URS:XaS nX4$![a Nf=f) z*Parse a public dataset URL or share token./z token parts)varzInvalid public z URL: zsmith.langchain.comhttps://api.smith.langchain.comzbeta.smith.langchain.comz$https://beta.api.smith.langchain.com)ruuidUUIDstrr urllib_parseurlparsepathsplitr_as_uuidrirlnetloc) url_or_tokenapi_url num_partskind parsed_url path_parts token_uuids rV_parse_token_or_urlr(s lDII . .$))L2I2IC -- -3J &&s<'89J&&s+J :)# + /))OD6 ~*VWW113     8 88  !    sAC)) C65C6c[R"U5RRS5SnUS:H=(d UR S5$![ a gf=f)zCheck if the URL is langchain hosted. Args: url: The URL to check. Returns: `True` if the URL is langchain hosted, `False` otherwise. :rz langchain.comz.langchain.comF)rurlsplitr rendswithr)urlr s rV_is_langchain_hostedr. sY&&s+2288=a@(MFOO)>s)C DEP*. &'  " " 2\ 22rZcN[RS5 UR5 g)z=Close the session. Args: session: The session to close. zClosing Client.sessionN)rQrclose)sessions rV close_sessionrMVs  LL)* MMOrZrFrscU(dY[U5(aHUS:waA[R"5(a&[R"S[R 5 ggggg)aPVerify API key is provided if url not localhost. Args: api_url: The API URL. api_key: The API key. tracing_mode: Resolved tracing mode; when ``"otel"`` the warning is suppressed because no LangSmith REST calls are made. Raises: LangSmithUserError: If the API key is not provided when using the hosted service. rGz8API key must be provided when using hosted LangSmith APIN)r.ritracing_is_enabledrorpLangSmithMissingAPIKeyWarning)r"api_keyrss rV_validate_api_key_if_hostedrS`sU"   ) )&++-- MMJ66 .' * rZcH[U[5(a [US5$U$)zFormat a feedback score by truncating numerical values to 4 decimal places. Args: score: The score to format, can be a number or any other type Returns: The formatted score )rfloatround)scores rV_format_feedback_scorerY}s"%UA LrZcUc[R"S5nU(dgO [U5n[U5nUS:dUS:a[R"SU35eU$)zHGet the tracing sampling rate. Returns: The tracing sampling rate. NTRACING_SAMPLING_RATErrKzELANGSMITH_TRACING_SAMPLING_RATE must be between 0 and 1 if set. Got: )rirjrrVrl)tracing_sampling_ratesampling_rate_str sampling_rates rV_get_tracing_sampling_rater_sy$$001HI ! 56+,MqMA-)) "O %   rZcU=(d 0n0nUR5Hup#UR5nU(d[R"S5eUR5RS5RS5R S5nUR5RS5RS5n[ XE5 XQU'M U$)Nz&LangSmith runs API URL cannot be empty"'r)itemsstriprirlrstriprS)_write_api_urlsprocessed_write_api_urlsr-rR processed_urlprocessed_api_keys rV_get_write_api_urlsrjs%+O!'--/  --.VW W%++-33C8>>sCJJ3O #MMO11#6<XPl[TU] XX45 gr)r~super__init__)selfpool_connections pool_maxsizery pool_block blocksize __class__s rVr_LangSmithHttpAdapter.__init__s$ )QrZc >[(aURUS'[TU] "XU40UD6$![aB SU;a:[ R S5 URS5 [TU] "XU40UD6s$ef=f)NrzWAn intermediate HTTPAdapter does not accept the 'blocksize' kwarg. Retrying without it.)URLLIB3_SUPPORTS_BLOCKSIZEr~rinit_poolmanagerrrQrRpop)r connectionsrblock pool_kwargsrs rVr&_LangSmithHttpAdapter.init_poolmanagers % %'+K $ 7+K%W;W W k)2 ,w/%3>  s/AA;9A;)r~) rrDrrDryzUnion[Retry, int, None]rboolrrDreturnNoneF) rrrr __attrs__requests_adaptersDEFAULT_POOLSIZEDEFAULT_RETRIESDEFAULT_POOLBLOCKrrr __classcell__rs@rVrxrxsI!2 B B->>/@/P/P,>> R R R- R  R  R  R RrZrxcL\rSrSr%SrS\S'S\S'S\S'S \S 'S \S 'S rg )ListThreadsItemiz-Item returned by :meth:`Client.list_threads`.r thread_idzlist[ls_schemas.Run]runsrDrU Optional[str]min_start_timemax_start_timerYN)rrrrr__annotations__rrYrZrVrrs!7N  J!!!!rZrc \rSrSr%Sr/SQrS\S'S\S'S\S'S \S 'S \S 'GS6S S S S S SS S S S SS S S S S S S S S S S S S SS S.GS7SjjjrGS8Sjr\ S S.GS9Sjj5r GS:Sjr GS;Sjr GS:Sjr \GS:Sj5r\GS:Sj5rGSSj5r\R(GS?Sj5r\GS>Sj5r\R(GS?Sj5r\GSS/.GSGS0jjr S S \R<R>S1.GSHS2jjr!GSIGSJS3jjr"GSKS4jr#GSLS5jr$SS6.GSMS7jjr%\GSNS8j5r&GSOS9jr'S S SS S S:.GSPS;jjr(S S S S S S S<.GSQS=jjr)S S S S S S S<.GSRS>jjr*GSSS?jr+GSTS@jr,GSUSAjr-\ GSVSBj5r.GSLSCjr/GSWSDjr0S S S S S S SE.GSXSFjjr1GSYGSZSGjjr2S S S S S S SE.GS[SHjjr3S S S S S S SE.GS\SIjjr4GSYSSJ.GS]SKjjjr5SLS S S S S S SM.GS^SNjjr6SLSO.GS_SPjjr7S S S S S S S S S S S SS S S S S S S SQ.GS`SRjjr8S S S S S S S<.GSaSSjjr9S S S S S S S<.GSbSTjjr:GScGSdSUjjr;GS6GSeSVjjrS S SS S S SYSZ.GSiS[jjr?S S S S S S S S S S S S S S S S S\.GSjS]jjr@S S S S^S S S_.GSkS`jjrAS S S S S S S S S S S S S S S S Sa.GSlSbjjrBS S Sc.GSmSdjjrCS Se.GSnSfjjrDGSoSgjrEGSpShjrFGSqSijrGGS6GSrSjjjrHGS6GSsSkjjrIGS6S Sl.GStSmjjjrJGS6S Sl.GStSnjjjrKGSuSojrLGSvSpjrMS S Sq.GSwSrjjrNS S S S Ss.GSxStjjrOS S SS S Su.GSySvjjrPS S S S S Sw.GSzSxjjrQGS{SyjrRGS|SzjrS\TR"S{5S S SS|.GS}S}jj5rVS S~.GS~SjjrWS S S{.GSSjjrXGSGSSjjrY\TR"Sc5S S Sc.GSSjj5rZS \R<R>S S S S S.GSSjjr[S S S.GSSjjr\\TR"S5S S S.GSSjj5r]GS6S Sl.GSSjjjr^GS6S Sl.GSSjjjr_S S S S S S S.GSSjjr`\TR"S5S S S.GSSjj5raS S S.GSSjjrbS S S S S.GSSjjrcS S S S S.GSSjjrdS S S.GSSjjreGSSjrf\TR"S5GSGSSjj5rg\TR"S5GSGSSjj5rhGSGSSjjriGSIGSSjjrjS SS.GSSjjrkS SS.GSSjjrlS SS.GSSjjrmGSSjrnGSGSSjjroS SS.GSSjjrpS SS.GSSjjrq\TR"S5S S S SS%S.GSSjj5rrSrsSrt\TR"S5GSGSSjj5ruS S.GSSjjrvGSS^S S S SS.GSSjjjrwS S S S S S S S.GSSjjrxS S S SS.GSSjjryGSSjrzSS.GSSjjr{S S S S.GSSjjr|S S SS.GSSjjr}GSSjr~GSSjrS S.GSSjjrS S SS.GSSjjrGSS S.GSSjjjrS S SS.GSSjjrGSS S S S S S \GRGR S S S SS S S S S S S S.GSSjjjrS S S S S.GSSjjrGSSjrS S S S S.GSSjjrGSSjrGS6S S S S S.GSSjjjrS S S S.GSSjjrS S S.GSSjjrS S.GSSjjrS S S S^S.GSSjjrGSSjrS S S.GSSjjrGSSjrGSSjrSS.GSSjjrS S S S^S.GSSjjrS S S.GSSjjrGSSjrS S S S S.GSSjjrS S S S S.GSSjjrGSSjrS S S S S.GSSjjrGSSjrGSSjrGSSjrGSSjrS S S S S S.GSSjjrGSSjrGSSjrGSGSSjjrGSSjrGSSjrGSSjrGSSjrGSSjrGSSjrSS^S S\GRTGRVSS S.GSSjjrGSSjrS S S SS.GSSjjrS S S S.GSSjjrS S S S S S.GSGSjjrGSGSjrGSgGSGSjjrGSgGSGSjjrSSSGS.GSGSjjrS S^SGS.GSGSjjrSS SSSGS.GSGS jjrS GS S S S S S S GS .GSGS jjrS GS .GSGSjjrS S S S S GS.GSGSjjrS GS .GSGSjjrS S S S S GS.GSGSjjrGSGSjrGSGSjrGSGSjrGSGSjrSS^S SS GS.GSGSjjrSS^S SS GS.GSGSjjrGSGSjrGSGSjrGSGSjrGSGSjrGSGSjrGSGSjrGSGS jrGS6GSeGS!jjr\GSGSGS"jj5r\GSGSGS#jj5rGSGSGS$jjrGSGSGS%jjrGSGSGS&jjrGSGSGS'jjr\\S S S S GS(.GSGS)jj5r\S S S GS*GS+SGS,.GSGS-jj5r\S S S SGS..GSGS/jj5rS S GS0.GSGS1jjrS S S GS2.GSGS3jjrGSGS4jrGS5rg (Clientiz.Client for interacting with the LangSmith API.). __weakref__r"_api_key _workspace_id_headers_custom_headers retry_config timeout_ms_timeoutrL_get_data_type_cached_web_url _tenant_idtracing_sample_rate_filtered_post_uuids tracing_queue _anonymizer _hide_inputs _hide_outputs_hide_metadata_omit_traced_runtime_info_process_buffered_run_ops_run_ops_buffer_size_run_ops_buffer_timeout_ms_run_ops_buffer_last_flush_time_inforf _settings_manual_cleanup _pyo3_clientcompressed_traces_data_available_event_futures_run_ops_buffer_run_ops_buffer_lock otel_exporter _otel_trace_set_span_in_context_max_batch_size_bytes_use_daemon_threads_tracing_error_callback_multipart_disabled_cache_failed_traces_dir_failed_traces_max_bytes _tracing_moderrdict[str, str]rrztuple[float, float]rrrNTF)rRrrweb_urlrLauto_batch_tracing anonymizer hide_inputs hide_outputs hide_metadataomit_traced_runtime_infoprocess_buffered_run_opsrun_ops_buffer_sizerun_ops_buffer_timeout_msinfoapi_urlsotel_tracer_providerrsrar\ workspace_idmax_batch_size_bytesheaderstracing_error_callbackdisable_prompt_cachecachecU(aU(a[R"S5e[R"S5(d[R"S5(a1[R"S5(a[R"S5e[ UUS9nUUl[ U5Ul[5Ul [U5Ul [R"U5Ul U=(d 0UlUR(aB[[!UR55UlURUR"UlOr[R&"U5Ul[R("U5Ul[+UR"UR$US9 UR"UR$0Ul U=(d [-5Ul[1U[25(aXD4O U=(d SUlUR4S S - UR4S S - 4UlXPlS UlU(aUO[<R>"5nUUl Ub[1U[BRD5(aUO[BRD"S20UD6Ul#[HRJ"U[LUR@5 [NRP"[LU5 S Ul)S Ul*S Ul+/Ul,[ZR\"5Ul/S Ul0UUl1S Ul2[Rf"S5S:HUl4US;a[k5unnn n!URl"5n"U"Ro[p5n#Uc[1U"URr5(aJ[uU#S5(a9[1[wURxU#5RzUR|5(d[w[~U"5nOU "5nUR"U5 U!"US9Ul0UUlAUUlBU(a[Rf"S5n$U$b [3U$5O[n%[U%S9UlH[ZR"[[HR"U54URhS9R5 OS UlH[UR.[[S -S9n&UR@RSU&5 UR@RSU&5 [R"SS9"UR5UlTXlUU bU O[Rf"S5S:HUlVU bU O[Rf"S 5S:HUlWU bU O[Rf"S!5S:HUlXXlYXlZXl[U=(d S"Ul\[R"5Ul^U bUc [S#5eU cUb [S$5eS Ul`[Rf"S%5bFS n'S S Kan'U'(a8S'n)S(n*S n+S n,U'RUR"UR$U)U*U+U,5Ul`S UlgS UlhUUliUb!U(a[R"S*[SS9 Ub@[R"S+[SS9 US LaS UlkO3US,La [UlkO"UUlkOU(d [UlkOS Ulk[Rf"S-5Ulm[Rf"S.5n-U-(a [3U-5OS n.U.S :a[3U.S/-S/-5OS0Ulng ![aK [R"SU<S3SS9 S Ul0SUl[+UR"UR$SS9 GNf=f![a*n([RS&[U(55 S n(A(GNS n(A(ff=f![a*n([RS)[U(55 S n(A(GNS n(A(ff=f![[4a [RS1U-5 S0Ulng f=f)3uwInitialize a `Client` instance. Args: api_url: URL for the LangSmith API. Defaults to the `LANGCHAIN_ENDPOINT` environment variable or `https://api.smith.langchain.com` if not set. api_key: API key for the LangSmith API. Defaults to the `LANGCHAIN_API_KEY` environment variable. retry_config: Retry configuration for the `HTTPAdapter`. timeout_ms: Timeout for the `HTTPAdapter`. Can also be a 2-tuple of `(connect timeout, read timeout)` to set them separately. web_url: URL for the LangSmith web app. Default is auto-inferred from the `ENDPOINT`. session: The session to use for requests. If `None`, a new session will be created. auto_batch_tracing: Whether to automatically batch tracing. anonymizer: A function applied for masking serialized run inputs and outputs, before sending to the API. hide_inputs: Whether to hide run inputs when tracing with this client. If `True`, hides the entire inputs. If a function, applied to all run inputs when creating runs. hide_outputs: Whether to hide run outputs when tracing with this client. If `True`, hides the entire outputs. If a function, applied to all run outputs when creating runs. hide_metadata: Whether to hide run metadata when tracing with this client. If `True`, hides the entire metadata. If a function, applied to all run metadata when creating runs. omit_traced_runtime_info: Whether to omit runtime information from traced runs. If `True`, runtime information (SDK version, platform, Python version, etc.) will not be stored in the `extra.runtime` field of runs. Defaults to `False`. process_buffered_run_ops: A function applied to buffered run operations that allows for modification of the raw run dicts before they are converted to multipart and compressed. Useful specifically for high throughput tracing where you need to apply a rate-limited API or other costly process to the runs before they are sent to the API. Note that the buffer will only flush automatically when `run_ops_buffer_size` is reached or a new run is added to the buffer after `run_ops_buffer_timeout_ms` has elapsed - it will not flush outside of these conditions unless you manually call `client.flush()`, so be sure to do this before your code exits. run_ops_buffer_size: Maximum number of run operations to collect in the buffer before applying `process_buffered_run_ops` and sending to the API. Required when `process_buffered_run_ops` is provided. run_ops_buffer_timeout_ms: Maximum time in milliseconds to wait before flushing the run ops buffer when new runs are added. Defaults to `5000`. Only used when `process_buffered_run_ops` is provided. info: The information about the LangSmith API. If not provided, it will be fetched from the API. api_urls: A dictionary of write API URLs and their corresponding API keys. Useful for multi-tenant setups. Data is only read from the first URL in the dictionary. However, ONLY Runs are written (`POST` and `PATCH`) to all URLs in the dictionary. Feedback, sessions, datasets, examples, annotation queues and evaluation results are only written to the first. otel_tracer_provider: Optional tracer provider for OpenTelemetry integration. If not provided, a LangSmith-specific tracer provider will be used. tracing_mode: Where to send traces. One of: - ``"langsmith"`` (default) — LangSmith only. - ``"otel"`` — OpenTelemetry export only. - ``"hybrid"`` — both OTel and LangSmith. Falls back to the ``LANGSMITH_TRACING_MODE`` env var, then to the legacy ``OTEL_ENABLED`` / ``OTEL_ONLY`` env vars, then to ``"langsmith"``. otel_enabled: *Deprecated.* Use ``tracing_mode`` instead. When ``True``, interpreted as ``tracing_mode="hybrid"`` (or ``"otel"`` if the ``OTEL_ONLY`` env var is set). Will be removed in the next minor version. tracing_sampling_rate: The sampling rate for tracing. If provided, overrides the `LANGCHAIN_TRACING_SAMPLING_RATE` environment variable. Should be a float between `0` and `1`, where `1` means trace everything and `0` means trace nothing. workspace_id: The workspace ID. Required for org-scoped API keys. max_batch_size_bytes: The maximum size of a batch of runs in bytes. If not provided, the default is set by the server. headers: Additional HTTP headers to include in all requests. These headers will be merged with the default headers (User-Agent, Accept, x-api-key, etc.). Custom headers will not override the default required headers. tracing_error_callback (Optional[Callable[[Exception], None]]): Optional callback function to handle errors. Called when exceptions occur during tracing operations. disable_prompt_cache: Disable prompt caching for this client. By default, prompt caching is enabled globally using a singleton cache. Set this to `True` to disable caching for this specific client instance. To configure the global cache, use `configure_global_prompt_cache()`. !!! example ```python from langsmith import Client, configure_global_prompt_cache # Use default global cache client = Client() # Disable caching for this client client_no_cache = Client(disable_prompt_cache=True) # Configure global cache settings configure_global_prompt_cache(max_size=200, ttl_seconds=7200) ``` cache: **[Deprecated]** Control prompt caching behavior. This parameter is deprecated. Use `configure_global_prompt_cache()` to configure caching, or `disable_prompt_cache=True` to disable it. - `True`: Enable caching with the global singleton (default behavior) - `False`: Disable caching (equivalent to `disable_prompt_cache=True`) - `Cache(...)`/`PromptCache(...)`: Use a custom cache instance !!! example ```python from langsmith import Client, Cache, configure_global_prompt_cache # Old API (deprecated but still supported) client = Client(cache=True) # Use global cache client = Client(cache=False) # Disable cache # Use custom cache instance my_cache = Cache(max_size=100, ttl_seconds=3600) client = Client(cache=my_cache) # New API (recommended) client = Client() # Use global cache (default) # Configure global cache for all clients configure_global_prompt_cache(max_size=200, ttl_seconds=7200) # Or disable for a specific client client = Client(disable_prompt_cache=True) ``` Raises: LangSmithUserError: If the API key is not provided when using the hosted service. LangSmithUserError: If both `api_url` and `api_urls` are provided. z-You cannot provide both api_url and api_urls.LANGSMITH_ENDPOINTLANGCHAIN_ENDPOINTLANGSMITH_RUNS_ENDPOINTSz]You cannot provide both LANGSMITH_ENDPOINT / LANGCHAIN_ENDPOINT and LANGSMITH_RUNS_ENDPOINTS.r`rN)i'i`rrKNF USE_DAEMONtrue)rGrH_tracer)tracer_providerz tracing_mode=zu requires OpenTelemetry packages. Install with `pip install langsmith[otel]`. Falling back to LangSmith-only tracing.rrgrFTRACING_QUEUE_MAX_SIZEr)targetargsdaemon)ryrrhttp://https://r] HIDE_INPUTS HIDE_OUTPUTS HIDE_METADATAizOrun_ops_buffer_size must be provided when process_buffered_run_ops is specifiedzOprocess_buffered_run_ops must be provided when run_ops_buffer_size is specifiedUSE_PYO3_CLIENTzaFailed to import `langsmith_pyo3` when PyO3 client was requested, falling back to Python impl: %s@Bdz|Failed to instantiate `langsmith_pyo3.BlockingTracingClient` when PyO3 client was requested, falling back to Python impl: %szBoth 'cache' and 'disable_prompt_cache' were provided. The 'cache' parameter is deprecated and will be removed in a future version. Using 'cache' parameter value.zThe 'cache' parameter is deprecated and will be removed in a future version. Use 'configure_global_prompt_cache()' to configure the global cache, or 'disable_prompt_cache=True' to disable caching for this client.TFAILED_TRACES_DIRFAILED_TRACES_MAX_MBii@zJInvalid value for LANGSMITH_FAILED_TRACES_MAX_MB: %r, using default 100 MBrY)prirlosgetenvrzrr_rrrrjrfget_workspace_idrrnextiterr"rR get_api_url get_api_keyrSrIrrrDrrrrrequestsSessionrL ls_schemas LangSmithInforweakreffinalizerMatexitregisterrrrr threadingLockrrrrrjrrget_tracer_provider get_tracerrProxyTracerProviderhasattrr ProxyTracerr NoOpTracerrset_tracer_providerrrrrorpr-rrThread_tracing_control_thread_funcrefstartrxr*r)mount functoolsr _get_data_typerrrrrrrrrrNrrrlangsmith_pyo3rQrRreprBlockingTracingClientrrrrDeprecationWarningrrArr OverflowError)/rr"rRrrrrLrrrrrrrrrrrrrsrar\rrrrrr resolved_modesession_rr~rrexisting_providertracerqueue_maxsize_str queue_maxsizeadapterrrqueue_capacity batch_sizebatch_timeout_millisworker_threads _max_mb_str_max_mbs/ rVrClient.__init__Bsd x--?  II* + +ryy9M/N/Nii233--0  .lV *7#=>S#T 47E!B QS$-NN$4!59%9"). #+#7#7 #E#O . .*  !N ', %/$B$B$D!*55h?'/"#4j6T6TUU#FI66& * 6 6 &&g&11 04NDU/V,/G/I,"667KL%1BV%W"#- ,?)  ( 4 45M N %0%&,  ;H%;D    3kk$')//   eg!%D ())&7:   9g. :w/%.%8%8%D   & "&& %%m4> ' %%n5? ( %%o6&@  *B&)A&$7!*C*Kt'/3yy{, $ /4G4Oa  $ +0C0Oa  !    1 2 >!N %!* '+$!"(6(L(L  &",& )D%EI$'=$  !5 MM1#     MMR#  ~59 $4 $ %0DKDK191E1E 2 **+AB  >*5c+&1G.5kGdNT)*?P  )E  #M#45>> !  &*"%0"+LL$,,[ n 6G .!NNZQnM* > NN'  ->D )  >sVC ]"$^:8/_1/2`("A^76^7: _._))_.1 `%;`  `%(-aacUR(dgU"5n[U[5(aURS5nUR URUSUUR S9 g![ a gf=f)uDump a failed trace payload to disk if a fallback directory is configured. *body_fn* is called lazily inside a try/except so that any serialization errors are silently swallowed — we must never raise from a failure path. Nzutf-8zruns/multipart)endpointr max_bytes)rrrencode_write_trace_to_fallback_dirrr)rbody_fnrbodys rV_dump_failed_traceClient._dump_failed_trace"sy&&  9D$$${{7+  - -'')77 .    sAA)) A65A6)r0c>SUU[R"U5R5S.nS[R"5SS[R "5R SSS3n[U5U- nU[;a/URRS S S 9 [RU5 UbPUS :aJ[U5n[S URS 555n X:a[RSUU U5 gUR!S5n U R#[$R&"U55 U R)S5 U R+U5 [RSU5 g![,a!n [R/SUU 5 Sn A gSn A ff=f)aPersist a failed trace payload to a local fallback directory. Saves a self-contained JSON file with the endpoint, the HTTP headers required for replay, and the base64-encoded request body. Can be replayed later with a simple POST: POST / Content-Type: [Content-Encoding: ] If *max_bytes* is set, new traces are dropped when the directory is already at or over the budget. rK)rr/r body_base64trace_z.6frNz.jsonT)parentsexist_okrc3# UH6nUR5(dMUR5Rv M8 g7fr)is_filestatst_size)rfs rVr6Client._write_trace_to_fallback_dir..bs.:yy{%AFFH$$:s A!Az trace_*.jsonzCould not write trace to fallback dir %s as it's already over size limit (%d bytes >= %d bytes). Increase LANGSMITH_FAILED_TRACES_MAX_MB if possible.z.tmpizALangSmith trace upload failed; data saved to %s for later replay.zELangSmith tracing error: could not write trace to fallback dir %s: %s)base64 b64encodedecoderNruuid4hexr rparentmkdiraddsumglobrQrR with_suffix write_textjsondumpschmodrenamererror) directoryr4r/rr0envelopefilenamefilepathdir_pathr8 temp_path write_excs rVr2#Client._write_trace_to_fallback_dir<s}0 !++D188:   DIIK,Adjjl.>.>r.B-C5I ?X-#  66%%dT%B&**95$Q ?%]]>: %NNO"!  ,,V4I  H!5 6 OOE "   X & NNS   LLW   s 6B E1A.E11 F;FFc&URnSUS3$)z}Return an HTML representation of the instance with a link to the URL. Returns: The HTML representation of the instance. z LangSmith Client) _host_url)rlinks rV _repr_html_Client._repr_html_s ~~4& VWWrZcUR(aURU5 gg![a [RSSS9 gf=f)zInvoke the background tracing error callback if configured. Args: error: The exception that occurred during background tracing. z!Error in tracing_error_callback: Texc_infoN)rrrQrS)rrSs rV_invoke_tracing_error_callback%Client._invoke_tracing_error_callbacksM  ' ' ,,U3 (  8! s&AAc"SURS3$)zReturn a string representation of the instance with a link to the URL. Returns: The string representation of the instance. zClient (API URL: )r"rs rV__repr__Client.__repr__s #4<<.22rZc,[UR5$r)rvr"ris rV_host Client._hosts$,,''rZcX[R"URUR5$)zThe web host url.)ri get_host_urlrr"ris rVr]Client._host_urls$$T]]DLLAArZcS[R3SS.nURUR5 UR(aURU[ 'UR (aUR US'U$)Nz langsmith-py/application/json)z User-AgentAccept X-Tenant-Id)rFrrrrR X_API_KEYr)rrs rV_compute_headersClient._compute_headerssd))*?*?)@A(  t++, <,>,J#113DJ::  400!#56 1H  / / 9#11DHMMODDJzz  4 NN*4<<.47)D $113DJJzz  4sAC D.AD))D.cURcQURSS5n[R"U5 [R "S0UR 5D6UlUR$)z\Get the settings for the current tenant. Returns: The settings for the current tenant. rz /settingsrY)rrrirrLangSmithSettingsrO)rrs rV _get_settingsClient._get_settingssS >> !00 DH  / / 9'99LHMMOLDN~~rZcUb URcg[[RUR5nURnU(dgUR =(d UR S5nUcgX:a SUSUS3$g)Nsize_limit_byteszThe content length of z) bytes exceeds the maximum size limit of z bytes.)rrrrbatch_ingest_configrget)rcontent_lengthrbic size_limits rV_content_above_sizeClient._content_above_sizes  !TZZ%7J,,djj9&&//N377;M3N    &((89))3 G= rZrK)request_kwargsstop_after_attemptretry_on to_ignorehandle_response_contextc U=(d 0nSUR0UEU ES0UREURS05EU RS05E0EnUS:wa-SU;a'SU;a!USRS5(dSUSS'[R"5[R "[ UR5S9/n /U=(d S Q[R[R[R4Q7n /U=(d S Q7n S n [U5Hun[R"[U 5 URR"U[!UR"U54S S 0UD6n S S S 5 [R$"U 5 U s $ [RL"S&USUS;35e!,(df  NG=f![&R(R*a`n[,R/S U5 US-U:XaeSU-[0R0"5S--n[2R4"U5 S nAGM S nAf[&R6Ga]nU Gb,Ub"US-U:aU"XS-5nU(aS nAGMaU R8S:Xa7[R"SU R8SUSUS[;U5U3 5eU R8S:Xa[R"SUUSU35eU R8S:Xa'[R<"SUS[;U5U35eU R8S:Xa'[R>"SUS[;U5U35eU R8S:Xa'[R@"SUS[;U5U35eU R8S:Xa'[RB"S US[;U5U35eU R8S!:Xa~U RE5nURS"S#5nUS$:Xa[RF"S%5eO![H[J4a Of=f[RL"S&USUS[;U535e[RL"S&USUS[;U535e[RF"S&USUS[;U535eS nAf[&RNGa3nUR"S':waS(OS)n[QUR(a.[ URRRRS*55OS#5nURUU5nU(aUnO![Ha S nOf=fUR(a.URRRRS+5=(d S#OS#nUS S,US-S nnS.[WS/[YU5S0- 5-nUUU3n[R"S1USUSUS[;U5S2US3UU3 5UeS nAf[Zan[]UR^5n[YU5S:aUSOS#nURaS4S55nU(aZS6Rc[ US/5/U/-[YU5S:aUSS O/Vs/sHn[ U5PM Os snfsn-5nOUn[RL"S&USUSUU35UeS nAff=f!U a.nU b[,R/S U5 U sS nAs $S nAGMS nAf[R<a US-U:XaeU b_[eU RRRS7S855n O8![Za+n[,RgS9[;U55 S:n S nAOS nAff=fW SU--[0R0"5-n [2R4"U 5 GMU aC US-U:XaeSU-[0R0"5S--n[2R4"U5 GMf=f)  }PO **66LL!:A>Qw"44!"C6==?S+@!AJJJz*))?+*6"Qw);;2A(RSG2T#2$,#//36"*"<"<"01E1E0FFZ[aZb"#H:.3377)#+*!.# &11S8"*"B"B"J6(#+*AhZ!9#&11S8"*"B"B":8*BtAwi#+*!.#&11S8"*"="="#O(33 (700')&+H,<,<,@,@PT,U&V $)< G') )*AsF2V]]_D  ;'' 700Vv}}'<=  :&  sG95F/.G/ F= 9G[AH92[9["R)3[:E!R)>PR)P.+R)-P..A;R))[W/A$T;:W/; U W/ U  B%W// [5Z  4Z>>[[_8 [5&_85$_8%]?_8 ]5 !]0+_80]557_80A_87_8paramsc#D# U(aUR5O0nURSS5nURSS5US'XCS'URSUUS9nUR5nU(dgUShvN [ U5US:agU[ U5- nMZN'7f)zGet a paginated list of items. Args: path: The path of the request URL. params: The query parameters. Yields: The items in the paginated list. offsetrlimitrrrN)copyrrrOr)rrrparams_rrrcs rV_get_paginated_listClient._get_paginated_lists$*&++-rXq)";;w4 &H 001H MMOE  5zGG,, c%j F sA4B 6B7(B POSTr)r4request_methoddata_keyc#d# U(aUR5O0nURUUS[U50S9nUR5nU(dgUR U5(dgXtShvN UR S5nU(dgUR S5(dgUSUS'MN>7f)a Get a cursor paginated list of items. Args: path: The path of the request URL. body: The query body. request_method: The HTTP request method. data_key: The key in the response body that contains the items. Yields: The items in the paginated list. rrNcursorsrcursor)rr _dumps_jsonrOr) rrr4rrrr response_bodyrs rV_get_cursor_paginated_list!Client._get_cursor_paginated_list!s&"&$))+200K0 1H%MMOM  $$X..$. . .#'' 2G;;v&& 'GH ' /sA-B0/B.0?B0) description data_typec [R"5nURUSS9 URS5 UR SU4UUUUUS9$)aUpload a dataframe as individual examples to the LangSmith API. Args: df: The dataframe to upload. name: The name of the dataset. input_keys: The input keys. output_keys: The output keys. description: The description of the dataset. data_type: The data type of the dataset. Returns: The uploaded dataset. Raises: ValueError: If the `csv_file` is not a `str` or `tuple`. Example: ```python from langsmith import Client import os import pandas as pd client = Client() df = pd.read_parquet("path/to/your/myfile.parquet") input_keys = ["column1", "column2"] # replace with your input column names output_keys = ["output1", "output2"] # replace with your output column names dataset = client.upload_dataframe( df=df, input_keys=input_keys, output_keys=output_keys, name="My Parquet Dataset", description="Dataset created from a parquet file", data_type="kv", # The default ) ``` F)indexrzdata.csv) input_keys output_keysrnamer)ioBytesIOto_csvseek upload_csv)rdfrrrrrcsv_files rVupload_dataframeClient.upload_dataframeJsY`::< (% ( a  "!##   rZ)rrrczUUS.nU(aXGS'U(aXWS'U(a[R"U5US'[[R"55US'[ U[5(aB[ US5nS[U5RUS40n URS S UU S 9n S S S 5 OQ[ U[5(a1Un [U5S :Xa USUSS4n URS S USU 0S 9n O [S5e[R"W 5 U R5n SU ;aHSU S;a?[ U[5(aUOUSn U RS5Sn [SU S35e[ R""S0U DUR$UR'5S.D6$!,(df  N=f)aBUpload a CSV file to the LangSmith API. Args: csv_file: The CSV file to upload. If a string, it should be the path. If a tuple, it should be a tuple containing the filename and a `BytesIO` object. input_keys: The input keys. output_keys: The output keys. name: The name of the dataset. description: The description of the dataset. data_type: The data type of the dataset. Returns: The uploaded dataset. Raises: ValueError: If the `csv_file` is not a string or tuple. Example: ```python from langsmith import Client import os client = Client() csv_file = "path/to/your/myfile.csv" input_keys = ["column1", "column2"] # replace with your input column names output_keys = ["output1", "output2"] # replace with your output column names dataset = client.upload_csv( csv_file=csv_file, input_keys=input_keys, output_keys=output_keys, name="My CSV Dataset", description="Dataset created from a CSV file", data_type="kv", # The default ) ``` )rrrrrrrbfileztext/csvrz/datasets/upload)rrNrrrKz"csv_file must be a string or tupledetailzalready existsrDataset z already existsr]rrY)riget_enum_valuerrrFropenr rrrrrrrOrrDatasetr]_get_optional_tenant_id)rrrrrrrrrAfile_r file_tupleresult file_names rVrClient.upload_csvsj%&  L "-   ( 7 7 BD &T h $ $h%$x."5"5q*!EF44& 5&%% ( (!J8}!&qk8A; C 00"z* 1HAB B++H5 v "2fX6F"F$.x$=$=8A;I!,R0Ix {/BC C!!  nn335  7&%s >-F,, F:c[US5(a+[[US55(aUR5nO[ [ U5nSU;a[ R"5US'O4[US[5(a[ R"US5US'SU;a@USb:U(a[R"US5US'URUS5US'SU;a@USb:U(a[R"US5US'URUS5US'SU;aUSbURUS5US'SU;ac[US[ 5(aKUSnSU;a@USb:U(a[R"US5US'UR!US5US'U(dPUR#S 5(d:[$R$R'[$R(R*5US 'S U;aTUR#S 5S ;aUR-S S5 U$UR#S 5(aUS R-S S5 U$)aTransform the given run object into a dictionary representation. Args: run: The run object to transform. update: Whether the payload is for an "update" event. copy: Whether to deepcopy run inputs/outputs. Returns: The transformed run object as a dictionary. model_dumprinputsNoutputseventsextrar start_time serializedrun_type)r1rgraph)rcallablegetattrrrrrrFrrrri deepish_copy_hide_run_inputs_hide_run_outputs_filter_new_token_events_hide_run_metadatardatetimerTtimezoneutcr)rrunrr run_creaters rV_run_transformClient._run_transforms; 3 % %(73 3M*N*N"~~/JdCJ z !#zz|Jt   4(# . .#yyD)9:Jt  z !j&:&F'/'<'S(T 9%$($:$::i;P$QJy ! z !j&:&F#'#@#@HAU#VJx j Z 70CT%J%Jw'EU"uZ'8'D(0(=(=eJ>O(PE*%$($;$;E*._val<sB 2xi( 2f$"3,,s11 2s$; ;;trace_idr)r+rrDrrErrr) rrrCrrappendremover=rJ)rrr@rFsampledr+rHs rV_filter_for_samplingClient._filter_for_sampling3s  # # +: 4: 2 G#Dj$9:#<#<<NN3'#t_0--44X>  NGZ6I$sD/888T?h.**,,s+1155h?NN3' NrZcUR$)zPer-client tracing mode.)rris rVrsClient.tracing_modearrZcURceURRU5 g![a& [SURRS35 gf=f)z3Put an item on the tracing queue, dropping if full.Nztracing queue full (maxsize=rg)r put_nowaitr rWr)ritems rV_put_tracing_queueClient._put_tracing_queuefs]!!---     ) )$ /  .t/A/A/I/I.J!L  s--AA) project_name revision_iddangerously_allow_filesystemrRr"c U RSS5n U RSS5n U RSS5n U RSS5n U=(d% U RS[R"55n0U EUUUUS.EnURU/5(dgUb X^SS S 'UR US S 9nUR U/5 UR S 5b]US R5HFn[U[5(dM[US[5(dM4U(aM=[S5e UR(awU R S5(daUR URRSUUUU U U U S.45 UR!5(aUR#5 SSS5 gUR%UUUU U U U S9 g!,(df  g=f)aPersist a run to the LangSmith API. Args: name (str): The name of the run. inputs (Dict[str, Any]): The input values for the run. run_type (str): The type of the run, such as tool, chain, llm, retriever, embedding, prompt, or parser. project_name (Optional[str]): The project name of the run. revision_id (Optional[Union[UUID, str]]): The revision ID of the run. api_key (Optional[str]): The API key to use for this specific run. api_url (Optional[str]): The API URL to use for this specific run. service_key (Optional[str]): The service JWT key for service-to-service auth. tenant_id (Optional[str]): The tenant ID for multi-tenant requests. authorization (Optional[str]): The Authorization header value. cookie (Optional[str]): The Cookie header value. **kwargs (Any): Additional keyword arguments. Returns: None Raises: LangSmithUserError: If the API key is not provided when using the hosted service. Example: ```python from langsmith import Client import datetime from uuid import uuid4 client = Client() run_id = uuid4() client.create_run( id=run_id, project_name=project_name, name="test_run", run_type="llm", inputs={"prompt": "hello world"}, outputs={"generation": "hi there"}, start_time=datetime.datetime.now(datetime.timezone.utc), end_time=datetime.datetime.now(datetime.timezone.utc), hide_inputs=True, hide_outputs=True, ) ``` service_keyN tenant_id authorizationcookie session_name)r]rrrrrrVFr attachmentsrKUMust set dangerously_allow_filesystem=True to allow passing in Paths for attachments.is_run_ops_buffer_flushpostr"rRrYrZr[r\rRr"rYrZr[r\)rriget_tracer_projectrLr-r:rrrrr rrrrrI_should_flush_run_ops_buffer_flush_run_ops_buffer _create_run)rrrrrUrVrWrRr"r rYrZr[r\r, attachments rV create_runClient.create_runpsv#)**]D"A & ; = $*JJ$E #ZZ$7# vzz   ' ' )(   (  ((*66   "=Hw  +M :((%(@    *. >>- ( 4(7>>@ z511":a=$7788$o A  ) )&**=V2W2W**$$++"'.'.+6)2-:&,   4466..0%+*(   '#+  )+*s +A G G$rdcURS5Gb"URS5GbURbURRU5 gURGbUcUcUcUcUcUcURc [ S5e[ SU5n[U5un n [RSSU R5 URR [U UR[5n U (a9UR=RS- slURR5 SSS5 [![#U R%555 gUR&b[ SU5n[RSSUR(UR*5 UR,bKUR/[1USUUUUUUUUR3UR4R755S 9 5 gUR/[1USUUUUUUUS 95 gUR9UUUUUUUS 9 gUR9UUUUUUUS 9 g!,(df  GN=f) NrH dotted_order@Run compression is enabled but threading event is not configuredrbr5Adding compressed multipart to queue with context: %srK/Adding to tracing queue: trace_id=%s, run_id=%srRr"rYrZr[r\ otel_contextrd)rrrjrrrr;r=rQlogcontextlockr9r+ trace_countr _close_filesrrrrHrrrSr%rrget_current_span_create_run_non_batch) rr,rRr"rYrZr[r\ serialized_opmultipart_form opened_filesenqueueds rVrhClient._create_runst NN: & 2~.:  ,!!,,Z8&&2OO'%!)N--5$Z!36: F L!"  K"** ++00C&..! H  ..::a?:226681T,"5"5"789##/ 26: F  E!**!$$  %%1++(&~6)$+$+(3&/*7#))-)B)B $ 0 0 A A C*  ++(&~6)$+$+(3&/*7#)  **## +'"/!+  & &'#+ ' {10s AI  Ic ./n[X2XFU/5n U (ajU=(d URn [URUUUUUURS9n UR SU S3[ U5U S.[R4S9 OmURR5HOup[URUSSSSSS9n UR SU S3[ U5U S.[R4S9 MQ U(aO[U5S:a'[R"US9nURU5 UeURUS5 USeg![an URU 5 Sn A NwSn A ff=f![an URU 5 Sn A MSn A ff=f) NrRrYrZr[r\fallback_api_keyr/runsrr)rrrK)rr)rr"_apply_auth_overridesrrRrrrirrrIrfrcrLangSmithExceptionGrouprd)rr,rRr"rYrZr[r\errors use_overridetarget_api_urlrr write_api_url write_api_keyexception_groups rVryClient._create_run_non_batch^ sGk&QR $4 N+ '#+!%G !))%&e, +J 7#*$ (>>@*150D0D0J0J0L, %3 -$("&&*#)-G--(//$/ $;'.($,#B#B"D.1M, 6{Q"*"B"Bf"U33OD%%33F1I>Qi 5 ! a   !0!%MM!$$%s10E"A E. E+E&&E+. F8FFcURSLa0$UR(a0[R"[ U55nURU5$URSLaU$URU5$NTF)rrr$loadsr)rr json_inputss rVr$Client._hide_run_inputs sf    $I   !-- F(;L##L1 1    &N!!'**rZchURSLa0$URSLaU$URU5$r)r)rrs rVr'Client._hide_run_metadata s9   $ &I   % 'O""8,,rZc U(dU$UVVVs/sHGnURS5S:Xa-UR5VVs0sHup#US:wdM X#_M snnOUPMI snnn$s snnfs snnnf)zFilter content from new_token events. This prevents streaming LLM output from being uploaded via events. Args: events: The events to filter. Returns: The filtered events with kwargs removed from new_token events. r new_tokenr )rrc)reventr8r9s rVr&Client._filter_new_token_events svM    yy K/$kkm =mdaqH}TQTm =    = s.A) A#A# A)#A)cUR(dgURb$[UR5UR:agURb5[R"5UR - nXRS- :agg)zDCheck if the run ops buffer should be flushed based on size or time.FTr)rrrrrNr)rtime_since_last_flushs rVrf#Client._should_flush_run_ops_buffer st##  % % 1D(()T-F-FF  * * 6$(IIK$2V2V$V !$)H)H4)OPrZcrUR(dg[UR5nURR5 [R"5UlSSKJnJn URX0U5nURbURRU5 gg![a U"X5 gf=f)z8Process and flush run ops buffer in a background thread.Nr)LANGSMITH_CLIENT_THREAD_POOL_process_buffered_run_ops_batch) rrclearrNr&langsmith._internal._background_threadrrsubmitrrJ RuntimeError)rbatch_to_processrrfutures rVrgClient._flush_run_ops_buffer s##   4 45 ""$/3yy{, D188/7GF}}( !!&)) D +D C Ds%:B!!B65B6rcc //S.nUGH4n [U [5(Ga[R"U R5n U R (a#[R "U R 5U S'U R(a#[R "U R5U S'U R(a#[R "U R5U S'U R(a#[R "U R5U S'U R(a#[R "U R5U S'U R(a#[R "U R5U S'U R(a[RS5 XRR!S U R"S U R$3[R&"U 545 GM[U [(5(a[RS 5 GM[RS [+U 55 GM7 UR,n UR.=(d1 U R0=(d 0R3S 5=(d [4n [6R8"[:5n [6R8"[:5nSn[=[:[>SSS/5GHn[6R@"UU5nU(dM&US:a|U[CUSS5-U :adURE[R&"U 5SUSSRGUU53UUUUUUS9 SnU RI5 URI5 URK5unnU[CU5- nU UR![R "U55 UUR!U5 U(aMGM U(aTSRGSURM555nURE[R&"U 5SU-UUUUUUS9 gg)N)rbr@rrrrrSrzBAttachments are not supported when use_multipart_endpoint is Falseztrace=z,id=z;Feedback operations are not supported in non-multipart modez&Unknown item type in tracing queue: %srrrbr@rKrr; )rr"rRrYrZr[r\c3R# UHupUSSRU53v M g7f)rrN)rm)rr8r9s rVr/Client._batch_ingest_run_ops..[ s'WCV411#R ! ~ 6CVs%')'rr7r$r_nonerFragmentrrrrSrr_rQrR operationrIrHrrPr6rrrrrr, collections defaultdictrrrdequer_post_batch_ingest_runsrmrpopleftrc)ropsr"rRrYrZr[r\ids_and_partial_bodyop curr_dictrr body_chunks context_ids body_sizerD body_dequecurr_id curr_bodyrts rV_batch_ingest_run_opsClient._batch_ingest_run_ops sy  B"455#MM"((3 99*1*:*:299*EIh'::+2+;+;BJJ+GIi(99*1*:*:299*EIh'88)0)9)9"(()CIg&88)0)9)9"(()CIg&==.5.>.>r}}.MIl+>>NN#%\\299bkk]$ruug6 i8PQB ;<<Q EtBxP9>yy  & & !((.B334FG !   ;F:Q:QRV:W :E:Q:QRV:W  W_568IJC$**+?+DEJ*M!C 1 a(8$99. sQLS+FC88Lc3:# UHn[SU5v M g7fr@Nrrs rVrr sR\c+GS99\r)rLrr-rrirlrrr:rr7r8rr0r)rcreaterr+ create_dicts update_dictsserialized_opss rVbatch_ingest_runsClient.batch_ingest_runsg s~f **6+>Y?**6+>Yd*Kf HNO++Ce+< OIO IO#  Du  =   C77:&&cggn.E.E11P  C77:&&cggn.E.E11P   ) )#D$B$B<$PQ #D$B$B<$PQ    !<= ' ( /OOQLQR\R    "">2KP s G3=G8c[X4XWU/5n U (a>U=(d URn U [0UREUUUUUURS94/n OIUR R 5V V s/sH"upU [0UREU SSSSSS94PM$ n n n U H/upURSU S3UU S.[R4SUS9 M1 gs sn n f![an[R"[U5U5nSRU5R5n[ R#SU35 O1![a$ [ R#S[%U535 Of=fUR'U5 SnAMSnAff=f) Nrrz /runs/batchrr\)rrrrrzFailed to batch ingest runs: )rr"rrrRrfrcrrirr tracebackformat_exception_onlyrrmrerQrRrrd)rr4rr"rRrYrZr[r\rr endpointstarget_api_keyrrexc_desc_linesexc_descs rVrClient._post_batch_ingest_runs sGk&QR $4 N#))4==) '$/"+&3%)-  I87;6J6J6P6P6R7S2N#))4==) .$("&&*#)- 7S  (1 #N 7))%&k2 $#*$ (>>@'(%* (1!: 7N%.%D%DT!Wa%PN!ww~6==?HNN%B8*#MN NNN%B47)#LMN33A66 7s=1)C%)C F $AD<;F<+E*'F)E**FF c /n0n UHn [U [5(a2[U 5un n URU 5 U R U 5 MJ[U [ 5(aUR[ U 55 M{[RS[U 55 M [U5n U (a9URU UUUUUUS9 [![#U R%555 gg![RaU SUlUV s/sHn [U [5(dMU PM Os sn fnn U(aURUUUUUUUS9 Nf=f![![#U R%555 f=f)Nz+Unknown operation type in tracing queue: %srcT)rr7r=rIrr6r<rQrSrr5_send_multipart_reqrirrrrwrr)rrr"rRrYrZr[r\partsopened_files_dictrpartr| acc_multipartrun_opss rV_multipart_ingest_opsClient._multipart_ingest_ops; sr13:<B"455LBO  T"!((6B ;<< PQST JDQSHU9?  ?((!## +'"/!)4T"3":":"<=>9 22 ,0((+V"z">T/U2VV.. ' '$/"+&3%/ "T"3":":"<=>s68C11ED4-D43$EEEE$F)rWc U(dU(dgURU=(d [5nURU=(d [SS9nU(dU(dgUVs/sHo@RU5PM nnUVs/sHo@RUSS9PM nnU(aLUHEnURS5(aURS5(aM1[R "S5e AU(aLUHEnURS5(aURS5(aM1[R "S5e AU(anU(agUVs0sHoDS U_M nn/nUHGnUS U;a*UR 5HupU cM XUS U 'M M6URU5 MI AUnU(dU(dgURU5 URU5 [[[R"S U5S U5555n U Hn [U [5(dMU R(dM-U RR!5HFn [U ["5(dM[U S [$5(dM4U(aM=['S 5e M UR)U 5 gs snfs snfs snf)aj Batch ingest/upsert multiple runs in the Langsmith system. Args: create (Optional[Sequence[Union[ls_schemas.Run, RunLikeDict]]]): A sequence of `Run` objects or equivalent dictionaries representing runs to be created / posted. update (Optional[Sequence[Union[ls_schemas.Run, RunLikeDict]]]): A sequence of `Run` objects or equivalent dictionaries representing runs that have already been created and should be updated / patched. Raises: LangsmithAPIError: If there is an error in the API request. !!! note The run objects MUST contain the `dotted_order` and `trace_id` fields to be accepted by the API. Example: ```python from langsmith import Client import datetime from uuid import uuid4 client = Client() _session = "__test_batch_ingest_runs" trace_id = uuid4() trace_id_2 = uuid4() run_id_2 = uuid4() current_time = datetime.datetime.now(datetime.timezone.utc).strftime( "%Y%m%dT%H%M%S%fZ" ) later_time = ( datetime.datetime.now(datetime.timezone.utc) + timedelta(seconds=1) ).strftime("%Y%m%dT%H%M%S%fZ") runs_to_create = [ { "id": str(trace_id), "session_name": _session, "name": "run 1", "run_type": "chain", "dotted_order": f"{current_time}{str(trace_id)}", "trace_id": str(trace_id), "inputs": {"input1": 1, "input2": 2}, "outputs": {"output1": 3, "output2": 4}, }, { "id": str(trace_id_2), "session_name": _session, "name": "run 3", "run_type": "chain", "dotted_order": f"{current_time}{str(trace_id_2)}", "trace_id": str(trace_id_2), "inputs": {"input1": 1, "input2": 2}, "error": "error", }, { "id": str(run_id_2), "session_name": _session, "name": "run 2", "run_type": "chain", "dotted_order": f"{current_time}{str(trace_id)}." f"{later_time}{str(run_id_2)}", "trace_id": str(trace_id), "parent_run_id": str(trace_id), "inputs": {"input1": 5, "input2": 6}, }, ] runs_to_update = [ { "id": str(run_id_2), "dotted_order": f"{current_time}{str(trace_id)}." f"{later_time}{str(run_id_2)}", "trace_id": str(trace_id), "parent_run_id": str(trace_id), "outputs": {"output1": 4, "output2": 5}, }, ] client.multipart_ingest(create=runs_to_create, update=runs_to_update) ``` NTr?)rrHrmzNMultipart ingest requires trace_id and dotted_order to be set in create dicts.zNMultipart ingest requires trace_id and dotted_order to be set in update dicts.rc3:# UHn[SU5v M g7frrrs rVr*Client.multipart_ingest.. sM '44 rc3:# UHn[SU5v M g7frrrs rVrr sN#'55rrKr`)rLrr-rrirlrcrIr:r8rrr0rr7r_rrr rr)rrrrWr+rr create_by_idstandalone_updatesr8r9rrris rVmultipart_ingestClient.multipart_ingestu so|& **6+>Y?**6+>Yd*Kf 2ADP,DsK>K Kr\)attemptsr"rRrYrZr[r\c4^^URmURn [X4XWU/5n U (a>U=(d URn U [ 0UR EUUUUUUR S94/n OIURR5V V s/sH"upU [ 0UR EU SSSSSS94PM$ n n n U Hup[SUS-5Hon[R"T[S9nURS::aUR5mOUmURSU S3T0UESUR 0E["S.SU S 9 M M gs sn n f![$R&a  M[$R([$R*[$R,4a/nX:Xa[.R1S U35 UnSnAOSnAMSnAf[2an[4R6"[9U5U5nS R;U5R=5n[.R1S U35 O1![2a$ [.R1S [?U535 Of=fUnSnAOSnAff=fURAUU4S jSS [305 URCU5 GM)NrrKboundary-1r/runs/multipartrrrrrrrz!Failed to multipart ingest runs: rc>[T[5(aT$[R"T[S9R 5$)Nr)rbytesrqtb_multipartMultipartEncoderr+ to_string)rrsrV,Client._send_multipart_req..} s9%dE22&+<<!I#)+ &rZmultipart/form-data; boundary=)"rrtrr"rrrRrfrcrrrr+rrr content_type_TRACING_SEND_TIMEOUTrirrrrrQrRrrrrrmrerr5rd)raccrr"rRrYrZr[r\rrrrrheaders_for_endpointrencoderr _fail_excrrrrrs @@rVrClient._send_multipart_req s ;;Gk&QR $4 N#))4==) '$/"+&3%)-  I87;6J6J6P6P6R7S2N#))4==) .$("&&*#)- 7S  5> 0NQ1 -(",==eiXG{{j0&002&--)*/:$(("6( .0D0D((= (,-!). +.5>!N 665544.. ! )J3%'PQ/2   "V)2)H)HaRS)T#%77>#:#A#A#C)J8*'UV$V)J4PQ7)'TUV !I"''$'Ei[%QR 33I>sP )EA*E  I"#2I"F?? I" AH%$I%+I II II"rc SR[US/55nURR5HupVUR S5 [ SUS-5Hn[ URUSSSSSS9nS[3US'S US 'U(a[US5OS US 'U(a[US5OS US '[RSU35 URSUS3UU[S.SUS9 M M g![Ra  M[R [R"[R$4aAn Xs:Xa[R'SU 35 U n Sn A OUR S5 Sn A GM(Sn A f[(an [*R,"[/U 5U 5n S RU 5R15n [R'SU 35 O1![(a$ [R'S[3U 535 Of=fU n Sn A OSn A ff=fUR S5 UR5UR6S[3S S.5 UR9U 5 GM?)zASend a zstd-compressed multipart form data stream to the backend.rrtrrKNrrrzstdContent-EncodingrzX-Pre-Compressed-SizezX-Post-Compressed-Sizez3Sending compressed multipart request with context: rrrrz,Failed to send compressed multipart ingest: )rr)rmr"rfrcrrrrr+rrQrrrrirrrrrRrrrrrerr5readrd)r data_streamcompressed_traces_inforrr"rRrrrrrrrs rV_send_compressed_multipart_req%Client._send_compressed_multipart_req s '+y""EF $ 4 4 : : < G   Q Q1 -="3 '$("&&*#)-G9 DN+39G./:P2156VX34;Q2156VX45LLMhZX--")?3$/'.'<( ,-!). G.!=N 665544.. ! J3%P03 #((+  " )2)H)HaRS)T#%77>#:#A#A#CJ8*U%J4PQ7)T!"I "  #''$$*H (T,233I>sP BC>>H'2H'F,F H'AG*)H"*+H H"H H""H')rrrend_timerSrrrrtagsr_rWreference_example_idrRr"rYrZr[r\c [US5UUURSS5URSS5URSS5U U URSS5URSS5S. nUbUR5US 'U (abU R5HIunn[ U[ 5(dM[ US [ 5(dM7U (aM@[S 5e U US 'URSL=(d URSL=(a USSL=(a USSLnURU/S S9(dgUbUR5US'OH[RR[RR5R5US'UbUUS'UbURU5US'Ub1U(d[ R""U5nUR%U5US'U bUR'U 5US'US(aDUR)U/5 USR+S5=n(aUR-U5USS'UbUUS'UR.(awUR+S5(daUR0 UR2R5SUUUUUUUS.45 UR75(aUR95 SSS5 gUR;UUUUUUUS9 g!,(df  g=f)aG Update a run in the LangSmith API. Args: run_id (Union[UUID, str]): The ID of the run to update. name (Optional[str]): The name of the run. run_type (Optional[str]): The type of the run (e.g., llm, chain, tool). start_time (Optional[datetime.datetime]): The start time of the run. end_time (Optional[datetime.datetime]): The end time of the run. error (Optional[str]): The error message of the run. inputs (Optional[Dict]): The input values for the run. outputs (Optional[Dict]): The output values for the run. events (Optional[Sequence[dict]]): The events for the run. extra (Optional[Dict]): The extra information for the run. tags (Optional[List[str]]): The tags for the run. attachments (Optional[Dict[str, Attachment]]): A dictionary of attachments to add to the run. The keys are the attachment names, and the values are Attachment objects containing the data and mime type. reference_example_id (Optional[Union[str, uuid.UUID]]): ID of the example that was the source of the run inputs. Used for runs that were part of an experiment. api_key (Optional[str]): The API key to use for this specific run. api_url (Optional[str]): The API URL to use for this specific run. service_key (Optional[str]): The service JWT key for service-to-service auth. tenant_id (Optional[str]): The tenant ID for multi-tenant requests. authorization (Optional[str]): The Authorization header value. cookie (Optional[str]): The Cookie header value. **kwargs (Any): Kwargs are ignored. Returns: None Examples: ```python from langsmith import Client import datetime from uuid import uuid4 client = Client() project_name = "__test_update_run" start_time = datetime.datetime.now() revision_id = uuid4() run: dict = dict( id=uuid4(), name="test_run", run_type="llm", inputs={"text": "hello world"}, project_name=project_name, api_url=os.getenv("LANGCHAIN_ENDPOINT"), start_time=start_time, extra={"extra": "extra"}, revision_id=revision_id, ) # Create the run client.create_run(**run) run["outputs"] = {"output": ["Hi"]} run["extra"]["foo"] = "bar" run["name"] = "test_run_updated" # Update the run client.update_run(run["id"], **run) ``` run_idrHN parent_run_idrm session_idr]) rrrrHr rmr rrr]rrKr`r_Tr?rrSrrrrrr rar@rcrd)rr isoformatrcrrr rrrrLr(rTr)r*r$rir#r%r&r:rr'rrrrIrfrg _update_run)rr rrrrrSrrrrr r_rWr rRr"rYrZr[r\r rrri use_multipartrs rV update_runClient.update_run sn68,  :t4#ZZ>"JJ~t< **\48"JJ~t<   !!+!5!5!7D  !,!2!2!4 :z511":a=$7788$o "5#.D    t + Qt/E/ET/Q 1Z , 1^$D0  (($t(<   '113D '0044X5F5F5J5JKUUWD   !DM  !226:DN   "//8"44W=DO  !::6BDN =  $ $dV ,=,,Z88x8,0,C,CH,MW j)  ++?D' (  ) )&**=V2W2W**$$++'.'.+6)2-:&,   4466..0%+*(   '#+  )+*s A K:: LcURSL=(d URSL=(a USSL=(a USSLnURbURRU5 gU(Ga[ SUS9n URbUcUcUcUcUcUc[ U 5un n [ RSSU R5 URR URc [S5e[U UR[5n U (a9UR=RS- slURR5 SSS5 [![#U R%555 gURb[ RSS U R&U R(5 UR*bKUR-[/USU UUUUUUUR1UR2R555S 9 5 gUR-[/USU UUUUUUS 95 ggUR7UUUUUUUS 9 g!,(df  N=f) NrHrmr@)rpayloadrrornrKrprqrd)rrrrr;r=rQrsrtrurrr9r+rvrrwrrrHrrrSr%rrrx_update_run_non_batch) r run_updaterRr"rYrZr[r\rrzr{r|r}s rVrClient._update_run su  t + Qt/E/ET/Q 7:&d2 7>*$6     (    ( ( 4 .*UM&&2OO'%!)N L!"  K"** ++00119(^ D&..! H  ..::a?:226681T,"5"5"789##/ E!**!$$  %%1++(&~6)$+$+(3&/*7#))-)B)B $ 0 0 A A C*  ++(&~6)$+$+(3&/*7#)  10J  & &'#+ ' g10s A4I  Ic [X2XFU/5nU(a^U=(d URn [URUUUUUURS9n UR SU SUS3[ U5U S.S9 gURR5HCup[URU SSSSSS9n UR SU SUS3[ U5U S.S9 ME g)NrPATCH/runs/rrr) rr"rrrRrrrfrc) rrrRr"rYrZr[r\rrrrrs rVrClient._update_run_non_batch sGk&QR $4 N+ '#+!%G  % %!"&D)9(:;' 3&  & 150D0D0J0J0L, /MM) $""&%)))$oVJt,<+=> +J 7#*$*1MrZcURcgURc [S5eSSKJnJn U"USSS9unnUb:SnUR URUUUS9nURRU5 UR(aG[UR5n[R"XS9upURRU 5 gg![a URXVUS9 Nuf=f) anForce flush the currently buffered compressed runs. Args: attempts: Retry attempts for the send request. timeout: Maximum seconds to wait for in-flight sends to complete. None (default) waits indefinitely. Bounds only the wait on already-submitted sends; the synchronous drain and submit steps above are not bounded by this timeout. Nz>Run compression is enabled but request pool futures is not setr)r'_tracing_thread_drain_compressed_bufferrK)rrrr)rrrrrrrrrJrrcfwaitdifference_update) rrrrrfinal_data_streamrrfuturesdoners rVflush_compressed_tracesClient.flush_compressed_traces*s  ! ! )  == P    4 Q   "  (F 5<<77%*% =  !!&) ==4==)Gggg7GD MM + +D 1   33%4 s7CC-,C-c&Ub[R"5U-OSnUR(a6UR UR(aUR 5 SSS5 UR bUcUR R5 OUR R UR R(aaU[R"5- nUS::aOBUR RRU5 UR R(aMaSSS5 URb7Ub"[SU[R"5- 5OSnURUS9 gg!,(df  GN=f!,(df  Ne=f)zFlush the tracing queue and compressed buffer. Args: timeout: Maximum seconds to wait for pending traces to drain. None (default) waits indefinitely. A timeout of 0 returns immediately without waiting. NrrEr)rN monotonicrrrrgrrmall_tasks_doneunfinished_tasksr!rrr&)rrdeadlinewait_for remainings rVflush Client.flushcs7291D4>>#g-$  ) )**''..0+    )""'')''66,,==#+dnn.>#>#q=!**99>>xH ,,===7  ! ! -9A9MCDNN$445SW   ( ( ( ; .!+*76s"E0&A;F0 E? FcURSURURS9n[R"[ 5n0n[ UR5n[USS9HnURc$[R"SURS35eURRS5Vs1sH nSU;dM URSS 5S iM" nnXX;dMURUR:wdMX6RRU5 XdUR'M URUR/5UlUR#5HupXU lM U$s snf) zLoad child runs for a given run. Args: run (Run): The run to load child runs for. Returns: Run: The run with loaded child runs. Raises: LangSmithError: If a child run has no parent. F)is_rootrrHcUR$r)rmrs rVr)Client._load_child_runs..s!..rZrDz Child run z has no parentr?ZrK) list_runsrrHrrrrrrnr rirrmrrIr child_runsrc) rr+r:treemapr run_id_str child_runseg ancestor_idsr childrens rV_load_child_runsClient._load_child_runssU^^cnns||$   # #D ) 13[  ( I&&.-- 9<<..WXX %1177<") # List LLM and Chat runs in the last 24 hours todays_llm_runs = client.list_runs( project_name="", start_time=datetime.now() - timedelta(days=1), run_type="llm", ) # List root traces in a project root_runs = client.list_runs(project_name="", is_root=1) # List runs without errors correct_runs = client.list_runs(project_name="", error=False) # List runs and only return their inputs/outputs (to speed up the query) input_output_runs = client.list_runs( project_name="", select=["inputs", "outputs"] ) # List runs by run ID run_ids = [ "a36092d2-4ad5-4fb4-9c0d-0dba9a2ed836", "9398e6be-964f-4aa4-8ae9-ad78cd4b7074", ] selected_runs = client.list_runs(id=run_ids) # List all "chain" type runs that took more than 10 seconds and had # `total_tokens` greater than 5000 chain_runs = client.list_runs( project_name="", filter='and(eq(run_type, "chain"), gt(latency, 10), gt(total_tokens, 5000))', ) # List all runs called "extractor" whose root of the trace was assigned feedback "user_score" score of 1 good_extractor_runs = client.list_runs( project_name="", filter='eq(name, "extractor")', trace_filter='and(eq(feedback_key, "user_score"), eq(feedback_score, 1))', ) # List all runs that started after a specific timestamp and either have "error" not equal to null or a "Correctness" feedback score equal to 0 complex_runs = client.list_runs( project_name="", filter='and(gt(start_time, "2023-07-15T12:34:56Z"), or(neq(error, null), and(eq(feedback_key, "Correctness"), eq(feedback_score, 0.0))))', ) # List all runs where `tags` include "experimental" or "beta" and `latency` is greater than 2 seconds tagged_runs = client.list_runs( project_name="", filter='and(or(has(tags, "experimental"), has(tags, "beta")), gt(latency, 2))', ) ``` NrU)app_pathcompletion_costcompletion_tokensrmrrSrrfeedback_statsfirst_token_timerrrrr parent_run_ids prompt_cost prompt_tokensr rrrstatusr  total_cost total_tokensrH child_run_idszOThe child_run_ids field is deprecated and will be removed in following versions)rLrreference_examplerXrPrYrZr2 parent_runrrSrr}rOr /runs/queryr4rDrEr_r]rKrY)rrrrrIrextend read_projectrrorprrrc enumeraterrGr"rrHr])rrNrUrrHr rXrPrYrZr2r rrSr[rOrr  project_idsrdefault_select body_queryr8r9ir+r_s rVr9Client.list_runs s^ j499c"2 3 3   z *  D ) )   z *  #,,, ,~   EQR\T"""588\R  <)> f $ MMa"  '2{t *>%&D(&'4>*..0D#& $%& ((2'7'7'9K'9tq!Qdad'9 K  + +M + K FAsJY K..'+.:>..  QUe^ {SxLs2B G  #G/BG 7 GG A G 0G ;G r)rNrUrrrPrc ZUcUc [S5eUbUb [S5eUbURUS9RnUce[[ US55nUcL[ R R [ RR5[ R"SS9- nURc'UR[ RRS9n/S QnU/S S S UUR5S .n UbXYS'U R5V V s0sH upU cM X_M n n n [R"[ 5n UR#SU S9H0n U R%S5nU(dMXR'U 5 M2 /nU R5HunnUR)SS9 /nUHMn [+U SUR,S9nUR'[.R0"SSU0U DSUR20D65 MO UVs/sH%nUR%S5cM[US5PM' nnUR'UU[5U5U(a [7U5OSU(a [9U5OSS.5 M UR)SS S9 US:aXSnUbUSUnU$s sn n fs snf)aList threads and fetch the runs for each thread. Args: project_id: The project (session) id. project_name: The project name (alternative to project_id). limit: Maximum number of threads to return. Default None (no limit). offset: Pagination offset for threads. Default 0. filter: Optional filter for threads and runs. start_time: Only include runs from this time. Default: 1 day ago. Returns: List of thread items, each with "thread_id", "runs", "count", "min_start_time", and "max_start_time". Nz2Either project_id or project_name must be providedz1Provide exactly one of project_id or project_namer]rNrK)days)tzinfo)rrrfrrrrHrrSr rr rhrgrmr rar^Trdesc)rLr2rrQrOrrPrlrmrcjURS5=(d SURS5=(d S4$)Nrrrmrr4s rVr%Client.list_threads..!s)EE,'-2EE.)/RrZr7rDrEr_r]r)rrrUrrc6URS5=(d S$)Nrrr{)ts rVrr|Bs!%% 017R7rZ)rDreverserrY)rrorrrr(rTr)r* timedeltarxrrrcrrrrrrIsortrGr"rrHr]rminr)rrNrUrrrPrr run_selectrsr8r9 threads_maprun_dicttidrr run_dictsrr_r5 start_timess rV list_threadsClient.list_threadss0  ,"6QR R  !l&>PQ Q  #** *EHHJ%%%*l;<  !**..!!%%""*+J    $#++83D3D3H3H+IJ *#| $..0 &   #)x '1'7'7'9K'9tqQdad'9 K-8-D-DT-J 77 J7WH,,{+Cs ''1X )+$/$5$5$7 Iy NN   D%Mi  NN$/"#'.. &#&"A55&%AlO$" & MM!*  Y:Ec+&64:Ec+&64  1%8D  7   A:G_F  FU^F gL8&s J"J"J(!J()rr}rkr project_namesrqreference_example_idsrrrSrXrPrYrZr2data_source_typecPSSKJnJn U=(d /nU(arU"5nUVs/sHnURURUS9PM nnU"U5H,nUR UR 5R5 M. SSS5 UUUUUUUU U U U U UUUS.nUR5VVs0sHunnUcM UU_M nnnURSSS[U50S 9n[R"U5 UR5$s snf!,(df  N=fs snnf) a Get aggregate statistics over queried runs. Takes in similar query parameters to `list_runs` and returns statistics based on the runs that match the query. Args: id (Optional[List[Union[UUID, str]]]): List of run IDs to filter by. trace (Optional[Union[UUID, str]]): Trace ID to filter by. parent_run (Optional[Union[UUID, str]]): Parent run ID to filter by. run_type (Optional[str]): Run type to filter by. project_names (Optional[List[str]]): List of project names to filter by. project_ids (Optional[List[Union[UUID, str]]]): List of project IDs to filter by. reference_example_ids (Optional[List[Union[UUID, str]]]): List of reference example IDs to filter by. start_time (Optional[str]): Start time to filter by. end_time (Optional[str]): End time to filter by. error (Optional[bool]): Filter by error status. query (Optional[str]): Query string to filter by. filter (Optional[str]): Filter string to apply. trace_filter (Optional[str]): Trace filter string to apply. tree_filter (Optional[str]): Tree filter string to apply. is_root (Optional[bool]): Filter by root run status. data_source_type (Optional[str]): Data source type to filter by. Returns: Dict[str, Any]: A dictionary containing the run statistics. rThreadPoolExecutor as_completedr]N)rr}rkrrLrjrrrSrXrPrYrZr2rrz /runs/statsrr)concurrent.futuresrrrrorIrrrcrrrirrO)rrr}rkrrrqrrrrSrXrPrYrZr2rrrexecutorrr$rrr8r9rs rV get_run_statsClient.get_run_statsKs;\ H!'R #%!. -OOD$5$5DOI -+73F&&v}}'9'9:4 &$ "!6$ (& 0 &%,MMOEODAqq41a4OE,,   G,-  ++H5}}G&%4Fs(D$D  :D2 D"D" D D)rUrNc[USS5=n(aO[USS5=n(aURUS9RnOQUbUnOKUbURUS9RnO.[R"5nURUS9Rn[ US5nUR SUR5SUSURS3$) akGet the URL for a run. Not recommended for use within your agent runtime. More for use interacting with runs after the fact for data analysis or ETL workloads. Args: run (RunBase): The run. project_name (Optional[str]): The name of the project. project_id (Optional[Union[UUID, str]]): The ID of the project. Returns: str: The URL for the run. rNr]r]z/o/z /projects/p/z/r/z ?poll=true)r"rorrirerr]_get_tenant_id)rr+rUrNrr] session_id_s rV get_run_urlClient.get_run_urls*!lD9 9: 9 $S.$? ?\ ?** *EHHJ  ##J  %** *EHHJ#668L** *EHHJz<8 ~~c$"5"5"7!8 [MR  $ rZ)share_idc6[US5n[U5U=(d [[R"55S.nUR SSUS3UR US9n[ R"U5 UR5SnURSUS 3$) aGet a share link for a run. Args: run_id (Union[UUID, str]): The ID of the run to share. share_id (Optional[Union[UUID, str]]): Custom share ID. If not provided, a random UUID will be generated. Returns: str: The URL of the shared run. r )r  share_tokenPUTr/sharerrOr/public//r) rrrrFrrrirrOr])rr rrun_id_rrrs rV share_runClient.share_runs68,'l#8s4::<'8 ,, WIV $MM -  ++H5mmom4 ..!+b99rZcURSS[US5S3URS9n[R"U5 g)ztDelete share link for a run. Args: run_id (Union[UUID, str]): The ID of the run to unshare. Returns: None DELETErr rrNrrrrir)rr rs rV unshare_runClient.unshare_runsG,, Xfh/0 7MM-  ++H5rZcURSS[US5S3URS9n[R"U5 UR 5nUbSU;agUR SUSS 3$) zRetrieve the shared link for a specific run. Args: run_id (Union[UUID, str]): The ID of the run. Returns: Optional[str]: The shared link for the run, or None if the link is not available. rrr rrNrrr)rrrrirrOr])rr rrs rVread_run_shared_linkClient.read_run_shared_links,, Xfh/0 7MM-  ++H5 >]&8..!&*?)@CCrZc@UR[US55nUSL$)zGet share state for a run. Args: run_id (Union[UUID, str]): The ID of the run. Returns: bool: True if the run is shared, False otherwise. r N)rr)rr r^s rV run_is_sharedClient.run_is_shareds&((&()CD4rZc[USSS9up4SUS3nUbUS[US53- nURSUURS 9n[R "U5 [ R"S 0UR5DS UR0D6$) aGet shared runs. Args: share_token (Union[UUID, str]): The share token or URL of the shared run. run_id (Optional[Union[UUID, str]]): The ID of the specific run to retrieve. If not provided, the full shared run will be returned. Returns: Run: The shared run. rr+r$r/runrr rrr]rY) r(rrrrirrrHrOr])rrr rr'rrs rVread_shared_runClient.read_shared_runs,K%H *T*   a234 4D,,  MM-  ++H5~~J J4>>JJrZc## U(aSU0O0n[USSS9upEURSUS3US9H)n[R"S 0UDSUR0D6v M+ g 7f) zGet shared runs. Args: share_token (Union[UUID, str]): The share token or URL of the shared run. run_ids (Optional[List[str]]): A list of run IDs to filter the results by. Yields: A shared run. rrr+rrrlrmr]NrY)r(rrrHr])rrr[r4rr'r+s rVlist_shared_runsClient.list_shared_runs-si#*gr+K%H 22zl+ .T3 C..A3A$..A A sAA dataset_namec tUcUc [S5eUcURUS9RnURSS[ US5S3UR S9n[ R"U5 UR5n[[R0UESURS [ US S 5S 30E5$) aRetrieve the shared schema of a dataset. Args: dataset_id (Optional[Union[UUID, str]]): The ID of the dataset. Either `dataset_id` or `dataset_name` must be given. dataset_name (Optional[str]): The name of the dataset. Either `dataset_id` or `dataset_name` must be given. Returns: ls_schemas.DatasetShareSchema: The shared schema of the dataset. Raises: ValueError: If neither `dataset_id` nor `dataset_name` is given. /Either dataset_id or dataset_name must be givenrr /datasets/ dataset_idrrr-rrzresponse.share_token/d) r read_datasetrrrrrirrOrrDatasetShareSchemar])rrrrds rVread_dataset_shared_schema!Client.read_dataset_shared_schema@s(  ,"6NO O  ** *EHHJ,, *l;B B)rqr name_containsrc## X#US.n[US5n[URSUS3US95H;up[R"S 0U DSUR 0D6v UcM0US-U:dM; g g7f) aList shared projects. Args: dataset_share_token (str): The share token of the dataset. project_ids (Optional[List[Union[UUID, str]]]): List of project IDs to filter the results, by default None. name (Optional[str]): Name of the project to filter the results, by default None. name_contains (Optional[str]): Substring to search for in project names, by default None. limit (Optional[int]): Maximum number of projects to return, by default None. Yields: The shared projects. )rrrdataset_share_tokenrz/datasets/sessionsrr]NrKrY)rrprrTracerSessionResultr]) rrrqrrrrrrtprojects rVlist_shared_projectsClient.list_shared_projectss*$MR24IJ #  $ $;-'9: %  JA 00U7UdnnU U QUe^ sAA1!A1,A1)rrupsert project_extrareference_dataset_idcURS3nUnU(a0U=(d 0ESU0EnUUU[[R"55S.n 0n U(aSU S'UbXiS'UR SU0UR ESS 0E[ U 5U S 9n [R"U 5 [R"S 0U R5DS UR0D6$) a3Create a project on the LangSmith API. Args: project_name (str): The name of the project. project_extra (Optional[dict]): Additional project information. metadata (Optional[dict]): Additional metadata to associate with the project. description (Optional[str]): The description of the project. upsert (bool, default=False): Whether to update the project if it already exists. reference_dataset_id (Optional[Union[UUID, str]): The ID of the reference dataset to associate with the project. Returns: TracerSession: The created project. /sessionsr)rrrrTrrrrrs)rrrr]rY) r"rrrFrrrrirr TracerSessionrOr]) rrUrrrrrr/rr4rrs rVcreate_projectClient.create_projects.ll^9- ; ;j(;E &djjl#    #F8   ++?' (,,  It}}In6HIT" -  ++H5''T(--/TT^^TTrZ)rrrrrcURS[US53nUnU(a0U=(d 0ESU0EnUUUU(aUR5OSS.n URSU0URESS0E[ U 5S 9n [ R"U 5 [R"S 0U R5DS UR0D6$) aUpdate a LangSmith project. Args: project_id (Union[UUID, str]): The ID of the project to update. name (Optional[str]): The new name to give the project. This is only valid if the project has been assigned an end_time, meaning it has been completed/closed. description (Optional[str]): The new description to give the project. metadata (Optional[dict]): Additional metadata to associate with the project. project_extra (Optional[dict]): Additional project information. end_time (Optional[datetime.datetime]): The time the project was completed. Returns: TracerSession: The updated project. /sessions/rNrN)rrrrrrrsrrr]rY) r"rrrrrrirrrrOr]) rrNrrrrrr/rr4rs rVupdate_projectClient.update_project(s<ll^:hz<.P-QR ; ;j(;E&08**,d   ,,  It}}In6HIT" -  ++H5''T(--/TT^^TTrZcURb UR$URSSSS0S9nUR5n[U[5(aS[ U5S:aD[ R"S 0USDSUR0D6nURUlUR$g![a(n[RS[U5S S 9 SnAgSnAff=f) NrrrrKrrr]z*Failed to get tenant ID from LangSmith: %sTrbrY)rrrOrrrrrr]rZrrQrr)rrrtracer_sessionrs rVrClient._get_optional_tenant_idYs ?? &?? " 00{GQ<1H]]_F&$''CK!O!+!?!?"Qi"+/>>"#1":":&    LL? ?rZ)rNrU)rNrU include_statscSnSS0nUbUS[US53- nOUbX%S'O [S5eX5S'URS XES 9nUR5n[ U[ 5(aP[ U5S :Xa[R"S US 35e[R"S0US DSUR0D6$[R"S0UR5DSUR0D6$)aRead a project from the LangSmith API. Args: project_id (Optional[str]): The ID of the project to read. project_name (Optional[str]): The name of the project to read. Only one of project_id or project_name may be given. include_stats (bool, default=False): Whether to include a project's aggregate statistics in the response. Returns: TracerSessionResult: The project. rrrKrrNr'Must provide project_name or project_idrrrrzProject not foundr]rY) rrrrOrrrrirrrr])rrNrUrrrrrs rVroClient.read_projectss*")1  ! a\:;< [@[B[D4URGSS955$s snnfs snnfs snnf!,(df  N=fs snnf)a,Read the record-level information from an experiment into a Pandas DF. !!! note This will fetch whatever data exists in the DB. Results are not immediately available in the DB upon evaluation run completion. Feedback score values will be returned as an average across all runs for the experiment. Non-numeric feedback scores will be omitted. Args: project_id (Optional[Union[UUID, str]]): The ID of the project. project_name (Optional[str]): The name of the project. Returns: pd.DataFrame: A dataframe containing the test results. z%Function get_test_results is in beta.rrgrrNT)rr rrrSrarr)rNrUr2rOc >TRUS9nUVVVs/sHMnSUR0UR=(d 0R5VVs0sH up4SU3U_M snnEPMO snnn$s snnfs snnnf)N)r example_idz reference.) list_examplesrrrc)batchexamplesrr8r9rs rVfetch_examples/Client.get_test_results..fetch_exampless))e)>3CD3C41as|Q3CD 78iio25L5L5NO5NTQ!~q(5NO  ::all2AAC!WW$$ ##JJ)*(8(8(>(>(@(@1$%KAEE*4E:isOQUU5\9(@''++F33$%$4$4V$<$@$@$LLL'/G ))&&q'='=>NNT!$$/L#MN{#j0A5 +V5H IIuIx~u'MN,s#EJ7#x~67?STUQ"RLL)33LA !-g!6 !6vV]]_6F_F!6   o6@@NJ"((tQU(VI  d38n%y'8'8 '8'J K  [EO#" !T sD3N3N! *N3N' BN3$#N- N-$D(N3$O!N33 Oc ## SU b [U S5OS0n UbXS'UbX+S'UbX;S'UbUb [S5eXKS'O UbURUS 9RnXKS'UbXkS 'UbX{S 'UbXS 'U b[R "U 5U S '[ URSU S95H;up[R"S0U DSUR0D6v U cM0U S-U :dM; g g7f)aList projects from the LangSmith API. Args: project_ids (Optional[List[Union[UUID, str]]]): A list of project IDs to filter by, by default None name (Optional[str]): The name of the project to filter by, by default None name_contains (Optional[str]): A string to search for in the project name, by default None reference_dataset_id (Optional[List[Union[UUID, str]]]): A dataset ID to filter by, by default None reference_dataset_name (Optional[str]): The name of the reference dataset to filter by, by default None reference_free (Optional[bool]): Whether to filter for only projects not associated with a dataset. limit (Optional[int]): The maximum number of projects to return, by default None metadata (Optional[Dict[str, Any]]): Metadata to filter by. Yields: The projects. Raises: ValueError: If both reference_dataset_id and reference_dataset_name are given. rNrrrrzGOnly one of reference_dataset_id or reference_dataset_name may be givenreference_datasetrreference_freerdataset_versionrrrr]rKrY) rrrrrOrPrprrrr])rrqrrrreference_dataset_namerrr rrrrtrs rV list_projectsClient.list_projects#s@P (9S_s"   "&4L  !6N  $&3? #  +%1 ;+?& ' # /#'#4#43$5$b !+?& '  %'5# $  $&3? #  &(7$ %  !%H!5F: #  $ $[ $ @ JA00U7UdnnU U QUe^  sCC+C+&C+cUb#[URUS9R5nOUc [S5eUR SS[ US53UR S9n[R"U5 g)aDelete a project from LangSmith. Args: project_name (Optional[str]): The name of the project to delete. project_id (Optional[str]): The ID of the project to delete. Returns: None Raises: ValueError: If neither project_name or project_id is provided. Nr]rrrrNr) rrorrrrrrir)rrUrNrs rVdelete_projectClient.delete_projectns{$  #T..L.ILLMJ  FG G,, *l;< =MM-  ++H5rZ)rr inputs_schemaoutputs_schematransformationsrc0S[R"50U=(d 0EnUURUS[R"50U=(d 0ESS.S.nUbX(S'UbXHS'UbXXS'URSS 0URES S 0E[ R "U5S 9n [R"U 5 U R5n U RS 5=(d UU S '[R"S0U DURUR5S.D6$)a(Create a dataset in the LangSmith API. Args: dataset_name (str): The name of the dataset. description (Optional[str]): The description of the dataset. data_type (DataType, default=DataType.kv): The data type of the dataset. inputs_schema (Optional[Dict[str, Any]]): The schema definition for the inputs of the dataset. outputs_schema (Optional[Dict[str, Any]]): The schema definition for the outputs of the dataset. transformations (Optional[List[DatasetTransformation]]): A list of transformations to apply to the dataset. metadata (Optional[dict]): Additional metadata to associate with the dataset. Returns: Dataset: The created dataset. Raises: requests.HTTPError: If the request to create the dataset fails. r0sdk)rsource)rrr)rrinputs_schema_definitionoutputs_schema_definitionrrrrsrrr rY)r1r2rlrrr$rPrirrOrrrr]r) rrrrr'r(r)rdatasetr json_responses rVcreate_datasetClient.create_datasets2Fv==?THNPRT ".v==?~2  #   "%0M "  $2?. /  %3A/ 0,,  It}}In6HIw' -  ++H5  $1$5$5j$A$MX j!!!  nn335  rZrrcXURXS9 g![Ra gf=f)aCheck whether a dataset exists in your tenant. Args: dataset_name (Optional[str]): The name of the dataset to check. dataset_id (Optional[Union[UUID, str]]): The ID of the dataset to check. Returns: bool: Whether the dataset exists. r3TF)rrir)rrrs rV has_datasetClient.has_datasets3"    <  O..  rcSnSS0nUbUS[US53- nOUbXS'O [S5eURSUUS 9nUR5n[ U[ 5(a_[ U5S :Xa[R"S US 35e[R"S0US DURUR5S .D6$[R"S0UDURUR5S .D6$)zRead a dataset from the LangSmith API. Args: dataset_name (Optional[str]): The name of the dataset to read. dataset_id (Optional[Union[UUID, str]]): The ID of the dataset to read. Returns: Dataset: The dataset. rrrKrrr'Must provide dataset_name or dataset_idrrrr rr rY) rrrrOrrrrirrrr]r)rrrrrrrs rVrClient.read_datasets$")1  ! a\:;< >zJKH)))NN,,@**=*=*@*@ ..!00 " 2 2 *G $$.67hHHh70891YY9&zz% !   ? ~N55  "s[)A>>BBBF(E7*E? E7 E! E76F(7 F!FF!!F((F+c8URUS9nUR$)N)r)rr)rrr/s rVrClient._get_data_types!##z#:   rZc0URSU0SU0UUUS9$)aAdd an example (row) to an LLM-type dataset. Args: prompt (str): The input prompt for the example. generation (Optional[str]): The output generation for the example. dataset_id (Optional[Union[UUID, str]]): The ID of the dataset. dataset_name (Optional[str]): The name of the dataset. created_at (Optional[datetime.datetime]): The creation timestamp of the example. Returns: Example: The created example inputrrrrr created_at)create_example)rr generationrrrps rVcreate_llm_exampleClient.create_llm_examples54""V$z*!%! #  rZc /nUH|n[R"U5(a?UR[R"[ [ R U555 M]UR[ [U55 M~ SnUbZ[R"U5(a/[R"[ [ R U55nO[ [U5nURSU0UbSU0OSUUUS9$)a+Add an example (row) to a Chat-type dataset. Args: messages (List[Union[Mapping[str, Any], BaseMessageLike]]): The input messages for the example. generations (Optional[Union[Mapping[str, Any], BaseMessageLike]]): The output messages for the example. dataset_id (Optional[Union[UUID, str]]): The ID of the dataset. dataset_name (Optional[str]): The name of the dataset. created_at (Optional[datetime.datetime]): The creation timestamp of the example. Returns: Example: The created example Nrnrro) riis_base_message_likerIconvert_langchain_messagerrBaseMessageLikerrq) rmessages generationsrrrp final_inputmessagefinal_generationss rVcreate_chat_exampleClient.create_chat_examples8 G,,W55""66Z77A ""4g#67 !  ",,[99$,$F$F33[A%!%){$;!""[)1B1N,-TX!%!#  rZc<UcURUS9RnSnURU5nU[RR :XaUR S:wa[SUR S35e[R"UR5nSU0nUR(dSnGO[R"UR5n S U 0nOU[RR:XaUR S:wa[SUR S 35eS[R"UR50nUR(dSnOjS [R"UR50nOFU[RR :XaURnURnO[SUS35eUR#UUUUUS9$![a& [SURSUR35ef=f![a& [S URS UR35ef=f![a& [S URSUR35ef=f![a& [SURS UR35ef=f)afAdd an example (row) to a dataset from a run. Args: run (Run): The run to create an example from. dataset_id (Optional[Union[UUID, str]]): The ID of the dataset. dataset_name (Optional[str]): The name of the dataset. created_at (Optional[datetime.datetime]): The creation timestamp of the example. Returns: Example: The created example Nrr1z Run type z+ is not supported for dataset of type 'LLM'z2Error converting LLM run inputs to prompt for run z with inputs rnz7Error converting LLM run outputs to generation for run z with outputs rz, is not supported for dataset of type 'chat'z9Error converting LLM run inputs to chat messages for run z=Error converting LLM run outputs to chat generations for run z Dataset type z not recognized.ro)rrrrrcr1rrriget_prompt_from_inputsrrget_llm_generation_from_outputschatget_messages_from_inputs#get_message_generation_from_outputsrdrq) rr+rrrp dataset_typerrrrrs rVcreate_example_from_runClient.create_example_from_run s$  ** *EHHJL11*= :..22 2||u$  ~.11 !88D '.v%6F;;48!)!I!I#++!VJ $Z0 Z0055 5||u$  ~.22 !8#D#DSZZ#PQ ;;  ("N"NKK#GZ0033 3ZZFkkG}\N:JKL L""!%! #  g  x}SZZL: "$FF8>#++@  x}SZZL: "$$$'FF8>#++Hs08 G2 H"H8"I+0H0H580I(+0Jc /n0nU(a<[US[R5(d [S5eUSRnUGHn[U[R 5(dI[U[R5(d*[U[R 5(d [S5eURb[UR5nO[[R"55n[U[R 5(aSn O URn [U[R 5(a%URn URn URn OSupn 0U(aSW0O0EU bSU 0O0EU (aSU 0O0EU (aSU 0O0EU (aS U 0O0En URbURU S 'UR bUR U S '[#U 5nUR%USUS 0445 UR&b/[#UR&5nUR%US 3SUS 0445 UR(b/[#UR(5nUR%US3SUS 0445 UR*(GaUR*R-5Hunn[U[.5(a USnUSnOUunn[U[05(aU(a|[2R4R7U5n[9US5nUU[U5[[R"55-'UR%USU3SUUSU30445 M[S5eUR%USU3SUUS[AU530445 M [U[R 5(dGMURB(dGM[#URB5nUR%US3SUS 0445 GM [DRF"U[HS9nUR@S::aURK5nOUnUUU4$![:a [<R?SUU5 GMf=f)NrzWThe examples must be of type ExampleUpsertWithAttachments if include_dataset_id is Truez[The examples must be of type ExampleCreate or ExampleUpsertWithAttachments or ExampleUpdateNNNrrpuse_source_run_iouse_source_run_attachments source_run_idrrrsz.inputsz.outputs mime_typerrz,Attachment file not found for example %s: %sz .attachment.z ; length=zMdangerously_allow_filesystem must be True to upload files from the filesystemz.attachments_operationsrr)&rrExampleUpsertWithAttachmentsrr ExampleCreate ExampleUpdaterrrrFrprrrrrrrIrrr_rcrr rrgetsizerFileNotFoundErrorrQrRrattachments_operationsrrr+r)rrinclude_dataset_idrWrrrrrrprrr example_bodyvalbinputsboutputsbrrirattachment_data file_sizer attachments_operationsbrrs rV_prepare_multipart_dataClient._prepare_multipart_dataes&(:< hqk:+R+RSS 5"!//JGw (@(@AA"7J,S,STT"7J,D,DEE ( zz% _  . ':#;#;<<! $// ':#;#;<<$+$=$=!-4-O-O* ' 5 5 PL!} 1CL*-1;1GL*-R ))*;< 223MN8EO]3"L++2+;+; Z(}}((/ W%|,D LL!l*  ~~)%gnn5 %,g. #.  *&w7 %,h/ $.  """(/(;(;(A(A(C$D*!*d33$.{$; *4V*<5?2 ?!/4887 ),.GGOOO,L '+OT'B!%. #O 4s4::<7H H"LL'1l,tf$E(,(,+4+Yyk(J(* %&!" #- o# #-,l4& A$($3'0k3;O:P$Q$& !" M)Df7J$<$<==222*5g6T6T*U' %,&=> 3.  k B!11%)L ;;* $$$&DD///}$5) &$R$.$3!" !) )s4+Q!Q98Q9)updatesrWc$URUUUS9$)z{Update examples using multipart. .. deprecated:: 0.3.9 Use Client.update_examples instead. Will be removed in 0.4.0. rrrW)_update_examples_multipart)rrrrWs rVupdate_examples_multipart Client.update_examples_multipart"%..!)E/  rZcURR=(d 0RSS5(d [S5eUc/nUR USUS9upEnUR S[ URU5U0URESUR0ES.S9n[R"U5 [[UR555 UR5$![[UR555 f=f) a Update examples using multipart. Args: dataset_id (Union[UUID, str]): The ID of the dataset to update. updates (Optional[List[ExampleUpdate]]): The updates to apply to the examples. Raises: ValueError: If the multipart examples endpoint is not enabled. "dataset_examples_multipart_enabledFzYour LangSmith deployment does not allow using the latest examples endpoints, please upgrade your deployment to the latest version or downgrade your SDK to langsmith<0.3.9.rrWrrrrrinstance_flagsrrrr_dataset_examples_pathr"rrrirrwrrrO)rrrrWrrrrs rVr!Client._update_examples_multipart5s  ((.B33 0%  &  ?G+/+G+G $)E,H, ((  ;00&t||Z@  -- &(<(<  1 H  / / 9 /6689 :}} /6689 : AC$$$D)uploadsrWc$URUUUS9$)z{Upload examples using multipart. .. deprecated:: 0.3.9 Use Client.create_examples instead. Will be removed in 0.4.0. rrrW)_upload_examples_multipart)rrrrWs rVupload_examples_multipart Client.upload_examples_multipartgrrZcSnUR(a!U[[UR55- nUR(a!U[[UR55- nUR(a!U[[UR55- nUR (aUR R 5Hpup4[U[5(aUSnOUup5[U[5(a$U[RRU5- nOU[U5- nUS- nMr U$![[4a US- nN"f=f)z?Estimate the size of an example in bytes for batching purposes.rrr)rrrrrr_rcrrr rrrrOSError)rrsizerrirs rV_estimate_example_sizeClient._estimate_example_sizezs >> C GNN34 4D ?? C GOO45 5D    C G$4$456 6D   !(!4!4!:!:!< j$//&0&8O)3&Aot44* @@C00D "= .w7* )*s>"D::EEcd/n/nSnUHnURU5nXr:a0U(aURU5 /nSnURU/5 MIXW-U:nU(aU(aURU5 U/nUnMvURU5 XW- nM U(aURU5 U$)zBatch examples by size limits.r)rrI) rrrbatches current_batch current_sizer example_size size_exceededs rV_batch_examples_by_sizeClient._batch_examples_by_sizes 8:  G66w?L2 NN=1$&M#$Ly)(7:NNM}-!( + $$W-, / 4  NN= )rZcURR=(d 0RSS5(d [S5eUc/nUR USUS9upEnUR S[ URU5U0URESUR0ES.S9n[R"U5 [[UR555 UR5$![[UR555 f=f) aUpload examples using multipart. Args: dataset_id (Union[UUID, str]): The ID of the dataset to upload to. uploads (Optional[List[ExampleCreate]]): The examples to upload. dangerously_allow_filesystem (bool): Whether to allow uploading files from the filesystem. Returns: ls_schemas.UpsertExamplesResponse: The count and ids of the successfully uploaded examples Raises: ValueError: If the multipart examples endpoint is not enabled. rFYour LangSmith deployment does not allow using the multipart examples endpoint, please upgrade your deployment to the latest version.rrrrrr)rrrrWrrrrs rVr!Client._upload_examples_multiparts( ((.B33 0%  X  ?G+/+G+G $)E,H, ((  ;00&t||Z@  -- &(<(<  1 H  / / 9 /6689 :}} /6689 :r)upsertsrWc>URR=(d 0RSS5(d [S5eUc/nUR USUS9up4nUR SUR SSS :waUR S SS :waS OS U0URESUR0ES.S9n[R"U5 [[UR555 UR5$![[UR555 f=f)zUpsert examples. .. deprecated:: 0.3.9 Use Client.create_examples and Client.update_examples instead. Will be removed in 0.4.0. examples_multipart_enabledFrNTrr/v1z/v1/z/v1/platform/examples/multipartz/platform/examples/multipartrrr)rrrrrrr"rrrirrwrrrO)rrrWrrrrs rVupsert_examples_multipart Client.upsert_examples_multiparts6 ((.B33 (%  X  ?G+/+G+G #)E,H, ((  ;00||BC(E1dll236G66Q67! -- &(<(<  1H  / / 9 /6689 :}} /6689 :s A,C88$D)rrrrWmax_concurrencyc RSUs=::aS::d O [S5eU(a<U(a5SRUVs/sH nSUS3PM sn5n[SUS35e1Skn U(a9[U5RU 5=n (a[S [ U 5S 35eU(dU(d [S 5eU(dUR US 9R nU(a?UV s/sH0n [U [5(a[R"S!0U D6OU PM2 n n GOURS 5n U (d [S5e[U 5nUR5H8unnUcM [U5U:wdM[SUS[U5SUS35e [U URS5=(d S[U55URS5=(d S[U55URS5=(d S[U55URS5=(d S[U55URS5=(d S[U555VVVVVVs/sHLunnnnnn[R"S!0UUUUU=(d [!["R$"55US.D6PMN n nnnnnnU (d[R&"/SSS 9$UR)U 5nUR+UX$U5$s snfs sn fs snnnnnnf)"aH Create examples in a dataset. Args: dataset_name (str | None): The name of the dataset to create the examples in. Must specify exactly one of dataset_name or dataset_id. dataset_id (UUID | str | None): The ID of the dataset to create the examples in. Must specify exactly one of dataset_name or dataset_id examples (Sequence[ExampleCreate | dict]): The examples to create. dangerously_allow_filesystem (bool): Whether to allow uploading files from the filesystem. **kwargs (Any): Legacy keyword args. Should not be specified if 'examples' is specified. - inputs (Sequence[Mapping[str, Any]]): The input values for the examples. - outputs (Optional[Sequence[Optional[Mapping[str, Any]]]]): The output values for the examples. - metadata (Optional[Sequence[Optional[Mapping[str, Any]]]]): The metadata for the examples. - splits (Optional[Sequence[Optional[str | List[str]]]]): The splits for the examples, which are divisions of your dataset such as 'train', 'test', or 'validation'. - source_run_ids (Optional[Sequence[Optional[Union[UUID, str]]]]): The IDs of the source runs associated with the examples. - ids (Optional[Sequence[Union[UUID, str]]]): The IDs of the examples. Raises: ValueError: If 'examples' and legacy args are both provided. Returns: The LangSmith JSON response. Includes 'count' and 'example_ids'. !!! warning "Behavior changed in `langsmith` 0.3.11" Updated to take argument 'examples', a single list where each element is the full example to create. This should be used instead of the legacy 'inputs', 'outputs', etc. arguments which split each examples attributes across arguments. Updated to support creating examples with attachments. Example: ```python from langsmith import Client client = Client() dataset = client.create_dataset("agent-qa") examples = [ { "inputs": {"question": "what's an agent"}, "outputs": {"answer": "an agent is..."}, "metadata": {"difficulty": "easy"}, }, { "inputs": { "question": "can you explain the agent architecture in this diagram?" }, "outputs": {"answer": "this diagram shows..."}, "attachments": {"diagram": {"mime_type": "image/png", "data": b"..."}}, "metadata": {"difficulty": "medium"}, }, # more examples... ] response = client.create_examples(dataset_name="agent-qa", examples=examples) # -> {"example_ids": [... ``` rKr\z'max_concurrency must be between 1 and 3rfrbzCannot specify z when 'examples' is specified.>idsrsplitsrrsource_run_idsz(Received unsupported keyword arguments: r?z3Either dataset_id or dataset_name must be provided.rrz+Must specify either 'examples' or 'inputs.'N Length of  (z#) does not match length of inputs (rgrc3&# UHnSv M g7frrYrrs rVr)Client.create_examples..s-MJ!*-KL   c&k&<&<=M&NN{N:5;M:NaP lRS S** *EHHJ "!A20$sL&7L:ALc /nSnSnSSKJn U"US9n UV s/sH!n U RURU UU5PM# n n [R "U 5Hnn U R 5n URU RS/55 XmRSS5- nU RS5nU(dMbUbX:dMlUnMp SSS5 [R"XVUS9$s sn f!,(df  N(=f)Nr)ContextThreadPoolExecutor max_workersrrUrSr) langsmith.utilsrr_upload_single_batchr rrrnrrr)rrrrWrall_examples_ids total_count latest_as_ofrrrr$rrrSs rVr(Client._upload_examples_batches_parallels  = &? Cx%%E --0  % //'2!==? '' ]B(GH||GQ77 ! W-5l2e6J#(L3D,00(<  )D Cs)C,(C'A6C,:C,C,'C,, C:ctURR=(d 0RSS5(a+UR[ [ R U5UUS9nU$UH/n[US5cMSUl[R"S5 M1 URSS0URES S 0E[UVs/sHn0[US S 9ES [U50EPM sn5S9n[ R""U5 UR%5nUVs/sHowSPM sn['U5SS.$s snfs snf)zIUpload a single batch of examples (used by both sequential and parallel).rFrr_Nz7Must upgrade your LangSmith version to use attachments.r/examples/bulkrrsT exclude_nonerrrr)rrrrrrrr"r_rorprrr dump_modelrrirrOr)rrrrWrupload response_datars rVrClient._upload_single_batchsQ II $ $ * / / 0%  66 :6-I7H O 6=1=)-F&MMQ 00 M4==M.:LM ',  ',F (dC(#j/',  1 H  / / 9$MMOM7DE}tT }E]+  Fs <#D0D5c UcU (d [S5eUcURUS9Rn[R"S0UUUUU U U U S.D6n U(aXMlU(a-[ U[5(a[R"U5OUO[R"5U lURR=(d 0RSS5(a)URX-/S9 URU RS9$[!U S 5bSU l[$R&"S 5 UR)S S 0UR*ES S0E[-0[/U 5R15VVs0sH upUcM X_M snnES[U50E5S9n[2R4"U5 UR75n[R8"S0UDUR:UR=5S.D6$s snnf)aCreate a dataset example in the LangSmith API. Examples are rows in a dataset, containing the inputs and expected outputs (or other reference information) for a model or chain. Args: inputs (Mapping[str, Any]): The input values for the example. dataset_id (Optional[Union[UUID, str]]): The ID of the dataset to create the example in. dataset_name (Optional[str]): The name of the dataset to create the example in. created_at (Optional[datetime.datetime]): The creation timestamp of the example. outputs (Optional[Mapping[str, Any]]): The output values for the example. metadata (Optional[Mapping[str, Any]]): The metadata for the example. split (Optional[str | List[str]]): The splits for the example, which are divisions of your dataset such as 'train', 'test', or 'validation'. example_id (Optional[Union[UUID, str]]): The ID of the example to create. If not provided, a new example will be created. source_run_id (Optional[Union[UUID, str]]): The ID of the source run associated with this example. use_source_run_io (bool): Whether to use the inputs, outputs, and attachments from the source run. use_source_run_attachments (Optional[List[str]]): Which attachments to use from the source run. If use_source_run_io is True, all attachments will be used regardless of this param. attachments (Optional[Attachments]): The attachments for the example. Returns: Example: The created example. Nz/Must provide either inputs or use_source_run_ior)rrrrrrrr_rF)rr)rr_z6Must upgrade your LangSmith version to use attachmentsrrrrsrrr rY)rrrrrrprrrrrFrrrr read_exampler"r_rorprrrrrcrirrOrr]r)rrrrrprrrrrrrr_rr8r9rrs rVrqClient.create_examplesl >"3NO O  ** *EHHJ''   "$!.%6.H*    (O'1S&A&ATYYz "z  II $ $ * / / 0%    + +z6 + R$$$8 8t]+7#'  VW00M4==M.:LM ,6t,<,B,B,DV,DDA414,DV$c*o 1 H  / / 9]]_F%%..779 Ws 3 G7G7)rSc URSS[US53SU(aUR5OS0S9nUR5n[ USUR S9n[ R"S 0UR5VVs0sHupgUS:wdM Xg_M snnDUURUR5S .D6$s snnf) aRead an example from the LangSmith API. Args: example_id (Union[UUID, str]): The ID of the example to read. as_of (Optional[datetime.datetime]): The dataset version tag OR timestamp to retrieve the example as of. Response examples will only be those that were present at the time of the tagged (or timestamped) version. Returns: Example: The example. r /examples/rrSNrattachment_urlsrEr_r]rrY) rrrrOrGr"rrrcr]r)rrrSrrr_r8r9s rVrClient.read_example|s,, *l;< =e*- --/E %6  !! ' J18I3ItqtJ #nn335   Js  B<B<)rrrrPinclude_attachmentsc +# 0U EUU[U[R5(aUR5OUUUUb [US5OSU S.En U b[ U 5U S'UbXS'O"UbUR US9R nXS'OU (a/SQU S'[URS U S 95Hup[US URS 9n[R"S0UR5VVs0sHunnUS :wdMUU_M snnDUURUR5S .D6v UcMUS-U:dM g gs snnf7f)a Retrieve the example rows of the specified dataset. Args: dataset_id (Optional[Union[UUID, str]]): The ID of the dataset to filter by. dataset_name (Optional[str]): The name of the dataset to filter by. example_ids (Optional[Sequence[Union[UUID, str]]): The IDs of the examples to filter by. as_of (Optional[Union[datetime.datetime, str]]): The dataset version tag OR timestamp to retrieve the examples as of. Response examples will only be those that were present at the time of the tagged (or timestamped) version. splits (Optional[Sequence[str]]): A list of dataset splits, which are divisions of your dataset such as 'train', 'test', or 'validation'. Returns examples only from the specified splits. inline_s3_urls (bool, default=True): Whether to inline S3 URLs. offset (int, default=0): The offset to start from. Defaults to 0. limit (Optional[int]): The maximum number of examples to return. metadata (Optional[dict]): A dictionary of metadata to filter by. filter (Optional[str]): A structured filter string to apply to the examples. include_attachments (bool, default=False): Whether to include the attachments in the response. **kwargs (Any): Additional keyword arguments are ignored. Yields: The examples. Examples: List all examples for a dataset: ```python from langsmith import Client client = Client() # By Dataset ID examples = client.list_examples( dataset_id="c9ace0d8-a82c-4b6c-13d2-83401d68e9ab" ) # By Dataset Name examples = client.list_examples(dataset_name="My Test Dataset") ``` List examples by id ```python example_ids = [ "734fc6a0-c187-4266-9721-90b7a025751a", "d6b4c1b9-6160-4d63-9b61-b034c585074f", "4d31df4e-f9c3-4a6e-8b6c-65701c2fed13", ] examples = client.list_examples(example_ids=example_ids) ``` List examples by metadata ```python examples = client.list_examples( dataset_name=dataset_name, metadata={"foo": "bar"} ) ``` List examples by structured filter ```python examples = client.list_examples( dataset_name=dataset_name, filter='and(not(has(metadata, \'{"foo": "bar"}\')), exists(metadata, "tenant_id"))', ) ``` Nr)rrrSrinline_s3_urlsrrPrr/r)rrrrOrrrrErrKrY)rr(rrrrrrprrGr"rrrcr]r)rrrrrSrr rrrrPrr rrtrr_r8r9s rVrClient.list_examplessjl "  " %/x7H7H%I%I!u,(-(9S_s "   !,X!6F:   ! *9   %** *EHHJ *9   IF8 #  $ $[ $ @ JAJ):DLLK$$$+MMONODAqq E )rrrrrrr_cJUb?URR=(d 0RSS5(d [S5e[ UUUUUUUS9n [ R "S 0U R5V V s0sH upU cM X_M sn n D6n UcURU5RnURR=(d 0RSS5(a[ URXl/S95$URSS[US530URES S 0E[0[U 5R5V V s0sH upU cM X_M sn n ES [!U50E5S 9n ["R$"U 5 U R'5$s sn n fs sn n f)aUpdate a specific example. Args: example_id (Union[UUID, str]): The ID of the example to update. inputs (Optional[Dict[str, Any]]): The input values to update. outputs (Optional[Mapping[str, Any]]): The output values to update. metadata (Optional[Dict]): The metadata to update. split (Optional[str | List[str]]): The dataset split to update, such as 'train', 'test', or 'validation'. dataset_id (Optional[Union[UUID, str]]): The ID of the dataset to update. attachments_operations (Optional[AttachmentsOperations]): The attachments operations to perform. attachments (Optional[Attachments]): The attachments to add to the example. Returns: Dict[str, Any]: The updated example. rFYour LangSmith deployment does not allow using the attachment operations, please upgrade your deployment to the latest version.)rrrrrrr_)rrrrrrrsrrrY)rrrrrrrrcrrrrrrrrrrirrO)rrrrrrrrr_ example_dictr8r9rrs rVupdate_exampleClient.update_examplesH " -II,,2774e!V#9# ** , 2 2 4F 4tqt 4F   **:6AAJ II $ $ * / / 0%  //)90 00Xj,?@AM4==M.:LM )37(;(A(A(C(C !AD(C %c*o   1H  / / 9==? "AG,s7 FF FF)rrrrWc b U(aU(a[S[U5S35eU(dU(d [S5eU(aU(a [S5eU(aURUS9RnU(avUVs/sH0n[ U[ 5(a[ R"S-0UD6OUPM2 nnU(d0USR(aUSRnGO[S5eGOURSS 5nU(d [S 5eU(dkS U;aURUS5RnOF[[US 55S :a [S 5eUS S(d [S5eUS SnURR=(d 0RS5n U (d6URS5=(d URS5b [S5e[U5n UR5H7upU cM [U 5U :wdM[SU S[U 5SU S35e [!UURSS[#U 555URSS[#U 555URSS[#U 555URSS[#U 555URS S[#U 555URSS [#U 555URSS![#U 5555V VVVVVVVs /sH*upnnnnnn[ R"S-0U UUUUUUUS".D6PM, nnnnnnnn nS nURR=(d 0RSS#5(aBUR%['[(R*U5UUS$9nS%URS&S5S'30UE$UR-S(S)0UR.ES*S+0E[1UVVVs/sH8n[3U5R5VVs0sHunnUcM UU_M snnPM: snnn5S,9n[4R6"U5 UR95$s snfs s nnnnnnnn fs snnfs snnnf).a Update multiple examples. Examples are expected to all be part of the same dataset. Args: dataset_name (str | None): The name of the dataset to update. Should specify exactly one of 'dataset_name' or 'dataset_id'. dataset_id (UUID | str | None): The ID of the dataset to update. Should specify exactly one of 'dataset_name' or 'dataset_id'. updates (Sequence[ExampleUpdate | dict] | None): The example updates. Overwrites any specified fields and does not update any unspecified fields. dangerously_allow_filesystem (bool): Whether to allow using filesystem paths as attachments. **kwargs (Any): Legacy keyword args. Should not be specified if 'updates' is specified. - example_ids (Sequence[UUID | str]): The IDs of the examples to update. - inputs (Sequence[dict | None] | None): The input values for the examples. - outputs (Sequence[dict | None] | None): The output values for the examples. - metadata (Sequence[dict | None] | None): The metadata for the examples. - splits (Sequence[str | list[str] | None] | None): The splits for the examples, which are divisions of your dataset such as 'train', 'test', or 'validation'. - attachments_operations (Sequence[AttachmentsOperations | None] | None): The operations to perform on the attachments. - dataset_ids (Sequence[UUID | str] | None): The IDs of the datasets to move the examples to. Returns: The LangSmith JSON response. Includes 'message', 'count', and 'example_ids'. !!! warning "Behavior changed in `langsmith` 0.3.9" Updated to ... Example: ```python from langsmith import Client client = Client() dataset = client.create_dataset("agent-qa") examples = [ { "inputs": {"question": "what's an agent"}, "outputs": {"answer": "an agent is..."}, "metadata": {"difficulty": "easy"}, }, { "inputs": { "question": "can you explain the agent architecture in this diagram?" }, "outputs": {"answer": "this diagram shows..."}, "attachments": {"diagram": {"mime_type": "image/png", "data": b"..."}}, "metadata": {"difficulty": "medium"}, }, # more examples... ] response = client.create_examples(dataset_name="agent-qa", examples=examples) example_ids = response["example_ids"] updates = [ { "id": example_ids[0], "inputs": {"question": "what isn't an agent"}, "outputs": {"answer": "an agent is not..."}, }, { "id": example_ids[1], "attachments_operations": [ {"rename": {"diagram": "agent_diagram"}, "retain": []} ], }, ] response = client.update_examples(dataset_name="agent-qa", updates=updates) # -> {"example_ids": [... ``` z&Must pass in either 'updates' or args z , not both.z6Please pass in a non-empty sequence for arg 'updates'.z;Must pass in exactly one of 'dataset_name' or 'dataset_id'.rrz=Must pass in (exactly) one of 'dataset_name' or 'dataset_id'.rNz9Must pass in (exactly) one of 'updates' or 'example_ids'.rHrKz-Dataset IDs must be the same for all examplesz+If specified, dataset_ids must be non-null.rrr_r rrz%) does not match length of examples (rgrc3&# UHnSv M g7frrYrs rVr)Client.update_examples..)L8K1$8Krrc3&# UHnSv M g7frrYrs rVrrs*M9LA49Lrrc3&# UHnSv M g7frrYrs rVrrs+N:MQD:Mrrc3&# UHnSv M g7frrYrs rVrrrrc3&# UHnSv M g7frrYrs rVrr .Q=Pt=Prc3&# UHnSv M g7frrYrs rVrr!rrc3&# UHnSv M g7frrYrs rVrr#s2UATA4ATr)rrrrrrr_rFrr|rUz examples updatedrrrrsrrY)rrrrrrrrrrrrrrrrcrrrrrrrrrrrirrO)rrrrrWr r updates_objrmultipart_enabled examples_lenrrrrrrr dataset_id_ attachments_attachments_operations_rrr8r9s rVupdate_examplesClient.update_examplesssp g8v{S 'UV V JM ** *EHHJ ! A2,,!,Q!:!:J$W !**]D9K O .!%!2!2;q>!B!M!MJVM234q8$%TUU .q1$%RSS!' !6q!9J!%!9!9!?R D D4! &ZZ 89VVZZ =V!X {+L'-||~#(S^|-K$$XJbY0@A00<~Q@(6&nqJJx)Ll8K)LMJJy*M|9L*MNJJz+N% :M+NOJJx)Ll8K)LMJJ}.QU<=P.QRJJ}.QU<=P.QRJJ02U|AT2U n niCdIv{LRi(( !"%#'&1$-!''32I   n 6 II $ $ * / / 0%  66 :6#-I7Hhll7A677HI  00 M4==M.:LM,7  ,7-7w,?,E,E,G,GDAq#$!%1,G ,7   1H"  / / 9==? "afds*7R1R#"R* R$R$R*$R*cURSS[US53URS9n[R"U5 g)zDelete an example by ID. Args: example_id (Union[UUID, str]): The ID of the example to delete. Returns: None rrrrNr)rrrs rVdelete_exampleClient.delete_exampleLsE,, *l;< =MM-  ++H5rZ) hard_deletec ZU(a[U5VVs/sHup4[[USUS355PM snnSS.n[URS5n[ URU5nUR RSU0URESS0E[U5URS 9nO[S [U5VVs/sHup4[[USUS355PM snn0n URS S 0URESS0EU S 9n[R"U5 gs snnfs snnf)zDelete multiple examples by ID. Parameters ---------- example_ids : Sequence[ID_TYPE] The IDs of the examples to delete. hard_delete : bool, default=False If True, permanently delete the examples. If False, soft delete them. example_ids[]T)rr'zdatasets/examples/deleterrrs)rrrrrrr>N)rprr_platform_pathr"rrLrrrrrrir) rrr'rtrr4rfull_urlrrs rVdelete_examplesClient.delete_examples]sB #,K"8 "8 QCq&9:;"8  $ D"$,,0JKD%dllD9H||++M4==M.:LM & ,H"+K"8 "8 QCq&9:;"8 &F 00M4==M.:LM 1H ++H5= & s %D!>%D')rrrScRUc'Uc [S5eURUS9Rn0nUb4[U[R5(aUR 5OUUS'UR SS[US5S3US9n[R"U5 UR5$) aGet the splits for a dataset. Args: dataset_id (Optional[Union[UUID, str]]): The ID of the dataset. dataset_name (Optional[str]): The name of the dataset. as_of (Optional[Union[str, datetime.datetime]]): The version of the dataset to retrieve splits for. Can be a timestamp or a string tag. Defaults to "latest". Returns: List[str]: The names of this dataset's splits. Must provide dataset name or IDrrSrrr/splitsr) rrrrr(rrrrirrO)rrrrSrrs rVlist_dataset_splitsClient.list_dataset_splitss&  # !BCC** *EHHJ  %/x7H7H%I%I!u 7O,, *l;[U[5(aV0UEnSU;aSU;aUSUS'URSS5 TbURS5cTUS'TR"S0UD6$U$)N reasoningcommentrDrY)rrrrEvaluationResult) single_result merged_resultrB ls_evaluators rV _cast_result1Client._select_eval_results.._cast_results-..0A=0A -/I]4R/<[/IM),!!+t4&=+<+._is_eval_results,sgt,Eg1E ErZz!Invalid evaluation results type: z.. Must be EvaluationResult, EvaluationResults.)rHz*Union[ls_evaluator.EvaluationResult, dict]rls_evaluator.EvaluationResult)rrrz)TypeGuard[ls_evaluator.EvaluationResults])langsmith.evaluationrrrGrrrr)rrrBrKrNresults_r5rJs ` @rV_select_eval_resultsClient._select_eval_resultss C !E ! * ! ! F g|<< = =yHg & &181CD1CA Q1CHD & &$T$%89:H  3DM?C@@  EsB*) source_inforjrIcURXS9nURXF5nURUUS9nURUUUS9n U S$)aEvaluate a run. Args: run (Union[Run, RunBase, str, UUID]): The run to evaluate. evaluator (RunEvaluator): The evaluator to use. source_info (Optional[Dict[str, Any]]): Additional information about the source of the evaluation to log as feedback metadata. reference_example (Optional[Union[Example, str, dict, UUID]]): The example to use as a reference for the evaluation. If not provided, the run's reference example will be used. load_child_runs (bool, default=False): Whether to load child runs when resolving the run ID. Returns: Feedback: The feedback object created by the evaluation. r:rrUr)r<r@ evaluate_run_log_evaluation_feedback rr+rrUrjrIr;r?evaluator_responsers rVrYClient.evaluate_run<sq<##C#I!556GN&33 &4 //  #0  qzrZ) _executorc H^^TRU5nUU4SjnUGHnU=(d 0n UR(a0UREU En Sn UR(a URn OUb URn URbURR SS5OSn U"S0SU _SUR _SUR_SUR_SUR_SUR_S U _S UR_S [[[RUR 5_S [R"R$_S U_SUR_SU(a UR&OS_SU(a UR(OS_SU(a UR*OS_SU _6 GM U$)Nct>T(aTR"TR40UD6 gTR"S0UD6 g)NrY)rcreate_feedback)r r^rs rV_submit_feedback9Client._log_evaluation_feedback.._submit_feedbackus0  !5!5@@$$.v.rZrSr rDrXrlrF correctionrUrfeedback_configfeedback_source_typerNrrHrrrY)rSevaluator_info target_run_idrrrrDrXrlrFrdrrrrFeedbackConfigreFeedbackSourceTypeMODELrHrr) rr\r+rUrNr^rrbres source_info_rrSs ` ` rVrZClient._log_evaluation_feedbackhs++,>? / C&,"L!!E#"4"4E E G  ++&&47II4ICIIMM'40tE   GG ii ii      >>  ) "// !%Z6679L9L! &0%B%B%H%H & ii *-$  .13>>d! ".13>>d# $% >rZc# URXS9nURXF5nURUUS9IShvN nURUUUS9n U S$N7f)aEvaluate a run asynchronously. Args: run (Union[Run, str, UUID]): The run to evaluate. evaluator (RunEvaluator): The evaluator to use. source_info (Optional[Dict[str, Any]]): Additional information about the source of the evaluation to log as feedback metadata. reference_example (Optional[Union[Example, str, dict, UUID]]): The example to use as a reference for the evaluation. If not provided, the run's reference example will be used. load_child_runs (bool, default=False): Whether to load child runs when resolving the run ID. Returns: EvaluationResult: The evaluation result object created by the evaluation. r:rWNrXr)r<r@ aevaluate_runrZr[s rVrpClient.aevaluate_runs~<##C#I!556GN#,#:#: &$;$   //  #0  qz s4AAAr])rXrlrHrdrFrUrfr feedback_idrerrNcomparative_experiment_idfeedback_group_idrrSrrc  U=(d UnUcUc [S5eUbUb [S5eU(a'[R"S[U53[5 [ U [ R5(d[ R"U 5n U [ RR:Xa[ R"US9nOAU [ RR:Xa[ R"US9nO[SU 35eURb URO0Ul U b*SUR;aS[U 50URS'UR(aSUR;aURSn[US 5(a*[[!US 55(aUR#5nSU;a([[%URSSS 55US'UURS'['UbUOUS S 9n[ R("S(0S ['U 5_S['US S 9_S['US S 9_SU_S[+U5_SU_SU_SU_SU_S[,R,R/[,R0R25_S[,R,R/[,R0R25_SU _SU_SU_S['US S 9_S['US S 9_SU_SU_6nUR4(+=(a/ UR6R8=(d 0R;SS 5nU(GamUR6R<(GaQ[>R@"UR6R<S5(Ga URBcURDGbURFbURHc[KU5nURDb[MU5nURDRN [QUURD[R5nU(aJURD=RTS - sl*URV(aURVRY5 SSS5 OxURBb.UR[[][UR^5U55 O<[aUR#S S!95nURcS"S#S$U0U [>Rd4S%9 [ Rf"S(0UR#5D6$!,(df  N2=f![han[jRmS&S S'9 UeSnAff=f))aCreate feedback for a run. !!! note To enable feedback to be batch uploaded in the background you must specify `trace_id`. *We highly encourage this for latency-sensitive environments.* Args: key (str): The name of the feedback metric. score (Optional[Union[float, int, bool]]): The score to rate this run on the metric or aspect. value (Optional[Union[float, int, bool, str, dict]]): The display value or non-numeric value for this feedback. run_id (Optional[Union[UUID, str]]): The ID of the run to provide feedback for. At least one of run_id, trace_id, or project_id must be specified. trace_id (Optional[Union[UUID, str]]): The ID of the trace (i.e. root parent run) of the run to provide feedback for (specified by run_id). If run_id and trace_id are the same, only trace_id needs to be specified. **NOTE**: trace_id is required feedback ingestion to be batched and backgrounded. correction (Optional[dict]): The proper ground truth for this run. comment (Optional[str]): A comment about this feedback, such as a justification for the score or chain-of-thought trajectory for an LLM judge. source_info (Optional[Dict[str, Any]]): Information about the source of this feedback. feedback_source_type (Union[FeedbackSourceType, str]): The type of feedback source, such as model (for model-generated feedback) or API. source_run_id (Optional[Union[UUID, str]]): The ID of the run that generated this feedback, if a "model" type. feedback_id (Optional[Union[UUID, str]]): The ID of the feedback to create. If not provided, a random UUID will be generated. feedback_config (Optional[FeedbackConfig]): The configuration specifying how to interpret feedback with this key. Examples include continuous (with min/max bounds), categorical, or freeform. stop_after_attempt (int, default=10): The number of times to retry the request before giving up. project_id (Optional[Union[UUID, str]]): The ID of the project (or experiment) to provide feedback on. This is used for creating summary metrics for experiments. Cannot specify run_id or trace_id if project_id is specified, and vice versa. comparative_experiment_id (Optional[Union[UUID, str]]): If this feedback was logged as a part of a comparative experiment, this associates the feedback with that experiment. feedback_group_id (Optional[Union[UUID, str]]): When logging preferences, ranking runs, or other comparative feedback, this is used to group feedback together. extra (Optional[Dict]): Metadata for the feedback. session_id (Optional[Union[UUID, str]]): The session (project) ID of the run this feedback is for. Used to optimize feedback ingestion by avoiding server-side lookups. start_time (Optional[datetime]): The start time of the run this feedback is for. Used to optimize feedback ingestion by avoiding server-side lookups. **kwargs (Any): Additional keyword arguments. Returns: Feedback: The created feedback object. Example: ```python from langsmith import trace, traceable, Client @traceable def foo(x): return {"y": x * 2} @traceable def bar(y): return {"z": y - 1} client = Client() inputs = {"x": 1} with trace(name="foobar", inputs=inputs) as root_run: result = foo(**inputs) result = bar(**result) root_run.outputs = result trace_id = root_run.id child_runs = root_run.child_runs # Provide feedback for a trace (a.k.a. a root run) client.create_feedback( key="user_feedback", score=1, trace_id=trace_id, ) # Provide feedback for a child run foo_run_id = [run for run in child_runs if run.name == "foo"][0].id client.create_feedback( key="correctness", score=0, run_id=foo_run_id, # trace_id= is optional but recommended to enable batched and backgrounded # feedback ingestion. trace_id=trace_id, ) ``` Nz8One of run_id, trace_id, or project_id must be providedz?project_id cannot be provided if run_id or trace_id is providedzLThe following arguments are no longer used in the create_feedback endpoint: )rzUnknown feedback source type __runr rz+feedback_source.metadata['__run']['run_id']TrorrHrDrXrlrdrFfeedback_sourcerp modified_atrerrrsrtrrSuse_multipart_endpointz0.8.10rKrr /feedbackr)rrrzError creating feedbackrbrY)7rrorprnrrrrjAPIAPIFeedbackSourcerkModelFeedbackSourcerrrr!r"rrrnFeedbackCreaterYr(rTr)r*rrrrrriis_version_greater_or_equalrrrHrr:r<rur9r+rvrrrSr%rrrrFeedbackrrQrS) rr rDrXrlrHrdrFrUrfrrrrerrNrsrtrrSrrr rw _run_meta _session_idfeedbackrrzr{r}feedback_blockrs rVraClient.create_feedbacksV#8 >j0WX X  *"8Q   MM$Vn-/"  p 2J4Q4QRR'1'D'D(($$z'D'D'H'HH00+F &)F)F)L)LL","@"@+"V #@AU@V!WXX,;,D,D,P((VX  $(WO.F.Fw.O 9l33I|499!* 4 4 6Iy(*- +44W=hGI+Ih' 5>((1&(4 *RVK"00 ,$F=  &hDA   -U3& !0$,,001B1B1F1FG%--11(2C2C2G2GH!0' &!"+7-4+#(#//@d"S)*+,-H2!% 8 882 --3c*D1  II%%%889J9JHUU&&2d6L6L6X%%1&&. 7 A ))5T)# //44#G* 22%$ $ 22>>!C>#99 $ : : > > @54''3++(X[[)9=I"-X-@-@d-@-S!T))$(:&==?*&&?)<)<)>? ?3544  LL2TL BG s3NT:A-T 'B$T TT U'T==U)rXrlrdrFc0nUb[U5US'UbX6S'UbXFS'UbXVS'URSS[US530URES S 0E[ U5S 9n[ R "U5 g) aUpdate a feedback in the LangSmith API. Args: feedback_id (Union[UUID, str]): The ID of the feedback to update. score (Optional[Union[float, int, bool]]): The score to update the feedback with. value (Optional[Union[float, int, bool, str, dict]]): The value to update the feedback with. correction (Optional[dict]): The correction to update the feedback with. comment (Optional[str]): The comment to update the feedback with. Returns: None NrXrlrdrFr /feedback/rrrrsr)rYrrrrrir)rrrrXrlrdrFfeedback_updaters rVupdate_feedbackClient.update_feedbacks4+-  '=e'DOG $  ',G $  !,6L )  )0I &,, +}=> ?It}}In6HI_- -  ++H5rZcURSS[US535n[R"S0UR 5D6$)zRead a feedback from the LangSmith API. Args: feedback_id (Union[UUID, str]): The ID of the feedback to read. Returns: Feedback: The feedback. rrrrrY)rrrrrOrrrrs rV read_feedbackClient.read_feedbacksE,, +}=> ? ""5X]]_55rZ)r[ feedback_keyrfrc+# UUb [US5OSS.UEnUbX&S'UbX6S'[URSUS95H-upx[R"S 0UD6v UcM"US-U:dM- g g7f) adList the feedback objects on the LangSmith API. Args: run_ids (Optional[Sequence[Union[UUID, str]]]): The IDs of the runs to filter by. feedback_key (Optional[Sequence[str]]): The feedback key(s) to filter by. Examples: 'correctness' The query performs a union of all feedback keys. feedback_source_type (Optional[Sequence[FeedbackSourceType]]): The type of feedback source, such as model or API. limit (Optional[int]): The maximum number of feedback to return. **kwargs (Any): Additional keyword arguments. Yields: The feedback objects. Nr)r+rrDr,rzrrKrY)rrprrr) rr[rrfrr rrtrs rV list_feedbackClient.list_feedback s8(-(9S_s    #(5M  +38 $  $ $[ $ @ KA%%11 1 QUe^  sAA2"A2-A2cURSS[US53URS9n[R"U5 g)zDelete a feedback by ID. Args: feedback_id (Union[UUID, str]): The ID of the feedback to delete. Returns: None rrrrrNrrs rVdelete_feedbackClient.delete_feedback8sE,, +}=> ?MM-  ++H5rZ)rlrdrFrc [XRSS9upxXpR:wa[SU35eURSS[ U53[ UUUUUS.5UR S9n [R"U 5 g) aMCreate feedback from a presigned token or URL. Args: token_or_url (Union[str, uuid.UUID]): The token or URL from which to create feedback. score (Optional[Union[float, int, bool]]): The score of the feedback. value (Optional[Union[float, int, bool, str, dict]]): The value of the feedback. correction (Optional[dict]): The correction of the feedback. comment (Optional[str]): The comment of the feedback. metadata (Optional[dict]): Additional metadata for the feedback. Raises: ValueError: If the source API URL is invalid. Returns: None rK)r#zInvalid source API URL. rz/feedback/tokens/)rXrlrdrFrrN) r(r"rrrrrrir) rrfrXrlrdrFrr`r'rs rVcreate_feedback_from_token!Client.create_feedback_from_tokenIs8&9 ,,!& " \\ )77GHI I,,  45 6""",& (  MM-  ++H5rZ) expirationrerrcUUUU=(d [[R"55S.nUc[R"SSSS9US'O[ U[ R 5(aUR5US'Ow[ U[ R5(aA[R"URURS-URS-S-S9US'O[S [U535eURS S [U5S 9n[R "U5 [R""S 0UR%5D6$)aCreate a pre-signed URL to send feedback data to. This is useful for giving browser-based clients a way to upload feedback data directly to LangSmith without accessing the API key. Args: run_id (Union[UUID, str]): The ID of the run. feedback_key (str): The key of the feedback to create. expiration (Optional[datetime.datetime | datetime.timedelta]): The expiration time of the pre-signed URL. Either a datetime or a timedelta offset from now. Default to 3 hours. feedback_config (Optional[FeedbackConfig]): If creating a feedback_key for the first time, this defines how the metric should be interpreted, such as a continuous score (w/ optional bounds), or distribution over categorical values. feedback_id (Optional[Union[UUID, str]): The ID of the feedback to create. If not provided, a new feedback will be created. Returns: FeedbackIngestToken: The pre-signed URL for uploading feedback data. )r rrerrr\rwhoursminutes expires_in expires_atrDUnknown expiration type: r/feedback/tokens)rrY)rrrFrTimeDeltaInputrr(rrrwsecondsrrrrrirFeedbackIngestTokenrO)rr rrrerrr4rs rVcreate_presigned_feedback_token&Client.create_presigned_feedback_token{s.F(.2TZZ\!2    !+!:!:"D   H$5$5 6 6!+!5!5!7D   H$6$6 7 7!+!:!:__ ((D0#++r1R7"D  8j9I8JKL L,,  T"-  ++H5--@ @@rZ)rfeedback_configsc b^^Ub#[U5[U5:wa [S5eU(dS/[U5-nSupVUc[R"SSSS9nO[ U[ R 5(aUR 5nOt[ U[ R5(a>[R"URURS-URS-S-S9nO[S [U535e[[X$5VVs/sH upxUUUUUS .PM snn5mS UU4S jjn /n [R"[TR5S 9n TRR!5V V s/sHupU R#XU 5PM nn n [R$"U5HInUR'5nU R)UVs/sHn[R*"S0UD6PM sn5 MK SSS5 U $s snnfs sn n fs snf!,(df  U $=f)aCreate a pre-signed URL to send feedback data to. This is useful for giving browser-based clients a way to upload feedback data directly to LangSmith without accessing the API key. Args: run_id (Union[UUID, str]): The ID of the run. feedback_keys (Sequence[str]): The key of the feedback to create. expiration (Optional[datetime.datetime | datetime.timedelta]): The expiration time of the pre-signed URL. Either a datetime or a timedelta offset from now. Default to 3 hours. feedback_configs (Optional[Sequence[Optional[FeedbackConfig]]]): If creating a feedback_key for the first time, this defines how the metric should be interpreted, such as a continuous score (w/ optional bounds), or distribution over categorical values. Returns: Sequence[FeedbackIngestToken]: The pre-signed URL for uploading feedback data. NzBThe length of feedback_keys and feedback_configs must be the same.NNrr\rrrDr)r rrerrc >TRSUS3T0TRE[U=(d TR0ES.S9n[R "U5 UR 5$)Nrrrr)rrrvrRrirrO)r"rRrr4rs rVreq4Client.create_presigned_feedback_tokens..reqsn00)+,  -- !7#:dll  1 H  / / 9==? "rZr)r"rrRrrrrY)rrrrrr(rrrwrrrrr rrfrcrrrrnr)rr  feedback_keysrrrrrrertokensrr"rRfutsfutrrr4s` @rV create_presigned_feedback_tokens'Client.create_presigned_feedback_tokenss"D  'C ,>#FVBW,WT  $vM(:: !+  #22J  H$5$5 6 6#--/J  H$6$6 7 7#22__ ((D0#++r1R7J 8j9I8JKL L69!6 61L %$0'6",", 6    # #  " "s43G3G/H IX)-(<(<(B(B(D(D$Gg6(D t,::< HPQZ33;d;Q- J Q >RJ I s0H "HH;H H 9 H H H.)rc## [US5Ub [US5OSS.n[URSUS95H-upE[R "S0UD6v UcM"US-U:dM- g g7f) zList the feedback ingest tokens for a run. Args: run_id (Union[UUID, str]): The ID of the run to filter by. limit (Optional[int]): The maximum number of tokens to return. Yields: The feedback ingest tokens. r Nr)r rrrrKrY)rrrprrr)rr rrrttokens rVlist_presigned_feedback_tokens%Client.list_presigned_feedback_tokens"sx vx0(-(9S_s "  $ $%7 $ G HA00959 9 QUe^  sAA,A,'A,rrrrc## Ub [US5OSUb [US5OSUb [US5OSUS.n[URSUS95H-upg[R "S 0UD6v UcM"US-U:dM- g g7f) aList feedback formulas. Args: dataset_id (Optional[Union[UUID, str]]): The ID of the dataset to filter by. session_id (Optional[Union[UUID, str]]): The ID of the session to filter by. limit (Optional[int]): The maximum number of feedback formulas to return. offset (int): The starting offset for pagination. Yields: The feedback formulas. Nrrrr/feedback/formulasrrKrY)rrrprrFeedbackFormula)rrrrrrrtfeedback_formulas rVlist_feedback_formulasClient.list_feedback_formulas<s27A6L\2RV7A6L\2RV(-(9S_s " $-  $ $%9& $ I$ A,,@/?@ @ QUe^ $ sA,B2B=BcURSS[US535n[R"U5 [R "S0UR 5D6$)zGet a feedback formula by ID. Args: feedback_formula_id (Union[UUID, str]): The ID of the feedback formula to retrieve. Returns: The requested feedback formula. r/feedback/formulas/feedback_formula_idrY)rrrirrrrOrrrs rVget_feedback_formula_by_id!Client.get_feedback_formula_by_iddsU,, !(+>@U"V!W X  ++H5))@U"V!W X G$6$6D$6$IJ-  ++H5))@U"V!W X  ++H5rZ)is_lower_score_bettercUUUS.nURSSUS9n[R"U5 [R"S0UR 5D6$)aLCreate a feedback configuration. Defines how feedback with a given key should be interpreted. If an identical configuration already exists for the key, it is returned unchanged. If a different configuration already exists for the key, an error is raised. Args: feedback_key (str): The feedback key to configure. feedback_config (FeedbackConfig): The configuration defining type, bounds, and categories. is_lower_score_better (Optional[bool]): Whether a lower score is considered better. Defaults to False. Returns: FeedbackConfigSchema: The created or existing feedback configuration. Raises: requests.HTTPError: If a conflicting configuration already exists for the given key (HTTP 400). Example: .. code-block:: python from langsmith import Client client = Client() config = client.create_feedback_config( feedback_key="user-rating", feedback_config={ "type": "continuous", "min": 0.0, "max": 5.0, }, ) )rrerr/feedback-configsr6rYrrirrFeedbackConfigSchemarOrrrerr4rs rVcreate_feedback_configClient.create_feedback_configs_^).%:  ,,  -  ++H5..AAArZ)rrrrc## Ub [US5OSUS.nUbXS'UbX%S'[URSUS95H-upg[R"S 0UD6v UcM"US-U:dM- g g7f) aList feedback configurations. Args: feedback_key (Optional[Sequence[str]]): Filter by specific feedback keys. name_contains (Optional[str]): Filter by substring match on the feedback key. limit (Optional[int]): The maximum number of configurations to return. offset (int): The number of configurations to skip. Defaults to 0. Yields: FeedbackConfigSchema: The feedback configurations. Example: .. code-block:: python from langsmith import Client client = Client() for config in client.list_feedback_configs(): print(f"{config.feedback_key}: {config.feedback_config}") NrrrrDrrrrKrY)rrprrr)rrrrrrrtrzs rVlist_feedback_configsClient.list_feedback_configs& sB).(9S_s"   #(5M  $&3? #"  $ $%8 $ H IA11;F; ; QUe^  sAA0 A0+A0)rercSU0nUbX$S'UbX4S'URSSUS9n[R"U5 [R"S0UR 5D6$)a Update a feedback configuration. Only the provided fields will be updated; others remain unchanged. Args: feedback_key (str): The feedback key of the configuration to update. feedback_config (Optional[FeedbackConfig]): The new configuration values. is_lower_score_better (Optional[bool]): Whether a lower score is considered better. Returns: FeedbackConfigSchema: The updated feedback configuration. Raises: LangSmithNotFoundError: If no configuration exists for the given feedback key (HTTP 404). Example: .. code-block:: python from langsmith import Client client = Client() config = client.update_feedback_config( "user-rating", is_lower_score_better=True, ) rrerrrr6rYrrs rVupdate_feedback_configClient.update_feedback_configU syL L   &&5" # ,,A( ),,  -  ++H5..AAArZcVURSSSU0S9n[R"U5 g)anDelete a feedback configuration. This performs a soft delete. The configuration can be recreated later with the same key. Args: feedback_key (str): The feedback key of the configuration to delete. Example: .. code-block:: python from langsmith import Client client = Client() client.delete_feedback_config("user-rating") rrrrNrrir)rrrs rVdelete_feedback_configClient.delete_feedback_config s7$,,  "L1-  ++H5rZ) queue_idsrrrc #:# Ub.[U5VVs/sHupV[USUS35PM snnOSUUUb [US5OSS.n[URSUS95H-upX[R "S 0UD6v UcM"US-U:dM- g gs snnf7f) aList the annotation queues on the LangSmith API. Args: queue_ids (Optional[List[Union[UUID, str]]]): The IDs of the queues to filter by. name (Optional[str]): The name of the queue to filter by. name_contains (Optional[str]): The substring that the queue name should contain. limit (Optional[int]): The maximum number of queues to return. Yields: The annotation queues. Nz queue_ids[r*r)rrrr/annotation-queuesrrKrY)rprrrrAnnotationQueue) rrrrrrtrrqueues rVlist_annotation_queuesClient.list_annotation_queues s4(AJ)@TU@Tfa#A3a01@TU*(-(9S_s  "  $ $%9& $ I HA,,  QUe^ VsBBABB B)rqueue_idrubric_instructions rubric_itemsc dUUUb [U5O[[R"55US.nUbXVS'URSSUR 5VVs0sH upxUcM Xx_M snnS9n [ R "U 5 [R"S0U R5D6$s snnf)aCreate an annotation queue on the LangSmith API. Args: name (str): The name of the annotation queue. description (Optional[str]): The description of the annotation queue. queue_id (Optional[Union[UUID, str]]): The ID of the annotation queue. rubric_instructions (Optional[str]): The rubric instructions for the annotation queue. rubric_items (Optional[list[AnnotationQueueRubricItem]]): The feedback configs to assign to this queue's rubric. Each item specifies a feedback_key and optional per-queue customization like description and value_descriptions. Returns: AnnotationQueue: The created annotation queue object. )rrrrrrrr6rY) rrrFrrcrirrAnnotationQueueWithDetailsrO) rrrrrrr4r8r9rs rVcreate_annotation_queueClient.create_annotation_queue s:&#+#7#h-S=N#6    ##/ ,,  #'::<A<411$!$<A-  ++H544 mmo  Bs  B,&B,cS[US53nURSUURS9n[R"U5 [ R "S0UR5D6$)zRead an annotation queue with the specified `queue_id`. Args: queue_id (Union[UUID, str]): The ID of the annotation queue to read. Returns: AnnotationQueue: The annotation queue object. /annotation-queues/rrrrY)rrrrirrrrO)rrbase_urlrs rVread_annotation_queueClient.read_annotation_queue sf)(J)G(HI,, jMM-  ++H544Gx}}GGrZ)rrrrc0nUbX&S'UbX6S'UbXFS'UbXVS'URSS[US53US 9n[R"U5 g) aUpdate an annotation queue with the specified `queue_id`. Args: queue_id (Union[UUID, str]): The ID of the annotation queue to update. name (Optional[str]): The new name for the annotation queue. description (Optional[str]): The new description for the annotation queue. rubric_instructions (Optional[str]): The new rubric instructions for the annotation queue. rubric_items (Optional[list[AnnotationQueueRubricItem]]): The feedback configs to assign to this queue's rubric. Returns: None Nrrrrrrrr6r)rrrrrrr4rs rVupdate_annotation_queueClient.update_annotation_queue!s~0 "  L  ""-   **=& '  ##/ ,, !(8Z"@!A B-  ++H5rZcURSS[US53SS0URES9n[R"U5 g)zDelete an annotation queue with the specified `queue_id`. Args: queue_id (Union[UUID, str]): The ID of the annotation queue to delete. Returns: None rrrrtrsrNr)rrrs rVdelete_annotation_queueClient.delete_annotation_queue6!sP,, !(8Z"@!A B1CT]]C-  ++H5rZcURSS[US5S3[U5VVs/sHup4[[USUS355PM snnS9n[R "U5 gs snnf) aAdd runs to an annotation queue with the specified `queue_id`. Args: queue_id (Union[UUID, str]): The ID of the annotation queue. run_ids (List[Union[UUID, str]]): The IDs of the runs to be added to the annotation queue. Returns: None rrrrzrun_ids[r*r6N)rrrprrir)rrr[rtrrs rVadd_runs_to_annotation_queue#Client.add_runs_to_annotation_queueF!su,, !(8Z"@!A GDMgDVWDV&!#hshqcO45DVW-  ++H5Xs%A,c URSS[US5S[US535n[R"U5 g)a Delete a run from an annotation queue with the specified `queue_id` and `run_id`. Args: queue_id (Union[UUID, str]): The ID of the annotation queue. run_id (Union[UUID, str]): The ID of the run to be added to the annotation queue. Returns: None rrrrr Nr)rrr rs rV delete_run_from_annotation_queue'Client.delete_run_from_annotation_queueZ!sI,, !(8Z"@!AQWYaHbGc d  ++H5rZcS[US5S3nURSUSU3URS9n[R"U5 [ R "S0UR5D6$)aGet a run from an annotation queue at the specified index. Args: queue_id (Union[UUID, str]): The ID of the annotation queue. index (int): The index of the run to retrieve. Returns: RunWithAnnotationQueueInfo: The run at the specified index. Raises: LangSmithNotFoundError: If the run is not found at the given index. LangSmithError: For other API-related errors. rrrrrrrY)rrrrirrRunWithAnnotationQueueInforO)rrrrrs rVget_run_from_annotation_queue$Client.get_run_from_annotation_queuem!sq )(J)G(HM,, j% !MM-  ++H544Gx}}GGrZ)rrrprrc|U(d [S5eUcURUSS9RnU(d [S5eU=(d [[R "55UUUUU=(d7 [ R R[ RR50S.nUbXhSS'[UR5V V s0sHupX_M sn n 5n URSS S U 0S 9n [R"U 5 U R5n [ R""S 0U D6$s sn n f) a-Create a comparative experiment on the LangSmith API. These experiments compare 2 or more experiment results over a shared dataset. Args: name (str): The name of the comparative experiment. experiments (Sequence[Union[UUID, str]]): The IDs of the experiments to compare. reference_dataset (Optional[Union[UUID, str]]): The ID of the dataset these experiments are compared on. description (Optional[str]): The description of the comparative experiment. created_at (Optional[datetime.datetime]): The creation time of the comparative experiment. metadata (Optional[Dict[str, Any]]): Additional metadata for the comparative experiment. id (Optional[Union[UUID, str]]): The ID of the comparative experiment. Returns: ComparativeExperiment: The created comparative experiment object. z$At least one experiment is required.rrz A reference dataset is required.)rrexperiment_idsrrrprrrrz/datasets/comparativerrrY)rrorrrrFr(rTr)r*rrcrrirrOrComparativeExperiment)rr experimentsrrrprrr4r8r9serr response_ds rVcreate_comparative_experiment$Client.create_comparative_experiment!s-6CD D  $ $ 1 1&q>!2!"" !?@ @)DJJL))$5&$T(9(9(=(=h>O>O>S>S(T   (0M* %DJJL9LDA14L9:,,  #-  ++H5]]_ //=*==:sD8 cXUR5nUS:H=(d URU:H$)zCheck if the current workspace has the same handle as owner. Args: owner (str): The owner to check against. Returns: bool: True if the current tenant is the owner, False otherwise. r)r tenant_handle)rrsettingss rV_current_tenant_is_ownerClient._current_tenant_is_owner!s,%%'|>x55>>rZcp[R"SUSUR5RSU35$)NzCannot z% for another tenant. Current tenant: z, Requested tenant: )rirlrr)ractionrs rV_owner_conflict_errorClient._owner_conflict_error!sH**fX#113AABC!!& )  rZcxURSSUS3X#S.S9nUR5SnU(aUSS$S $) agGet the latest commit hash for a prompt. Args: prompt_owner_and_name (str): The owner and name of the prompt. limit (int, default=1): The maximum number of commits to fetch. Defaults to 1. offset (int, default=0): The number of commits to skip. Defaults to 0. Returns: Optional[str]: The latest commit hash, or None if no commits are found. r /commits/rrrcommitsrrN)rrO)rprompt_owner_and_namerrrrs rV_get_latest_commit_hashClient._get_latest_commit_hash!sX,, -.a 0"5-  --/),,3wqz-(==rZcT^^^[U[5(aU/OUnSUUU4Sjjn[R"5nUVs/sHovR XW5PM nn[R "U5Hn U R 5 M SSS5 gs snf!,(df  g=f)aOUpdate tags for a prompt commit. Args: prompt_owner_and_name (str): The owner and name of the prompt in the format 'owner/repo'. commit_id (str): The commit hash/ID to tag. tags (Union[str, list[str]]): A single tag or list of tags to apply to the commit. Raises: requests.exceptions.HTTPError: If the request fails. cf>UTS.nTRSSTS3US9n[R"U5 g)N)tag_name commit_idr/repos/rQr6r)rTrrr%rrs rV create_tag.Client._create_commit_tags..create_tag!sI&G00'"7!8>W1H  / / 9rZN)rTr)rrr rrrr) rrr%r tag_listr'rrTr$rs ``` rV_create_commit_tagsClient._create_commit_tags!s(c22D6 : : " " $CKL8Cz78GL//'2 3% $L% $sBB/BB B'c[R"U5up4nURSSUSU3SU0S9nUR5 UR 5$)aRLike or unlike a prompt. Args: prompt_identifier (str): The identifier of the prompt. like (bool): True to like the prompt, False to unlike it. Returns: A dictionary with the key 'likes' and the count of likes as the value. Raises: requests.exceptions.HTTPError: If the prompt is not found or another error occurs. rz/likes/rliker6)rirrraise_for_statusrO)rrr-r prompt_namerrs rV_like_or_unlike_promptClient._like_or_unlike_prompt "sa !) @ @AR SA,, geWAk]364.-  !!#}}rZc [R"U5up#nURU5(dURSUSUSUSS3$UR 5nURSUSUSSSUR 3$)zGet a URL for a prompt. Args: prompt_identifier (str): The identifier of the prompt. Returns: str: The URL for the prompt. z/hub/rr*Nr:z /prompts/z?organizationId=)rirrr]rr)rrrr/rrs rV_get_prompt_urlClient._get_prompt_url$"s+3*J*J + 'K,,U33nn%U5';-qRa@QR R%%'~~i }Ak"1o5Fx{{m - rZc:URU5nU(aS$S$)zCheck if a prompt exists. Args: prompt_identifier (str): The identifier of the prompt. Returns: bool: True if the prompt exists, False otherwise. TF) get_prompt)rrrs rV_prompt_existsClient._prompt_exists;"s !23t(5(rZc"URUSS9$)zLike a prompt. Args: prompt_identifier (str): The identifier of the prompt. Returns: Dict[str, int]: A dictionary with the key 'likes' and the count of likes as the value. Tr-r0rrs rV like_promptClient.like_promptG"s**+<4*HHrZc"URUSS9$)zUnlike a prompt. Args: prompt_identifier (str): The identifier of the prompt. Returns: Dict[str, int]: A dictionary with the key 'likes' and the count of likes as the value. Fr:r;r<s rV unlike_promptClient.unlike_promptS"s**+<5*IIrZrry)rr is_public is_archived sort_fieldsort_directionrXc UUU(aSOUbSOSU(aSOSUUUU(aSOSS.nURSSUS9n [R"S0U R5D6$) aList prompts with pagination. Args: limit (int, default=100): The maximum number of prompts to return. Defaults to 100. offset (int, default=0): The number of prompts to skip. Defaults to 0. is_public (Optional[bool]): Filter prompts by if they are public. is_archived (Optional[bool]): Filter prompts by if they are archived. sort_field (PromptSortField): The field to sort by. Defaults to "updated_at". sort_direction (Literal["desc", "asc"], default="desc"): The order to sort by. Defaults to "desc". query (Optional[str]): Filter prompts by a search query. Returns: ListPromptsResponse: A response object containing the list of prompts. rNfalse)rrrBrCrDrErX match_prefixrr&rrY)rrListPromptsResponserO) rrrrBrCrDrErXrrs rV list_promptsClient.list_prompts_"si:#I4It%06g$,&+F  ,,UIf,M--@ @@rZc[R"U5up#nURSSUSU35n[R"S0UR 5SD6$![R a gf=f)a[Get a specific prompt by its identifier. Args: prompt_identifier (str): The identifier of the prompt. The identifier should be in the format "prompt_name" or "owner/prompt_name". Returns: Optional[Prompt]: The prompt object. Raises: requests.exceptions.HTTPError: If the prompt is not found or another error occurs. rr&rrNrY)rirrrPromptrOrrrrr/rrs rVr6Client.get_prompt"s{!) @ @AR SA 00%+/H$$?x}}v'>? ?..  s>AA10A1rreadmer rBcUR5nU(a'UR(d[R"S5e[R"U5upxn UR US9(dUR SU5eUU=(d SU=(d SU=(d /US.n URSSU S9n U R5 [R"S 0U R5S D6$) aCreate a new prompt. Does not attach prompt object, just creates an empty prompt. Args: prompt_identifier (str): The identifier of the prompt. The identifier should be in the formatof owner/name:hash, name:hash, owner/name, or name description (Optional[str]): A description of the prompt. readme (Optional[str]): A readme for the prompt. tags (Optional[Sequence[str]]): A list of tags for the prompt. is_public (bool): Whether the prompt should be public. Returns: Prompt: The created prompt object. Raises: ValueError: If the current tenant is not the owner. HTTPError: If the server request fails. Cannot create a public prompt without first creating a LangChain Hub handle. You can add a handle by creating a public prompt at: https://smith.langchain.com/prompts)rzcreate a promptr) repo_handlerrQr rBrr&r6rrY) rrrirlrrrrr.rrMrO) rrrrQr rBrrr/rrOrs rV create_promptClient.create_prompt"s8%%' X33--6 !) @ @AR SA,,5,9,,->F F'&,"lJB" < ,,VYT,J!!#  ;8==?6#:;;rZparent_commit_hashr rcURU5(d[R"S5e[U5n[ U[ 5(aSU;a SU;aUnO%SSKJn U"U5n [R"U 5n[R"U5upn U SU 3n US:XdUcURU 5nUUS .nUbX^S 'URS S U 3US 9nUR5SnUSnUSnU(aURU UU5 URU SU35$![a [S5ef=f)aCreate a commit for an existing prompt. Args: prompt_identifier (str): The identifier of the prompt. object (Any): The LangChain object to commit. parent_commit_hash (Optional[str]): The hash of the parent commit. Defaults to latest commit. tags (Optional[str | list[str]]): A single tag or list of tags to apply to the commit. Defaults to None. description (Optional[str]): Optional human-readable description for the commit (max 1000 chars). Defaults to None. Returns: str: The url of the prompt commit. Raises: HTTPError: If the server request fails. ValueError: If the prompt does not exist. z0Prompt does not exist, you must create it first.rrr)rPvThe client.create_commit function requires the langchain-corepackage to run. Install with `pip install langchain-core`rlatest) parent_commitrrrrr6commitrr*)r7rirprep_obj_for_pushrrlangchain_core.loadrPrrOrrr rr*r3)rrrzrXr rprepped manifest_dictrP json_objectrr/rr request_dictr commit_jsonrr%s rV create_commitClient.create_commit"s8""#45511B  $F+ gt $ $TW_#M 5 .K JJ{3M ( @ @AR SA#(';- 8  )-?-G!%!=!=>S!T 0%(   "*5 ',, i 567l- mmoh/ !-0 %   $ $%:It L##'<&=Q{m$LMM= !Q s D((D>)rrQr rBrCcRUR5nU(aUR(d [S5e0nUbX(S'UbX8S'UbXXS'UbXhS'UbXHS'[R"U5upn UR SSU S U 3US 9n U R 5 U R5$) aUpdate a prompt's metadata. To update the content of a prompt, use push_prompt or create_commit instead. Args: prompt_identifier (str): The identifier of the prompt to update. description (Optional[str]): New description for the prompt. readme (Optional[str]): New readme for the prompt. tags (Optional[Sequence[str]]): New list of tags for the prompt. is_public (Optional[bool]): New public status for the prompt. is_archived (Optional[bool]): New archived status for the prompt. Returns: Dict[str, Any]: The updated prompt data as returned by the server. Raises: ValueError: If the prompt_identifier is empty. HTTPError: If the server request fails. rSrrQrBrCr rr&rr6)rrrrirrr.rO) rrrrQr rBrCrrOrr/rrs rV update_promptClient.update_prompt #s:%%' X336 <>  ""-   #N  )   ""-   L ( @ @AR SA,, wugQ{m44-  !!#}}rZc[R"U5up#nURU5(dURSU5eUR SSUSU35nUR 5 g)zDelete a prompt. Args: prompt_identifier (str): The identifier of the prompt to delete. Returns: bool: True if the prompt was successfully deleted, False otherwise. Raises: ValueError: If the current tenant is not the owner of the prompt. zdelete a promptrr&rN)rirrrrr.rNs rV delete_promptClient.delete_promptZ#sj!) @ @AR SA,,U33,,->F F,,Xq 7VW!!#rZc$U(aSOSnUU3$)zGenerate a cache key for a prompt. Args: prompt_identifier: The prompt identifier. include_model: Whether model info is included. Returns: The cache key string. z :with_modelrrY)rrr rs rV_get_cache_keyClient._get_cache_keym#s#0R#$VH--rZc [R"U5up4nURSSUSUSUU(aSOS35n[R"S0X4S.UR 5ED6$)zFetch a prompt directly from the API (no cache). Args: prompt_identifier: The prompt identifier. include_model: Whether to include model information. Returns: The fetched PromptCommit. rrrz?include_model=truerrrrY)rirrr PromptCommitrO)rrr rr/rrs rV_fetch_prompt_from_apiClient._fetch_prompt_from_api|#s+3*J*J + 'K,, E7!K=+,9(rBD && FhmmoF  rZr  skip_cacherc`[UUS9 [URX5nU(d>URb1UR X5nURR Xe5nUbU$U"5nU(d:URb-UR X5nURR XhU5 U$)aPull a prompt object from the LangSmith API. Public prompts referenced by owner/name cross a trust boundary because the prompt manifest may contain serialized LangChain objects and configuration that affect runtime behavior. For example, a prompt can intentionally configure a model with a custom base URL, headers, model name, or other constructor arguments. These are supported features, but they also mean the prompt contents should be treated as executable configuration rather than plain text. Set `dangerously_pull_public_prompt=True` only after reviewing and trusting the prompt contents, not merely the publishing account. Prompts from your own or your organization's account can still be unsafe if that account or prompt was compromised. Args: prompt_identifier: The identifier of the prompt. include_model: Whether to include model information. skip_cache: Whether to skip the prompt cache. Defaults to `False`. dangerously_pull_public_prompt: Set to `True` to allow pulling a public prompt by owner/name, for example `username/promptname`. Defaults to `False`. Returns: The prompt object. Raises: ValueError: If no commits are found for the prompt. )r)rr rsrrnrr) rrr rvr refresh_func cache_keycachedrs rVpull_prompt_commitClient.pull_prompt_commit#sJ % +I   ' '): dkk5++,=MI[[__Y=F! dkk5++,=MI KKOOI| < rZrrr c#|# [R"U5upVnUb [SU5OUUUS.nSn X8S'URSSUSUS3US 9n U R 5n U S n U S n U (dgU H.nUbX:a g[ R "S0XVS .UED6v U S - n M0 U[U 5- nX=:agM7f)aList commits for a given prompt. Args: prompt_identifier (str): The identifier of the prompt in the format 'owner/repo_name'. limit (Optional[int]): The maximum number of commits to return. If None, returns all commits. offset (int, default=0): The number of commits to skip before starting to return results. include_model (bool, default=False): Whether to include the model information in the commit data. Yields: A ListedPromptCommit object for each commit. !!! note This method uses pagination to retrieve commits. It will make multiple API calls if necessary to retrieve all commits or up to the specified limit. Nrr}rrrrrrrr8rqrKrY)rirrrrOrListedPromptCommitr)rrrrr rr/rrrtrvalrcr8its rVlist_prompt_commitsClient.list_prompt_commits#s0!) @ @AR SA).(9Se_u*  %8 00E7!K=21H --/C NELE$ 33 %AbAQ  c%j F/sB:B<)r r rrvrc>URUUUUS9n[UUUUS9$)aPull a prompt and return it as a LangChain `PromptTemplate`. This method requires [`langchain-core`](https://pypi.org/project/langchain-core). Args: prompt_identifier: The identifier of the prompt. include_model: Whether to include the model information in the prompt data. secrets: A map of secrets to use when loading, e.g. `{'OPENAI_API_KEY': 'sk-...'}`. If a secret is not found in the map, it will be loaded from the environment if `secrets_from_env` is `True`. Should only be needed when `include_model=True`. secrets_from_env: Whether to load secrets from the environment. **SECURITY NOTE**: Should only be set to `True` when pulling trusted prompts. skip_cache: Whether to skip the prompt cache. Defaults to `False`. dangerously_pull_public_prompt: Set to `True` to allow pulling a public prompt by owner/name (for example, `username/promptname`). Only do this for trusted prompts. Defaults to `False`. Returns: Any: The prompt object in the specified format. !!! warning "Behavior changed in `langsmith` 0.5.1" Updated to take arguments `secrets` and `secrets_from_env` which default to None and False, respectively. By default secrets needed to initialize a pulled object will no longer be read from environment variables. This is relevant when `include_model=True`. For example, to load an OpenAI model you need to have an OPENAI_API_KEY. Previously this was read from environment variables by default. To do so now you must specify `secrets={"OPENAI_API_KEY": "sk-..."}` or `secrets_from_env=True`. `secrets_from_env` should only be used when pulling trusted prompts. These updates were made to remediate vulnerability [GHSA-c67j-w6g6-q2cm](https://github.com/langchain-ai/langchain/security/advisories/GHSA-c67j-w6g6-q2cm) in the `langchain-core` package which this method (but not the entire langsmith package) depends on. ru)r r r)r{r)rrr r rrvrr s rV pull_promptClient.pull_prompt$sAj// '!+I 0 ( '-   rZr[)rzrXrBrrQr  commit_tagscommit_descriptioncURU5(a-[SXEXg455(aURUUUUUS9 OURUUbUOSUUUS9 UcUR US9$UR UUUUU S9n U $)aPush a prompt to the LangSmith API. Can be used to update prompt metadata or prompt content. If the prompt does not exist, it will be created. If the prompt exists, it will be updated. Args: prompt_identifier (str): The identifier of the prompt. object (Optional[Any]): The LangChain object to push. parent_commit_hash (str): The parent commit hash. Defaults to "latest". is_public (Optional[bool]): Whether the prompt should be public. If None (default), the current visibility status is maintained for existing prompts. For new prompts, None defaults to private. Set to True to make public, or False to make private. description (Optional[str]): A description of the prompt. Defaults to an empty string. readme (Optional[str]): A readme for the prompt. Defaults to an empty string. tags (Optional[Sequence[str]]): A list of tags for the prompt. Defaults to an empty list. commit_tags (Optional[str | list[str]]): A single tag or list of tags for the prompt commit. Defaults to an empty list. commit_description (Optional[str]): Optional human-readable description for the commit (max 1000 chars). Defaults to None. Returns: str: The URL of the prompt. c3(# UHoSLv M g7frrY)rparams rVr%Client.push_prompt..$s/UeT!/UrPF)rBrrQr )rrW)r7rrhrUr3re) rrrzrXrBrrQr rrr-s rV push_promptClient.push_promptS$sX   0 1 109/U""% +!' #   !'0'<)%'   >'':K'L L   1* !  rZrcbURUSUS9n[RRU5$)zPull an agent from Hub. Args: identifier: Repo identifier (owner/name:hash, owner/name, or name). version: Commit hash or tag; overrides any hash in identifier. Returns: AgentContext: The agent snapshot. agentr)_pull_hub_directoryr AgentContextmodel_validater identifierrrs rV pull_agentClient.pull_agent$s2'' GW'M&&55d;;rZ)r\rrQr rBc .URUSUUUUUUS9$)z=Push an agent to Hub, creating the repo if it does not exist.rrr\rrQr rB_push_hub_directoryrrrr\rrQr rBs rV push_agentClient.push_agent$4''  '#(  rZcbURUSUS9n[RRU5$)zPull a skill from Hub.skillr)rr SkillContextrrs rV pull_skillClient.pull_skill$s2'' GW'M&&55d;;rZc .URUSUUUUUUS9$)zPush a skill to Hub.rrrrs rV push_skillClient.push_skill$rrZc&URU5 g)z/Delete an agent and its owned child file repos.N_delete_hub_directoryrrs rV delete_agentClient.delete_agent$ "":.rZc&URU5 g)z.Delete a skill and its owned child file repos.Nrrs rV delete_skillClient.delete_skill$rrZcV[R"U5up#nURX#5$)zCheck if an agent repo exists.riparse_hub_identifier_hub_repo_existsrrrrrs rV agent_existsClient.agent_exists$'!66zBQ$$U11rZcV[R"U5up#nURX#5$)zCheck if a skill repo exists.rrs rV skill_existsClient.skill_exists$rrZrrrBrCrXc *URSUUUUUS9$)zList agents with pagination.rr_list_hub_reposrrrrBrCrXs rV list_agentsClient.list_agents$.## # $  rZc *URSUUUUUS9$)zList skills with pagination.rrrrs rV list_skillsClient.list_skills%rrZc [R"U5upEnUbUO US:waUOSnSU0nU(aXxS'URS[SUSUS3US9n U R 5$) zDFetch hub directory payload, merged with owner/repo from identifier.Nr[ repo_typer]rr /directoriesr)rirrr/rO) rrrrrrr]rrrs rVrClient._pull_hub_directory"%s';;JGV*G6X;MSW #.y!9 %8 ,, nAeWAdV< 8-  }}rZc [U5 [R"U5upn URU 5(dUR SU3U 5eUR X5(a.[ SXVXx455(aURU U UUUUS9 Od[R"U 5(d,[R"SU <S[RS35eURU UUUU[U5S9 0n UR5HupUcSX'MURSS 9X'M! S U 0nUbXOS 'UR!S ["S U S U S3US9nUR%5SSn['UR(XU5$)zFCreate a hub directory commit, creating the repo if it does not exist.zpush c3(# UHoSLv M g7frrY)rr9s rVr-Client._push_hub_directory..L%sQ*PQD=*PrrPzInvalid repo_handle z : must match r?NTrrr\rrz/directories/commitsr6r]r)r2rirrrrr_update_hub_repo_metadatar0matchrlpattern_create_hub_reporrcrrr/rOr1r])rrrrr\rrQr rBrrr request_filesrentryr4rrs rVrClient._push_hub_directory8%s }-!66zBQ,,U33,,uYK-@%H H   - -Q;*PQQQ.. +!' /',,T2211*4(3""5"="=!>aA  ! !'y/ " >@  ;;=KD}&* #&+&6&6D&6&I # ) !(7  $$1 !,, nAeWAdV+? @-  mmoh/ > [IIrZc [R"U5up#nURU5(dURSU5eUR S[ SUSUS35 g)zDelete a hub directory repo.deleterrrN)rirrrrr/rs rVrClient._delete_hub_directoryw%sa!66zBQ,,U33,,Xu= = !! nAeWAdV< 8 rZcUUUU(aSOSS.nUbU(aSOSUS'U(a XgS'SUS'URS[US9n[R"S 0UR 5D6$) uList hub repos filtered by type. Returns ``ListPromptsResponse`` because ``/repos`` is polymorphic — the list shape is shared across prompt, agent, and skill repos. rrG)rrrrCrBrXrHrrrY)rr.rrIrO) rrrrrBrCrXrrs rVrClient._list_hub_repos%sv "%06g "   ,5&7F;  #7O%+F> ",,UC,G--@ @@rZcvURS[SUSU35 g![Ra gf=f)zCheck if a hub repo exists.rrTF)rr.rir)rrrs rVrClient._hub_repo_exists%sC   % %euAeWAdV-D E..  s !88cUUUS.nUbX7S'UbXGS'Ub[U5US'URSSUS9 g![Ra gf=f) z(Create a new hub repo of the given type.)rTrrBNrrQr rr&r6)rrrir)rrrrrQr rBr4s rVrClient._create_hub_repo%sv ""   ""-   #N  :DL   % %fid % C..   s9AAc0nUbX7S'UbXGS'Ub[U5US'UbXgS'U(aURS[SUSU3US9 gg) z=Patch hub repo metadata fields that were explicitly provided.NrrQr rBrrr6)rrr.)rrrrrQr rBr4s rVr Client._update_hub_repo_metadata%ss "  ""-   #N  :DL  )    % %g#awav/FT % R rZcURUS9 SUlUR bUR R 5 gg![a n[RSU5 SnANUSnAff=f)ayManually trigger cleanup of background threads. Drains pending traces via ``flush()`` before stopping the background threads. Pass ``timeout=0`` to skip the drain entirely (e.g. in error paths or signal handlers where blocking on network I/O is unacceptable). Args: timeout: Maximum seconds to wait for pending traces to flush. None (default) waits indefinitely. rz(Error flushing traces during cleanup: %sNT)r/rrQrRrrshutdown)rrrs rVcleanupClient.cleanup%se J JJwJ ' $ ;; " KK " # J NNEq I I JsA A+ A&&A+c  grrYrrr evaluatorssummary_evaluatorsrexperiment_prefixrrnum_repetitionsblocking experimentupload_resultsr s rVevaluateClient.evaluate%s" rZc  grrYrs rVrr%s"(+rZc <SSKJn U"U4UUUUUUUU UU U U U S. UD6$)a6Evaluate a target system on a given dataset. Args: target (Union[TARGET_T, Runnable, EXPERIMENT_T, Tuple[EXPERIMENT_T, EXPERIMENT_T]]): The target system or experiment(s) to evaluate. Can be a function that takes a `dict` and returns a `dict`, a langchain `Runnable`, an existing experiment ID, or a two-tuple of experiment IDs. data (DATA_T): The dataset to evaluate on. Can be a dataset name, a list of examples, or a generator of examples. evaluators (Optional[Union[Sequence[EVALUATOR_T], Sequence[COMPARATIVE_EVALUATOR_T]]]): A list of evaluators to run on each example. The evaluator signature depends on the target type. Default to None. summary_evaluators (Optional[Sequence[SUMMARY_EVALUATOR_T]]): A list of summary evaluators to run on the entire dataset. Should not be specified if comparing two existing experiments. metadata (Optional[dict]): Metadata to attach to the experiment. experiment_prefix (Optional[str]): A prefix to provide for your experiment name. description (Optional[str]): A free-form text description for the experiment. max_concurrency (Optional[int], default=0): The maximum number of concurrent evaluations to run. If `None` then no limit is set. If `0` then no concurrency. blocking (bool, default=True): Whether to block until the evaluation is complete. num_repetitions (int, default=1): The number of times to run the evaluation. Each item in the dataset will be run and evaluated this many times. Defaults to 1. experiment (Optional[EXPERIMENT_T]): An existing experiment to extend. If provided, `experiment_prefix` is ignored. For advanced usage only. Should not be specified if target is an existing experiment or two-tuple fo experiments. upload_results (bool, default=True): Whether to upload the results to LangSmith. error_handling (str, default="log"): How to handle individual run errors. `'log'` will trace the runs with the error message as part of the experiment, `'ignore'` will not count the run as part of the experiment at all. **kwargs (Any): Additional keyword arguments to pass to the evaluator. Returns: ExperimentResults: If target is a function, Runnable, or existing experiment. ComparativeExperimentResults: If target is a two-tuple of existing experiments. Examples: Prepare the dataset: ```python from langsmith import Client client = Client() dataset = client.clone_public_dataset( "https://smith.langchain.com/public/419dcab2-1d66-4b94-8901-0357ead390df/d" ) dataset_name = "Evaluate Examples" ``` Basic usage: ```python def accuracy(outputs: dict, reference_outputs: dict) -> dict: # Row-level evaluator for accuracy. pred = outputs["response"] expected = reference_outputs["answer"] return {"score": expected.lower() == pred.lower()} ``` ```python def precision(outputs: list[dict], reference_outputs: list[dict]) -> dict: # Experiment-level evaluator for precision. # TP / (TP + FP) predictions = [out["response"].lower() for out in outputs] expected = [ref["answer"].lower() for ref in reference_outputs] # yes and no are the only possible answers tp = sum([p == e for p, e in zip(predictions, expected) if p == "yes"]) fp = sum([p == "yes" and e == "no" for p, e in zip(predictions, expected)]) return {"score": tp / (tp + fp)} def predict(inputs: dict) -> dict: # This can be any function or just an API call to your app. return {"response": "Yes"} results = client.evaluate( predict, data=dataset_name, evaluators=[accuracy], summary_evaluators=[precision], experiment_prefix="My Experiment", description="Evaluating the accuracy of a simple prediction model.", metadata={ "my-prompt-version": "abcd-1234", }, ) ``` Evaluating over only a subset of the examples ```python experiment_name = results.experiment_name examples = client.list_examples(dataset_name=dataset_name, limit=5) results = client.evaluate( predict, data=examples, evaluators=[accuracy], summary_evaluators=[precision], experiment_prefix="My Experiment", description="Just testing a subset synchronously.", ) ``` Streaming each prediction to more easily + eagerly debug. ```python results = client.evaluate( predict, data=dataset_name, evaluators=[accuracy], summary_evaluators=[precision], description="I don't even have to block!", blocking=False, ) for i, result in enumerate(results): # doctest: +ELLIPSIS pass ``` View the evaluation results for experiment:... Evaluating a LangChain object: ```python from langchain_core.runnables import chain as as_runnable @as_runnable def nested_predict(inputs): return {"response": "Yes"} @as_runnable def lc_predict(inputs): return nested_predict.invoke(inputs) results = client.evaluate( lc_predict, data=dataset_name, evaluators=[accuracy], description="This time we're evaluating a LangChain object.", summary_evaluators=[precision], ) ``` Comparative evaluation: ```python results = client.evaluate( # The target is a tuple of the experiment IDs to compare target=( "12345678-1234-1234-1234-123456789012", "98765432-1234-1234-1234-123456789012", ), evaluators=[accuracy], summary_evaluators=[precision], ) ``` Evaluate an existing experiment: ```python results = client.evaluate( # The target is the ID of the experiment we are evaluating target="12345678-1234-1234-1234-123456789012", evaluators=[accuracy], summary_evaluators=[precision], ) ``` !!! version-added "Added in `langsmith` 0.2.0" r)r rrrrrrrrclientrrrerror_handling)langsmith.evaluation._runnerr)rrrrrrrrrrrrrrr  evaluate_s rVrr&sQ\ G  !1/#++!))   rZcX# SSKJn U"U4UUUUUUUU UU U U U S. UD6IShvN $N7f)aYEvaluate an async target system on a given dataset. Args: target (Union[ATARGET_T, AsyncIterable[dict], Runnable, str, uuid.UUID, TracerSession]): The target system or experiment(s) to evaluate. Can be an async function that takes a `dict` and returns a `dict`, a langchain `Runnable`, an existing experiment ID, or a two-tuple of experiment IDs. data (Union[DATA_T, AsyncIterable[Example]]): The dataset to evaluate on. Can be a dataset name, a list of examples, an async generator of examples, or an async iterable of examples. evaluators (Optional[Sequence[EVALUATOR_T]]): A list of evaluators to run on each example. summary_evaluators (Optional[Sequence[SUMMARY_EVALUATOR_T]]): A list of summary evaluators to run on the entire dataset. metadata (Optional[dict]): Metadata to attach to the experiment. experiment_prefix (Optional[str]): A prefix to provide for your experiment name. description (Optional[str]): A description of the experiment. max_concurrency (Optional[int], default=0): The maximum number of concurrent evaluations to run. If `None` then no limit is set. If `0` then no concurrency. num_repetitions (int, default=1): The number of times to run the evaluation. Each item in the dataset will be run and evaluated this many times. Defaults to 1. blocking (bool, default=True): Whether to block until the evaluation is complete. experiment (Optional[TracerSession]): An existing experiment to extend. If provided, `experiment_prefix` is ignored. For advanced usage only. upload_results (bool, default=True): Whether to upload the results to LangSmith. error_handling (str, default="log"): How to handle individual run errors. `'log'` will trace the runs with the error message as part of the experiment, `'ignore'` will not count the run as part of the experiment at all. **kwargs (Any): Additional keyword arguments to pass to the evaluator. Returns: An async iterator over the experiment results. Environment: - `LANGSMITH_TEST_CACHE`: If set, API calls will be cached to disk to save time and cost during testing. Recommended to commit the cache files to your repository for faster CI/CD runs. Requires the `'langsmith[vcr]'` package to be installed. Examples: Prepare the dataset: ```python import asyncio from langsmith import Client client = Client() dataset = client.clone_public_dataset( "https://smith.langchain.com/public/419dcab2-1d66-4b94-8901-0357ead390df/d" ) dataset_name = "Evaluate Examples" ``` Basic usage: ```python def accuracy(outputs: dict, reference_outputs: dict) -> dict: # Row-level evaluator for accuracy. pred = outputs["resposen"] expected = reference_outputs["answer"] return {"score": expected.lower() == pred.lower()} def precision(outputs: list[dict], reference_outputs: list[dict]) -> dict: # Experiment-level evaluator for precision. # TP / (TP + FP) predictions = [out["response"].lower() for out in outputs] expected = [ref["answer"].lower() for ref in reference_outputs] # yes and no are the only possible answers tp = sum([p == e for p, e in zip(predictions, expected) if p == "yes"]) fp = sum([p == "yes" and e == "no" for p, e in zip(predictions, expected)]) return {"score": tp / (tp + fp)} async def apredict(inputs: dict) -> dict: # This can be any async function or just an API call to your app. await asyncio.sleep(0.1) return {"response": "Yes"} results = asyncio.run( client.aevaluate( apredict, data=dataset_name, evaluators=[accuracy], summary_evaluators=[precision], experiment_prefix="My Experiment", description="Evaluate the accuracy of the model asynchronously.", metadata={ "my-prompt-version": "abcd-1234", }, ) ) ``` Evaluating over only a subset of the examples using an async generator: ```python async def example_generator(): examples = client.list_examples(dataset_name=dataset_name, limit=5) for example in examples: yield example results = asyncio.run( client.aevaluate( apredict, data=example_generator(), evaluators=[accuracy], summary_evaluators=[precision], experiment_prefix="My Subset Experiment", description="Evaluate a subset of examples asynchronously.", ) ) ``` Streaming each prediction to more easily + eagerly debug. ```python results = asyncio.run( client.aevaluate( apredict, data=dataset_name, evaluators=[accuracy], summary_evaluators=[precision], experiment_prefix="My Streaming Experiment", description="Streaming predictions for debugging.", blocking=False, ) ) async def aenumerate(iterable): async for elem in iterable: print(elem) asyncio.run(aenumerate(results)) ``` Running without concurrency: ```python results = asyncio.run( client.aevaluate( apredict, data=dataset_name, evaluators=[accuracy], summary_evaluators=[precision], experiment_prefix="My Experiment Without Concurrency", description="This was run without concurrency.", max_concurrency=0, ) ) ``` Using Async evaluators: ```python async def helpfulness(outputs: dict) -> dict: # Row-level evaluator for helpfulness. await asyncio.sleep(5) # Replace with your LLM API call return {"score": outputs["output"] == "Yes"} results = asyncio.run( client.aevaluate( apredict, data=dataset_name, evaluators=[helpfulness], summary_evaluators=[precision], experiment_prefix="My Helpful Experiment", description="Applying async evaluators example.", ) ) ``` Evaluate an existing experiment: ```python results = asyncio.run( client.aevaluate( # The target is the ID of the experiment we are evaluating target="419dcab2-1d66-4b94-8901-0357ead390df", evaluators=[accuracy, helpfulness], summary_evaluators=[precision], ) ) ``` !!! version-added "Added in `langsmith` 0.2.0" r) aevaluaterN)langsmith.evaluation._arunnerr)rrrrrrrrrrrrrrr  aevaluate_s rVrClient.aevaluate&s\N J  !1/#++!))     s !*(*c## SnSnU(aXh- OSn U (a [SU 5OSn U/UU UUUS.n URSSUS3S[U 50S 9n U R5n U (dgU Vs/sHn[R "S 0UD6PM nnUv U[ U 5- n[ U 5U :d U(aX:agU[ U 5- nMs snf7f) aPaginate through examples with runs and yield batches. Args: dataset_id: Dataset UUID to fetch examples with runs session_id: Session UUID to filter runs by, same as project_id preview: Whether to return preview data only comparative_experiment_id: Optional comparative experiment UUID filters: Optional filters to apply limit: Maximum total number of results to return Yields: Batches of run results as lists of ExampleWithRuns instances rNr) session_idsrrpreviewrsfiltersrrrrrrY)rrrrOrrCr)rrrrrsrrr results_countr. batch_limitr4rrrexamples_batchs rV_paginate_examples_with_runs#Client._paginate_examples_with_runs's, 38.dI1:#c9-K!+| $"-F" D00ZL. & D(9:1H MMOERWWQVvj88B6BQVNW SZ 'M5zK'Em6L c%j F?0XsA.C0 CACc8^^^^^^ TRXSS9m T (d[SUS35eUUUUU U4SjnT RT RT RT R T R T RT RT RT RT RT RT RT RT RS.n0T R =(d 0ET R"=(d 0En [$R&"U UU"5S9$)aGet results for an experiment, including experiment session aggregated stats and experiment runs for each dataset example. Experiment results may not be available immediately after the experiment is created. Args: name: The experiment name. project_id: Experiment's tracing project id, also called session_id, can be found in the url of the LS experiment page preview: Whether to return lightweight preview data only. When True, fetches inputs_preview/outputs_preview summaries instead of full inputs/outputs from S3 storage. Faster and less bandwidth. comparative_experiment_id: Optional comparative experiment UUID for pairwise comparison experiment results. filters: Optional filters to apply to results limit: Maximum number of results to return Returns: ExperimentResults with: - feedback_stats: Combined feedback statistics including session-level feedback - run_stats: Aggregated run statistics (latency, tokens, cost, etc.) - examples_with_runs: Iterator of ExampleWithRuns Raises: ValueError: If project not found for the given session_id Example: ```python client = Client() results = client.get_experiment_results( project_id="037ae90f-f297-4926-b93c-37d8abf6899f", ) for example_with_runs in results["examples_with_runs"]: print(example_with_runs.model_dump()) # Access aggregated experiment statistics print(f"Total runs: {results['run_stats']['run_count']}") print(f"Total cost: {results['run_stats']['total_cost']}") print(f"P50 latency: {results['run_stats']['latency_p50']}") # Access feedback statistics print(f"Feedback stats: {results['feedback_stats']}") ``` T)rUrNrz&No experiment found with project_id: 'rbc 3># TRTRTRTTTTS9H nUShvN M gN 7f)z2Yield examples with corresponding experiment runs.)rrrrsrrN)rrr)rrsrrrrrs rV _get_examples_with_runs_iteratorGClient.get_experiment_results.._get_examples_with_runs_iteratord(sL::"77"::*C ;!  !s1A> A) run_count latency_p50 latency_p99rhrer`last_run_start_time run_facetsrgrdr_first_token_p50first_token_p99 error_rate)ra run_statsexamples_with_runs)rorr r r rhrer`rrrgrdr_rrrrasession_feedback_statsrr) rrrNrrsrrr rrars ` ```` @rVget_experiment_resultsClient.get_experiment_results+(sd##D$ Ej\QRST T ! !!**".."..#00$22!(!:!:#*#>#>!,,!,,"..&66&66&66!,,4  %%+ --3 ++)?A  rZ) instructionsrmodelopenai_api_keyanthropic_api_keycURUUUS9nURX5nUSUS.SUS.nURSSURS3US 9n [R "U 5 U R 5n [R"S 0U DURUR5URS .D6n [S U R35 U $) ajGenerate Insights over your agent chat histories. !!! note - Only available to Plus and higher tier LangSmith users. - Insights Agent uses user's model API key. The cost of the report grows linearly with the number of chat histories you upload and the size of each history. For more see [insights](https://docs.langchain.com/langsmith/insights). - This method will upload your chat histories as traces to LangSmith. - If you pass in a model API key this will be set as a workspace secret meaning it will be usedin for evaluators and the playground. Args: chat_histories: A list of chat histories. Each chat history should be a list of messages. We recommend formatting these as OpenAI messages with a "role" and "content" key. Max length 1000 items. instructions: Instructions for the Insights agent. Should focus on what your agent does and what types of insights you want to generate. name: Name for the generated Insights report. model: Whether to use OpenAI or Anthropic models. This will impact the cost of generating the Insights Report. openai_api_key: OpenAI API key to use. Only needed if you have not already stored this in LangSmith as a workspace secret. anthropic_api_key: Anthropic API key to use. Only needed if you have not already stored this in LangSmith as a workspace secret. Example: ```python import os from langsmith import Client client = client() chat_histories = [ [ {"role": "user", "content": "how are you"}, {"role": "assistant", "content": "good!"}, ], [ {"role": "user", "content": "do you like art"}, {"role": "assistant", "content": "only Tarkovsky"}, ], ] report = client.generate_insights( chat_histories=chat_histories, name="Conversation Topics", instructions="What are the high-level topics of conversations users are having with the assistant?", openai_api_key=os.environ["OPENAI_API_KEY"], ) # client.poll_insights(report=report) ``` rrrzxThe run.outputs.messages field contains a chat history between the user and the agent. This is all the context you need.)z%How are your agent traces structured?z.What would you like to learn about your agent?rK)r user_context last_n_hoursrrrz /insightsr6rNrZhost_urlzThe Insights Agent is running! This can take up to 30 minutes to complete. Once the report is completed, you'll be able to see results here: rY) _ensure_insights_api_key_ingest_insights_runsrrrirrOrInsightsReportrr]printr^) rchat_historiesrrrrrrrzrrlreports rVgenerate_insightsClient.generate_insights(sD--)/.  ,,^B:tBN ,, j I6V-  ++H5mmo**  zz))+^^    R{{m   rZri)r'rrNraterverbosec U(aU(dU(d [S5eU(dU(aU(a [S5eU(aURnURn[SXT-5n[ U5HnUR SSUSU35n [ R"U 5 U R5n U SS:XaN[R"S0U DUUR5URS .D6n [S U R5 U s $U SS :Xa[S U S 35eU(a[S X-35 [R "U5 M [#S5e)zPoll the status of an Insights report. Args: report: THe InsightsReport. id: The Insights report ID. Should only specify if 'report' is not specified. project_id: The Tracing project ID. Should only specify if 'report' is not specified. 1Must specify ('id' and 'project_id') or 'report'.@Must specify exactly one of ('id' and 'project_id') or 'report'.rKrr /insights/rfsuccessr z1Insights report completed! View the results at %srSzFailed to generate insights: zPolling time: zInsights still pendingrY)rrrNrrrrirrOrr$rr]r%r^rNr TimeoutError) rr'rrNr*rr+ max_triesrtr resp_jsonjobs rV poll_insightsClient.poll_insights(sP$ vPQ QJFR B**J7?+ y!A00JH  / / 9 I"i/ //)"113!^^  GHH 8$/ #@7AS@T!UVVqxj12 JJt -".344rZ)rr'rN include_runscUb*UcUb [S5eURnURnOUbUc [S5eUnUnURSSUSU35n[R "U5 UR 5nU(d*[R"S0UD6n U RXU5 U $[R"UUUS9US'[R"S0UD6n U RXU5 U $) aFetch an Insights report by ID or from a prior report object. Args: id: The Insights report ID (aka clustering job ID). Provide with ``project_id`` if ``report`` is not provided. report: An ``InsightsReport`` object returned by ``generate_insights`` or ``poll_insights``. If provided, ``id`` and ``project_id`` must be omitted. project_id: The tracing project (session) ID associated with the report. Required if ``report`` is not provided. include_runs: Whether to include all runs for the report. Returns: An ``InsightsReportResult`` with job metadata, clusters, summary report, and optionally ``runs``. Raises: ValueError: If the required identifiers are not provided. r.r-rrr/)rrjob_idrrY) rrrNrrirrOrInsightsReportResult_attach_client_fetch_insights_runs) rrr'rNr7r9rresp report_jsonrs rVget_insights_reportClient.get_insights_report#)s 6  ~!7 VYYF**JzZ/ !TUUF#J(( Z |:fX>  ++D1iik 44C{CF  ! !$F ;M(==! F 00?;?d7 rZ)rfpriorityc,SU0nUbX$S'UbX4S'[URS5n[URU5nURR SUUUR UR S9n[R"U5 UR5$)azList issues associated with a tracing project (forge issues board). Retrieves all issues from the forge issues board that are linked to the given tracing project (identified by its session name). Args: project_name (str): The name of the tracing project (session) whose issues you want to list. status (Optional[str]): Filter issues by status (e.g. ``"open"``, ``"resolved"``). If ``None``, issues of all statuses are returned. priority (Optional[str]): Filter issues by priority (e.g. ``"high"``, ``"medium"``, ``"low"``). If ``None``, issues of all priorities are returned. Returns: List[dict]: A list of issue objects. Each dict contains the following keys: - ``id`` (str): Issue UUID. - ``tenant_id`` (str): Workspace/tenant UUID. - ``issue_board_id`` (str): UUID of the issue board this issue belongs to. - ``title`` (str): Issue title. - ``description`` (str): Issue description. - ``priority`` (str): Issue priority. - ``status`` (str): Issue status. - ``category`` (str | None): Optional category label. - ``trace_ids`` (List[str]): Run/trace IDs associated with the issue. - ``github_issue_url`` (str | None): URL of the linked GitHub issue. - ``github_issue_number`` (int | None): Number of the linked GitHub issue. - ``created_at`` (str): ISO-8601 creation timestamp. - ``updated_at`` (str): ISO-8601 last-updated timestamp. - ``resolved_at`` (str | None): ISO-8601 resolution timestamp, or ``None``. Example: ```python from langsmith import Client client = Client() # List all issues for a project issues = client.list_project_issues("my-project") # Filter by status and priority open_high = client.list_project_issues( "my-project", status="open", priority="high" ) for issue in open_high: print(issue["id"], issue["title"]) ``` r]rfrAz forge-issuesr)rrr) r+r"rrLrrrrirrO)rrUrfrArrr,rs rVlist_project_issuesClient.list_project_issues_)sr#1,!?  %8   !): dllN;!$,,5<<''  MMMM (  ++H5}}rZrcURSS5n[R"U5 UR5Vs1sHoUR S5iM nn[ 5nUS;aUR [5 US;aUR [5 URU5=n(a[U;aS$S$US:Xa Un [n OGUS:Xa Un [n O8U(dU(aU=(d Un U(a[O[n O [S5eURS SXS ./S 9n[R"U5 U [:XaS$S$s snf) Nrz/workspaces/current/secretsrD)Nopenai)N anthropicrFrGz1Must specify openai_api_key or anthropic_api_key.r)rDrlr6) rrirrOrrrJ_OPENAI_API_KEY_ANTHROPIC_API_KEY intersectionr) rrrrrsworkspace_keys target_keys existing_keysrRapi_vars rVr"Client._ensure_insights_api_key)s6,,U4QR++H508 @1%%,@e $ $ OOO , ' ' OO. /*77 D D= D.-?8 P[ P h $G%G k !'G(G 0$9(9G)7o=OGPQ Q,,  )!45-  ++H5"o5xF;F5AsE c[U5S:a[R"S5 USSn[RR [R R 5nURU=(d0 S[RR 5RS5-5n[[U55Vs/sH!n[[R"55PM# nn[Xa5VVs/sHTupxSUSS0SU0UUURS5[U53U[R"SS9- US URS S . PMV n nnUR!U S 9 UR#5 U$s snfs snnf) NrzDCan only generate insights over 1000 data. Truncating to first 1000.z insights z%Y-%m-%d %H:%M:%SryrKz%Y%m%dT%H%M%S%fZ)rr0r}) rrrrHrmrrrrr)r)rrorpr(rTr)r*rstrftimerrrrFrrrrr/) rrrrTrrr[r rrs rVr#Client._ingest_insights_runs)sh t9t  MMV ;D##H$5$5$9$9:%%  Uh//335>>?RSS /4CI.>?.>3tzz|$.>?!/ 0 &q!u-&?"#&<<0B#C"DS[M R!H$6$6q$AA#%jj 0   d+ %@ s (E78AE<),rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrfrRr"rrrrLrrrr)8r"rrRrrzOptional[Retry]rz%Optional[Union[int, tuple[int, int]]]rrrLzOptional[requests.Session]rrrz Optional[Callable[[dict], dict]]r-Optional[Union[Callable[[dict], dict], bool]]rrTrrTrrrz4Optional[Callable[[Sequence[dict]], Sequence[dict]]]r Optional[int]rOptional[float]rz/Optional[Union[dict, ls_schemas.LangSmithInfo]]rOptional[dict[str, str]]rzOptional[TracerProvider]rsOptional[TracingMode]raOptional[bool]r\rVrrrrUrrWrz%Optional[Callable[[Exception], None]]rrrz"Optional[Union[bool, PromptCache]]rr)r3zCallable[[], bytes]rrrr) rTrr4rr/rrrr0rUrr)rr)rSrrr)rr)r|rrlrrr)rr)rlrrr)rlrWrr)rzls_schemas.LangSmithInfo)rzls_schemas.LangSmithSettings)rrUrr)rz0Literal['GET', 'POST', 'PUT', 'PATCH', 'DELETE']rrrzOptional[Mapping]rrDr'Optional[Sequence[type[BaseException]]]rrZrz1Optional[Callable[[requests.Response, int], Any]]rrr rrzrequests.Response)rrrOptional[dict]rIterator[dict]) rrr4r[rzLiteral['GET', 'POST']rrrr\)r pd.DataFramerrr Sequence[str]rr^rrrOptional[ls_schemas.DataType]rls_schemas.Dataset)rz?Union[str, tuple[str, io.BytesIO], tuple[str, io.BytesIO, str]]rr^rr^rrrrrr_rr`)FF)r+z3Union[ls_schemas.Run, dict, ls_schemas.RunLikeDict]rrrrrr)rzSequence[dict]rrrr)rz=Iterable[Union[dict, ls_schemas.Run, ls_schemas.RunLikeDict]]r@rrr)r TracingMode)rRr%rr)rrrdict[str, Any]r RUN_TYPE_TrUrrVrrWrrRrr"rr rrr)r,rrRrr"rrYrrZrr[rr\rrr)r,rrRrr"rrYrrZrr[rr\r)rr)rr)rrrr)rOptional[Sequence[dict]]rzOptional[list[dict]]rr)rzlist[SerializedRunOperation]r"rrRrrYrrZrr[rr\rrrr)rGOptional[Sequence[Union[ls_schemas.Run, ls_schemas.RunLikeDict, dict]]]rrgrr)r4rrrr"rrRrrYrrZrr[rr\r)rz@list[Union[SerializedRunOperation, SerializedFeedbackOperation]]r"rrRrrYrrZrr[rr\rrr)rrgrrgrWrrr)rr4rrDr"rrRrrYrrZrr[rr\r)rz io.BytesIOrzOptional[tuple[int, int]]rrD),r ID_TYPErrrzOptional[RUN_TYPE_T]rOptional[datetime.datetime]rrirSrrr[rr[rrerr[r Optional[list[str]]r_ Optional[ls_schemas.Attachments]rWrr str | uuid.UUID | NonerRrr"rrYrrZrr[rr\rr rrr)rrrRrr"rrYrrZrr[rr\r)rrrRrr"rrYrrZrr[rr\rrr)r\N)rrDrrVrr)rrVrr)r+ls_schemas.Runrrmr)r rhrIrrrm)rrrN+Optional[Union[ID_TYPE, Sequence[ID_TYPE]]]rU#Optional[Union[str, Sequence[str]]]r2rrrUrOOptional[Sequence[str]]rPrrQzLiteral['asc', 'desc']r rrIterator[ls_schemas.Run])$rNrnrUrorrrHOptional[ID_TYPE]r rrrXrrPrrYrrZrr2rYr rrrrirSrYr[Optional[Sequence[ID_TYPE]]rOrprrUr rrrq)rNrrrUrrrUrrDrPrrrirzlist[ListThreadsItem])"rOptional[list[ID_TYPE]]r}rrrkrrrrrrjrqrtrrtrrrrrSrYrXrrPrrYrrZrr2rYrrrrc)r+zls_schemas.RunBaserUrrNrrrr)r rhrrrrr)r rhrr)r rhrr)r rhrr)rUnion[ID_TYPE, str]r rrrrm)rrur[rjrrq)rrrrrrzls_schemas.DatasetShareSchema)rrhrr)rrrr`)rrrrtrrUrIterator[ls_schemas.Example]) rrrqrtrrrrrrUr(Iterator[ls_schemas.TracerSessionResult])rUrrrrr[rrrr[rrrrls_schemas.TracerSession)rNrhrrrrrr[rr[rrirrx)rOptional[uuid.UUID])r uuid.UUID)rNrrUrrrrzls_schemas.TracerSessionResult)rUrrNrrr)rNrrrUrrr]) NNNNNNNNNN)rqrtrrrrrrrr!rrrYrrYr rrrUrOptional[dict[str, Any]]rrw)rUrrNrrr)rrrrrls_schemas.DataTyper'r{r(r{r)z0Optional[list[ls_schemas.DatasetTransformation]]rr[rr`)rrrrrrr)rrrrrrr`) rrrrrr<Union[str, datetime.datetime]r=r}rzls_schemas.DatasetDiffInfo)rrrrrrr)rHrtrrrrrIrrr{rrUrzIterator[ls_schemas.Dataset])rrrrrrr) rrrrrrSzdatetime.datetimerTrrr) rrrrrrXrrrUrz#Iterator[ls_schemas.DatasetVersion]) rrrrrrSrirTrrzls_schemas.DatasetVersion)rfrr`rrrrr`)rrhrr|)NNNN) rrrrrrrrrrrprirls_schemas.Example) ryz:list[Union[Mapping[str, Any], ls_schemas.BaseMessageLike]]rzz>Optional[Union[Mapping[str, Any], ls_schemas.BaseMessageLike]]rrrrrrprirr~r) r+rmrrrrrrprirr~)rzwUnion[list[ls_schemas.ExampleCreate] | list[ls_schemas.ExampleUpsertWithAttachments] | list[ls_schemas.ExampleUpdate],]rrrWrrz/tuple[Any, bytes, dict[str, io.BufferedReader]])rrhrz(Optional[list[ls_schemas.ExampleUpdate]]rWrr!ls_schemas.UpsertExamplesResponse)rrhrz(Optional[list[ls_schemas.ExampleCreate]]rWrrr)rzls_schemas.ExampleCreaterrD)r)rzlist[ls_schemas.ExampleCreate]rrDrz$list[list[ls_schemas.ExampleCreate]])rz7Optional[list[ls_schemas.ExampleUpsertWithAttachments]]rWrrr)rrrrrrz3Optional[Sequence[ls_schemas.ExampleCreate | dict]]rWrrz!Annotated[int, Field(ge=1, le=3)]r rrz2ls_schemas.UpsertExamplesResponse | dict[str, Any]) NNNNNNNNNFNN)rOptional[Mapping[str, Any]]rrrrrrprirrrrrOptional[str | list[str]]rrrrrrrrrrjr_rkrr~)rrhrSrirr~)NNNNNT)rrrrrrrsrSz'Optional[Union[datetime.datetime, str]]rrpr rrrDrrUrr[rPrrrr rrrv)rrhrr{rrrr[rrrrrrz*Optional[ls_schemas.AttachmentsOperations]r_rkrrc) r str | NonerzID_TYPE | Nonerz3Optional[Sequence[ls_schemas.ExampleUpdate | dict]]rWrr rrrc)rrhrr)rSequence[ID_TYPE]r'rrr)rrrrrrSz'Optional[Union[str, datetime.datetime]]rz list[str]) rrrrrr5rr list[ID_TYPE]rJrrr)r+9Union[ls_schemas.Run, ls_schemas.RunBase, str, uuid.UUID]rIrrrm)rz5Union[ls_schemas.Example, str, uuid.UUID, dict, None]r+rmrzOptional[ls_schemas.Example])rJUnion[ls_evaluator.EvaluationResult, ls_evaluator.EvaluationResults, dict]rBrr#list[ls_evaluator.EvaluationResult]) r+rrls_evaluator.RunEvaluatorrUr{rj9Optional[Union[ls_schemas.Example, str, dict, uuid.UUID]]rIrrrP) r\rr+zOptional[ls_schemas.Run]rUr{rNrrr^zOptional[cf.ThreadPoolExecutor]rr) r+z%Union[ls_schemas.Run, str, uuid.UUID]rrrUr{rjrrIrrrP)Nunnamed),r rrrDrrXUnion[float, int, bool, None]rl(Union[float, int, bool, str, dict, None]rHrrrdUnion[dict, None]rFUnion[str, None]rUr{rfz)Union[ls_schemas.FeedbackSourceType, str]rrrrrrrre#Optional[ls_schemas.FeedbackConfig]rrDrNrrrsrrrtrrrr[rSrYrrrrrir rrls_schemas.Feedback) rrrhrXrrlrrdrrFrrr)rrrhrr) r[rsrrprfz1Optional[Sequence[ls_schemas.FeedbackSourceType]]rrUr rrzIterator[ls_schemas.Feedback])rrrhrr)rfUnion[str, uuid.UUID]rXrrlrrdrrFrrr[rr) r rhrrr0Optional[datetime.datetime | datetime.timedelta]rerrrrrrzls_schemas.FeedbackIngestToken) r rhrr^rrrz7Optional[Sequence[Optional[ls_schemas.FeedbackConfig]]]rz(Sequence[ls_schemas.FeedbackIngestToken])r rhrrUrz(Iterator[ls_schemas.FeedbackIngestToken]) rrrrrrrrUrrDrz$Iterator[ls_schemas.FeedbackFormula])rrhrls_schemas.FeedbackFormula) rrrLiteral['sum', 'avg']rASequence[Union[ls_schemas.FeedbackFormulaWeightedVariable, dict]]rrrrrrrr) rrhrrrrrrrr)rrhrr)rrrezls_schemas.FeedbackConfigrrYrls_schemas.FeedbackConfigSchema) rrprrrrUrrDrz)Iterator[ls_schemas.FeedbackConfigSchema])rrrerrrYrr)rrrr) rrtrrrrrrUrz$Iterator[ls_schemas.AnnotationQueue]) rrrrrrrrrr4Optional[list[ls_schemas.AnnotationQueueRubricItem]]rz%ls_schemas.AnnotationQueueWithDetails)rrhrzls_schemas.AnnotationQueue) rrhrrrrrrrrrr)rrhrr)rrhr[rrr)rrhr rhrr)rrhrrDrz%ls_schemas.RunWithAnnotationQueueInfo)rrrrrrrrrrprirr{rrrrz ls_schemas.ComparativeExperiment)rrrr)rrrrrzls_utils.LangSmithUserError)rKr)rrrrDrrDrr)rrr%rr zUnion[str, list[str]]rr)rrr-rrdict[str, int])rrrr)rrrr)rrrr)rrDrrDrBrYrCrYrDzls_schemas.PromptSortFieldrEzLiteral['desc', 'asc']rXrrls_schemas.ListPromptsResponse)rrrzOptional[ls_schemas.Prompt]) rrrrrQrr rprBrrzls_schemas.Prompt) rrrzrrXrr rrrrr)rrrrrQrr rprBrYrCrYrrc)rrrr)rrr rYrr)rrr rYrls_schemas.PromptCommit) rrr rYrvrrrrr) rrrrUrrDr rrz'Iterator[ls_schemas.ListedPromptCommit])rrr  bool | Noner dict[str, str] | Nonerrrvrrrrr)rrrzz Optional[Any]rXrrBrYrrrQrr rprrrrrr)rrrrrzls_schemas.AgentContext)rrrz%dict[str, Optional[ls_schemas.Entry]]r\rrrrQrr rprBrYrr)rrrrrzls_schemas.SkillContext)rrrr)rrrr) rrDrrDrBrYrCrYrXrrr)rrrLiteral['agent', 'skill']rrrrc)rrrrrrcr\rrrrQrr rprBrYrr)rrrrDrrDrBrYrCrYrXrrr)rrrrrr)rrrrrrrQrr rprBrrr)rrrrrrrQrr rprBrYrr) NNNNNNrrKTNT)rOptional[DATA_T]rzOptional[Sequence[EVALUATOR_T]]r'Optional[Sequence[SUMMARY_EVALUATOR_T]]rr[rrrrrrUrrDrrrOptional[EXPERIMENT_T]rrrz'Union[TARGET_T, Runnable, EXPERIMENT_T]r rrr)rrrz+Optional[Sequence[COMPARATIVE_EVALUATOR_T]]rrrr[rrrrrrUrrDrrrrrrrz(Union[tuple[EXPERIMENT_T, EXPERIMENT_T]]r rrr) NNNNNNrrKTNTrs)rrrzIOptional[Union[Sequence[EVALUATOR_T], Sequence[COMPARATIVE_EVALUATOR_T]]]rrrr[rrrrrrUrrDrrrrrrrLiteral['log', 'ignore']rzJUnion[TARGET_T, Runnable, EXPERIMENT_T, tuple[EXPERIMENT_T, EXPERIMENT_T]]r rrz6Union[ExperimentResults, ComparativeExperimentResults])rzNUnion[DATA_T, AsyncIterable[schemas.Example], Iterable[schemas.Example], None]rz4Optional[Sequence[Union[EVALUATOR_T, AEVALUATOR_T]]]rrrr[rrrrrrUrrDrrrz6Optional[Union[schemas.TracerSession, str, uuid.UUID]]rrrrrzVUnion[ATARGET_T, AsyncIterable[dict], Runnable, str, uuid.UUID, schemas.TracerSession]r rrr)FNNN)rrhrrzrrrsryr!dict[uuid.UUID, list[str]] | NonerrUrzIterator[list[ExampleWithRuns]])NNFNNN)rrrNryrrrsryrrrrUrzls_schemas.ExperimentResults)r&zlist[list[dict]]rrrrr%Literal['openai', 'anthropic'] | Nonerrrrrls_schemas.InsightsReport)r' ls_schemas.InsightsReport | NonerrlrNrlr*rDrrDr+rrr) rrlr'rrNrlr7rrzls_schemas.InsightsReportResult)rUrrfrrArrz list[dict])rrrrrrrzLiteral['openai', 'anthropic'])rrrr)rrrrr __slots__rrr5 staticmethodr2r_rdrjpropertyrmr]rwr}rRsetterrrrrrrrrrrcrdrrr-r:r=rLrsrSrjrhryr$r%r'r&rfrgrrrrrrrrrrr&r/rArJrVr9rrrrrrrrrrrrrrrrrrrrixor_argsrorrr"r%r1r5rrArFrKrNrVr[r^rirrsr~rrrrrrrrrrerrrqrrrr"r%r-r2r7r<r@rSrYrZrprjr{rarrrrrrrrrrrrrrrrrrrrrrrrr rrrr r*r0r3r7r=r@PromptSortField updated_atrJr6rUrerhrkrnrsr{rrrrrrrrrrrrrrrrrrrrrrrrrrr'DEFAULT_INSTRUCTIONSr(r5r?rCr"r#rrYrZrVrrs08/Ib##!!"&^>"&(,<@!%.2#'7;EIFJGK). -159@D-19=.2'+15&*.2,0HL%*48?^>^> ^> & ^> : ^>^>,^>!^>5^>C^>D^>E^>#'^># ^>$+%^>&$3'^>(>)^>*++^>,7-^>.,/^>0%1^>2 /3^>4$5^>6,7^>8*9^>:!F;^><#=^>>2?^>@ A^>@$  4$( AAA A  A ! A AAFX 3((BB F  ^^;;""@@$$ ^^HH@ .-1"#<@=AMQj Aj  j * j  j :j ;j Kj j j  j Z6:!!$2! !H $17 '0'0 '0 / '0  '0 '0`&*3=3F3F3I3I: : : " : # : #: 1:  : D#%)3=3F3F3I3I_ Q_ "_ # _  _ #_ 1_  _ H 5 @55 5  5n : ,K, ,  ,\"" '+%)-2!%!%zzz z $ z#z'+zzzz z@"&!%%)#''+ $pp p  p # p!p%pp pl"&!%%)#''+ $E E  E  E # E !E %E E N)+- (   ,(D>"&!%%)#''+ $_ )_ _  _ # _!_%__ _J  L3 L3  L3 L3f"&!%%)#''+ $D7D7 D7  D7  D7#D7!D7%D7D7T"&!%%)#''+ $8? M8? 8?  8? # 8?!8?%8?8? 8?|  f3.3f3 f3  f3'+f3 f3X!%!%%)#''+ $j %j j  j  j#j!j%jjb VV!:V  Vx#)-2604#!%"&+/ $$(8<-27;!%!%%)#''+ $-jj j ' j 0 j.jjj j)jj"j6j'+j 5!j"#j$%j&#'j(!)j*%+j,-j./j0 1j`"&!%%)#''+ $dd d  d # d!d%ddT"&!%%)#''+ $55 5  5 # 5!5%55 5p=A7272*972 72r<)-&* :6&:6$ :6 ! :6  :6 :6~)-&* $# !&!$ !  !  ! -!L)-&*-1! /<&/<$ /< + /<  /< #/[ $[ & [ F [ '+ [ ;[ [  <[ 7[ z  D(T56/3(,&*26/304+/(,+/"':>8<j+j&j$ j 0 j - j.j)j&j)j j%8j6j j7jZLP! !! -H! ! J)-&*379=*.#{##' $$){%{${1 { 7 { ( {{{{!{{"{{ &{B,0/3#'+/(,MQ8<U#U#) U# - U# ! U#)U#&U#!KU#6U# U#t$(%)GK-2 W#!W## W# E W# '+ W#W# W#r6$FK/6,/6>B/6 /6h)-&*9= #&#$ # 7 #  #P)-&*'6&'6$ '6  '6 # '6'6 '6R F  2"F"" & "L"& % %  % -%X15  %* F*-* . *  ** '*b)-04(,26:2 2 & 2 . 2&232 -2r15  %* 2*-* . *  ** '*^%) H 04:>&*(,$(04  ) ) - -+/)-?C"$(,7;/3 $ $(,263H"H H - H8H$H&H"H.H H)H '!H"=#H$ %H&&'H($5)H*-+H,-H./H0&1H203H45H6 7H\04:>(,$()6)6- )6 8 )6 & )6")6 )6V6&0404RV# )-). ) P )  )) ')V6(0406 ;?(,$(#'06+06-06 8 06 & 06"06!06 06nHL?C)-?A?A?A E ?A = ?A'?A (?ALHL dd%d E d  d 2dT $   2 :)-(,# &&&& &  &  & .&P=#*= #=6)-(,5=5=0 5=  5=&5=&5= $5=n,=$,= ,= 0 ,=  ,= $,=\ 6(16 9B9B3 9B . 9B ) 9B|15'+# -.-% -  -  - 3-f@D04 2B2B= 2B . 2B ) 2Bh6<.2"'+# (+( ( % (  ( .(\&*&*-1MQ, , # , $ , + , K,  /, \H,#%)-1MQ&6&6 &6 # &6 + &6K&6 &6P6 66-:6 6(66,36 6&HH+.H .H<04%)26-1 $9>9>'9> - 9> # 9>09>+9> 9> *9>v ?  "% $ IJ>%(>14>BE> >*  %(  58  @U    D!$,0 . . ) I J$(&+1;1K1K1V1V17#*A*A *A " *A $ *A/*A/*A*A (*AX8&* $(,3<3<# 3<  3< & 3<3< 3>& >  > )- > !>H $# 77 7  7  7 17z&+)-!& /4@ @ # @ ' @  @ @ )-@  @ L!%"*$(%) $(,15,0KK K  K " K#KK&K/K*K Kb"& << < ! <.(,%) $(,$(  5  %  #   & "   6"& << < ! <(,%) $(,$(  5  %  #   & "   .//2 2$(&+#    "  $    ( ,$(&+#    "  $    ( &-    ,=J=J-=J  =J % =J#=J=J&=J"=J =J~ A,A A  A " A$AA (A:- #   & 6SSS # S  S&S"S S.##( "&6:FJ#'+/%))* -1#   4  D  ! ) # '   +  7  !  $ "&BFFJ#'+/%))* -1#+ + @ + D +!+)+#+'++++++8++ &!++0"& FJ#'+/%))* -1#38'b  b  b Db !b )b #b 'b b !b "+#b $%b &1'b  b ()b * @+b b KOFJ#'+/%))* MQ#381y  y Iy Dy !!y ")#y $#%y &''y ()y *+y ,K-y ./y 011y  y 23y 4 5y ~9=59#8!8!8! 8! $7 8! 3 8!8! )8!x#*.9=59#] ] (]  ] $7 ] 3 ] ]  &] ~ 17;%)(,`)` `  ` 5 `#`&` #``D48%)-135135 # 35 + 35  353535 #3535j&*37-1! 9 #91 9 + 9  9 )99~!%"& HH H  H  HZ&*(,7; #G##G& #G 5 #G ( #GJrZrc SSKJn U"5nU=(d 0nURSS5nUR"U4SU0UD6$![a [S5ef=f![ an[ R"SU35eSnAff=f)aConvert a prompt to OpenAI format. Requires the `langchain_openai` package to be installed. Args: messages (Any): The messages to convert. model_kwargs (Optional[Dict[str, Any]]): Model configuration arguments including `stop` and any other required arguments. Returns: The prompt in OpenAI format. Raises: ImportError: If the `langchain_openai` package is not installed. ls_utils.LangSmithError: If there is an error during the conversion process. r) ChatOpenAIzThe convert_prompt_to_openai_format function requires the langchain_openaipackage to run. Install with `pip install langchain_openai`stopNz#Error converting to OpenAI format: )langchain_openairrr_get_request_payloadrrir)ry model_kwargsrrFrrs rVconvert_prompt_to_openai_formatr)s( /\F%2L   FD )DQ**8O$O,OO   K    Q%%(KA3&OPPQs"AAA B%A>>BcPSSKJn U=(d 0nURSS5nURSS5nURSS5nU"S X5US .UD6nUR XS 9$![a [S5ef=f![ an[ R"S U35eSnAff=f) a;Convert a prompt to Anthropic format. Requires the `langchain_anthropic` package to be installed. Args: messages (Any): The messages to convert. model_kwargs (Optional[Dict[str, Any]]): Model configuration arguments including `model_name` and `stop`. Returns: The prompt in Anthropic format. r) ChatAnthropiczThe convert_prompt_to_anthropic_format function requires the langchain_anthropic package to run. Install with `pip install langchain_anthropic` model_namezclaude-3-haiku-20240307rNr)rrr)rz&Error converting to Anthropic format: rY)langchain_anthropicrrrrrrir)ryrrrrrrGrs rV"convert_prompt_to_anthropic_formatr*s  5 %2L!!,0IJJ   FD )Dy$/GT=IIT--h-BB#   =   $ T%%(Nqc&RSSTs#A$A=$A:= B%B  B%c>^\rSrSrSrSU4SjjrSSSjjrSrU=r$) _FailedAttachmentReaderi=*zHBytesIO that raises an error when read, for failed attachment downloads.c.>[TU]5 Xlgr)rr_error)rrSrs rVr _FailedAttachmentReader.__init__@*s  rZc^[R"SUR35URe)NzFailed to download attachment: )rirr)rrs rVr_FailedAttachmentReader.readD*s*%%-dkk] ; {{ rZ)r)rSr)r )rrUrr) rrrrrrrrrrs@rVrr=*sRrZrrhc $0nX;aX(aXR5HupEURS5(dMUb[X%S5nOUSn[R"USS9nUR 5 [ R"UR5n[S 0USUUR S5S .D6X4RS5'M U$![a0n [RSUSU 35 [U 5nSn A NmSn A ff=f) zOConvert attachments from the backend database format to the user facing format.z attachment.N presigned_urlT)rzError downloading attachment rr)rreaderrrY)rc startswithrrrr.rrcontentrrQrRrrB removeprefix) rrFr"attachments_dictrDrlr,rrrs rVrGrGJ*s4#8/557JC>>-00")'3IJ 1 4#<<>))+H$4$45AOA%*?%;$!&;!7A --m< =8*  4!>se2aSIJ03 4sAC D&D  DcUHnUR5 M g![a$ [RSUR5 MFf=f)z2Close all opened files used in multipart requests.zCould not close file: %sN)rKrrQrr)rr s rVrwrwh*sA  JJL  LL3TYY ?  s*A  A cdURS5RS5(aSUS3$SUS3$)Nrrz/platform/datasets/rz/v1/platform/datasets/rer,)r"rs rVrrr*s:~~c##E**$ZL ::' |9==rZc`URS5RS5(aSU3$SU3$)z=Construct a platform API path based on the API URL structure.rrz /platform/z /v1/platform/r)r"rs rVr+r+y*s6~~c##E**D6""tf%%rZc@URS5(aU$URS5(aSnU[S5SnO6URS5(aSnU[S5SnO[SU<35eURS5R S5nUR S5R S5nU(d[SU<SU<35eU(dU$USS :XaUS S :XaUSS nOUS SS S /:XaUSS nX4-nUSR S U55-$)NhttprrzBapi_url must start with 'http://' or 'https://'. Received api_url=rzPMust specify non-empty api_url or pathname must be a full url. Received api_url=z , pathname=rapir rv1c36# UHo(dM Uv M g7frrY)rps rVr!_construct_url..*s1eq11es  )rrrrerlstriprm)r"rr api_partsr&rs rVrr*sB6""*%%#j/+,   I & &#i.*+Q S  s#))#.I%++C0J  !zH; 0   !} R=E !!#2I rs^t} ,!#2I  "E #((1e11 11rZrc[US5(aURUS9$[US5(aURUS9$[S5e)z)Dump model depending on pydantic version.rrrzUnsupported model type)rrrr)rrs rVrr*sMul##\::   zz|z44011rZcSSKJn SSKJn SSKJnJn Un[X5(Ga8[URU5(Ga[URSU5(aS[UR5s=::aS::aO U$URnURSnURnURn [Xb5(d3S UR;a#URS n U"S S UR0U D6n[Xb5(aTXh-RSRn U R5V V s0sHupX;dM X_M sn n Ul U"Xg5nU$![a [S5ef=fs sn n f) z/Format the object so its Prompt Hub compatible.r)ChatPromptTemplaterrrZrKrr\ls_structured_output_formatryrY)rrrrlangchain_core.runnablesrrrrrrrrr ryrc)objrrrr chain_to_pushr bound_modelrr output_formatstructured_kwargsr8r9s rVr^r^*sk =FNM3)) syy"4 5 5 syy|_ 5 5 SYY $1 $6 3iil !!")) 644-1C1CC'../LMM%PvP-PF f / /!' 6 6q 9 @ @ ".!3!3!5"!59S!5"K  -VAM Q   I   B"sE#; E< E<#E9c0UEn[X$U/5nU(a [US5nUb X['OU(d [X5nUbX S'UbX0S'UbX@S'UbXPS'U$)Nz X-Service-Keyru AuthorizationCookie)r_apply_optional_api_keyrv)rrRrYrZr[r\rhas_non_api_keys rVrr*skG;v>?O)'48$  )'D#. !*  #0  " NrZcU(a X['U$UR[S5 UR[R5S5 U$r)rvrupper)rrRs rVrr*s@$  N  It$ IOO%t, NrZ)rSrrrrf)rsrXrarYrrbra) rz dict | listrrDrrDrrDrr)rrrrrr) r rr rr rrrrr)rr/) r!rr"rr#rDr$rrztuple[str, str])r-rrr)rr )rLzrequests.Sessionrr)r"rrRrrsrbrr)rXrr)r\rVrz float | None)rfrWrr)rlrhrrrrz)rlOptional[Union[str, uuid.UUID]]rrz)rlrrprrry)rlrrpr)ryrrr{rr)rrrFrr"rrzdict[str, AttachmentInfo])rzlist[io.BufferedReader]rr)r"rrrhrr)r"rrrrr)r"rrrrr)rrrrc)rrrr)rzMapping[str, str]rRrrYrrZrr[rr\rrrrr)rrrRrrr)r __future__rr rCrrr$r rr(rrBimportlib.metadatarrrOloggingrrr rNrtypingrrorcollections.abcrrrrrr r inspectr pathlibr rr rrrrrrrrrrrurllibrrrrpydanticrrrrequests_toolbeltrrtyping_extensionsrrurllib3.poolmanagerr urllib3.utilr rFr!r1r"rr#rilangsmith._internalr$rr%r&r#langsmith._internal._beta_decoratorr'&langsmith._internal._compressed_tracesr(langsmith._internal._constantsr)r*r+r,r-langsmith._internal._hubr.r/r0r1r2langsmith._internal._multipartr3r4r5langsmith._internal._operationsr6r7r8r9r:r;r<r=langsmith._internal._serder>rlangsmith._internal._uuidr?langsmith.prompt_cacher@rAlangsmith.schemasrBrC getLoggerrrQrPrMrOr rLrb frozensetrIrrWr[rrHrIrzrzoneinforropentelemetry.sdk.tracerr rrrrr}rrrrQrrJrrrrrrrrrrrrrrrvrrzrC parametersrrrrrrrrr(r.rrrhrdrIrMrSrYr_rjrrnrv HTTPAdapterrxrrrrrrrGrwrr+rrr^rrrYrZrVr s #      PP(%   )22'#+'':C   A+F=   8 $!"nn&34 '0+1F'GnG$+ "( $(B'B!B BJ $(!.61!5H?      8 $##$<=    , '0B0M0MMS#&5( 1%%4,-rTV!!%(!9<!NQ! !H    ?C     FFF# F  F  FX '     :   3  H  Q3 3>!,      :  .2*. $ JJCG *<@ QV Q  %-99%P"i"c[c[Pw.2$Q$Q*$Q $QR.2%T%T*%T %TP bjj CG $'2?<>&2D/42.b     !  $: &3MP((((...."   s6P&P=- Q&P:9P:=QQ Q"!Q"