Mi&SSKJr SSKJrJrJrJr SSKrSSK J r J r SSK J r SSKJrJr SSKJr \(aSSKJr 0S S _S S _S S_SS_SS_SS_SS_SS_SS_SS_SS_SS_SS_SS_SS_SS_SS_rSr"SS \5r\"S!S"\55rg)#) dataclass) TYPE_CHECKINGIterableListOptionalN)Field SecretStr)Element)BaseEmbeddingEncoderEmbeddingConfig)requires_dependenciesClientzvoyage-context-3i}zvoyage-3.5-litei@Bz voyage-3.5izvoyage-2z voyage-02zvoyage-3-largez voyage-code-3zvoyage-large-2-instructzvoyage-finance-2zvoyage-multilingual-2z voyage-law-2zvoyage-large-2zvoyage-3z voyage-3-litez voyage-code-2zvoyage-3-m-expzvoyage-multimodal-3ic\rSrSr%\\S'\\S'Sr\\S'\ "SS9r \ \ \S'\ "SS9r \ \\S '\ "SS9r\ \ \S '\"S /S S 9SSj5rS\ 4SjrSrg)VoyageAIEmbeddingConfig'api_key model_nameFshow_progress_barN)default batch_size truncationoutput_dimensionvoyageaizembed-voyageai)extrasreturncJSSKJn U"URR5S9$)z3Creates a VoyageAI python client to embed elements.rr)r)rrrget_secret_value)selfrs u/var/www/eduai.edurigo.com/storigo/production/storigo_env/lib/python3.13/site-packages/unstructured/embed/voyageai.py get_client"VoyageAIEmbeddingConfig.get_client/s$ $LL113  cB[RURS5$)z*Get the token limit for the current model.r)VOYAGE_TOTAL_TOKEN_LIMITSgetrr s r!get_token_limit'VoyageAIEmbeddingConfig.get_token_limit;s(,,T__gFFr$)rr)__name__ __module__ __qualname____firstlineno__r __annotations__strrboolrrrintrrr r"r)__static_attributes__r+r$r!rr's O#t# %d 3J 3!&t!4J4&+D&9hsm9    GGr$rc N\rSrSr%\\S'S\\4SjrSr \ S\ \ S44Sj5r \ S\4Sj5rS\4S jrS \\S S S\\\4S jrSS\\S S S\S\\\4SjjrS\\S\\4SjrS\S\\4SjrS \\S\\ 4Sjr\S\\4Sj5rSrg)VoyageAIEmbeddingEncoder@configrc URSS9$)NzA sample query.)query) embed_queryr(s r!get_exemplary_embedding0VoyageAIEmbeddingEncoder.get_exemplary_embeddingDs&788r$cgNr+r(s r! initialize#VoyageAIEmbeddingEncoder.initializeGs r$.cNUR5n[R"U5$r?)r<npshaper exemplary_embeddings r!num_of_dimensions*VoyageAIEmbeddingEncoder.num_of_dimensionsJs!"::<xx+,,r$cUR5n[R"[RR U5S5$)Ng?)r<rCiscloselinalgnormrEs r!is_unit_vector'VoyageAIEmbeddingEncoder.is_unit_vectorOs0"::<zz"))..)<=sCCr$c4SURR;$)z7Check if the model is a contextualized embedding model.context)r8rr(s r!_is_context_model*VoyageAIEmbeddingEncoder._is_context_modelTsDKK2222r$textsclientrc## U(dgURR5n/nSnURXRRS9nUVs/sHn[ U5PM nn[ U5HHupXn U(a#[ U5[ :dX[-U:aUv /nSnURU 5 X[- nMJ U(aUv ggs snf7f)z Generate batches of texts based on token limits. Args: texts: List of texts to batch. client: VoyageAI client instance to use for tokenization. Yields: Batches of texts as lists. Nrmodel)r8r)tokenizerlen enumerateMAX_BATCH_SIZEappend) r rSrTmax_tokens_per_batch current_batchcurrent_batch_tokensall_token_liststokens token_countsitextn_tokenss r!_build_batches'VoyageAIEmbeddingEncoder._build_batchesXs #{{::<#%  !//%{{7M7M/N2AB/F / B 'GA#HM"n4(36JJ$# " '($   & , ("   'CsACC %A+Cbatch input_typecUR5(anURU/URRUURRS9nUR SR Vs/sHn[U5PM sn$URUURRUURRURRS9nUR Vs/sHn[U5PM sn$s snfs snf)a  Embed a batch of texts using the appropriate method for the model. Args: batch: List of texts to embed. client: VoyageAI client instance to use for embedding. input_type: Type of input ("document" or "query"). Returns: List of embedding vectors. )inputsrWrirr)rSrWrirr) rQcontextualized_embedr8rrresults embeddingslistembedr)r rhrTriresultembs r! _embed_batch%VoyageAIEmbeddingEncoder._embed_batchs  ! ! # #00wkk,,%!%!=!= 1F *0):)E)EF)E#DI)EF F\\kk,,%;;11!%!=!= "F*0):):;):#DI):; ;GelementscU(d/$URR5nUVs/sHn[U5PM nn/n[UR XB55nURR (aSSKJn U"USS9nUH$nURXSS9n URU 5 M& URX5$s snf![an[S5UeSnAff=f) z Embed documents with automatic batching based on token limits. Args: elements: List of elements to embed. Returns: List of elements with embeddings added. r)tqdmzEmbedding batches)desczgMust have tqdm installed if `show_progress_bar` is set to True. Please install with `pip install tqdm`.Ndocumentri) r8r"r1rorfr tqdm.autorw ImportErrorrsextend_add_embeddings_to_elements) r rurTerSall_embeddingsbatchesrwrhbatch_embeddingss r!embed_documents(VoyageAIEmbeddingEncoder.embed_documentssI'')!)*AQ*,.t**59: ;; ( ( *w-@AE#00:0V   ! !"2 3//II++ !> sB?7 C C CCr:cdURR5nURU/USS9nUS$)zg Embed a single query string. Args: query: Query string to embed. Returns: Embedding vector. r:rzr)r8r"rs)r r:rTrs r!r;$VoyageAIEmbeddingEncoder.embed_querys;''),,eWf,Q""r$cU(d/$URR5nURXRRS9nUVs/sHn[ U5PM sn$s snf)z Count tokens for the given texts. Args: texts: List of texts to count tokens for. Returns: List of token counts for each text. rV)r8r"rXrrY)r rSrT token_lists token_lists r! count_tokens%VoyageAIEmbeddingEncoder.count_tokenssTI'')ooe;;3I3IoJ 2=>+JJ+>>>s A#c[U5[U5:Xde/n[U5Hup4XUlURU5 M! U$r?)rYrZrnr\)rurnelements_w_embeddingrcelements r!r~4VoyageAIEmbeddingEncoder._add_embeddings_to_elementssN8}J///!#H-JA!+G  ' ' 0.r$r+N)ry)r,r-r.r/rr0rfloatr<r@propertytupler3rGr2rMrQr1rrfrsr rr;r staticmethodr~r4r+r$r!r6r6@s= ##9e9 -5c?--DDD343( DI( x( HTRUYDW( VEO<#Y<(0<>A< d5k <@#JW #J$w-#JJ # #e #?$s)?S ?"T']r$r6) dataclassesrtypingrrrrnumpyrCpydanticrr unstructured.documents.elementsr unstructured.embed.interfacesr r unstructured.utilsr rrr&r[rr6r+r$r!rs'!::%3O4y'    g WwWGgWW g!"7#*GoG2 l3l lr$