Mi6F %SrSSKJr SSKrSSKrSSKrSSKJrJrJ r J r SSK r SSK J r SSKJrJr SSKJrJrJr SSKJr SS KJr SS KJr SS KJr SS KJr SS KJ r SSK!J"r" \ S\#\4r$S\%S'SSSSS0SS\ RL/SQSSSSSSSSSSSSSSS.SSjjjr'S0SS4SSjjr(SSjr)"SS5r*g) z9Provides partitioning with automatic file-type detection.) annotationsN)IOAnyCallableOptional) TypeAlias)DataSourceMetadataElement)detect_filetypeis_json_processableis_ndjson_processable)FileType)logger)UnsupportedFileFormatError) exactly_one)check_language_args)PartitionStrategy)dependency_exists.r PartitionerT)pdfjpgpngheicF)fileencoding content_typeurlheaders ssl_verifyrequest_timeoutstrategyskip_infer_table_types ocr_languages languagesdetect_language_per_elementlanguage_fallbackpdf_infer_table_structureextract_images_in_pdfextract_image_block_typesextract_image_block_output_dirextract_image_block_to_payloaddata_source_metadatametadata_filenamehi_res_model_name model_namestarting_page_numberc @^^^^$[XTS9 URSU5 U(a[R"S5 [ U =(d /U 5n Tb[ TTUUUS9unm$O)U0:wa[R"S5 [ UUUTUS9m$UbURS5 URS S5nUbUO [T$U U5n[5nSUUU$U4S jjnT$[R:Xa9URT$5nU"SUUSUUU U U U=(d UUUUUUS .UD6nU"U5$T$R(aIT$RS :Xa9URT$5nU"SUUSUUU U U U=(d UUUUUUS .UD6nU"U5$T$[R:Xa<[!XS 9(d [#S5eURT$5n U "SXS .UD6nU"U5$T$[R$:Xa<['XS 9(d [#S5eURT$5n!U!"SXS .UD6nU"U5$T$[R(:Xa/$[*R,"U5n"U U"S'U U"S'UU"S'UU"S 'U U"S'UU"S'UU"S'UU"S'UU"S'URT$5n#U#"SXS .U"D6nU"U5$)aPartitions a document into its constituent elements. Uses libmagic to determine the file's type and route it to the appropriate partitioning function. Applies the default parameters for each partitioning function. Use the document-type specific partitioning functions if you need access to additional kwarg options. Parameters ---------- filename A string defining the target filename path. file A file-like object using "rb" mode --> open(filename, "rb"). encoding The character-encoding used to decode the input bytes when drawn from `filename` or `file`. Defaults to "utf-8". url The url for a remote document. Pass in content_type if you want partition to treat the document as a specific content_type. headers The headers to be used in conjunction with the HTTP request if URL is set. ssl_verify If the URL parameter is set, determines whether or not partition uses SSL verification in the HTTP request. request_timeout The timeout for the HTTP request if URL is set. Defaults to None meaning no timeout and requests will block indefinitely. content_type A string defining the file content in MIME type metadata_filename When file is not None, the filename (string) to store in element metadata. E.g. "foo.txt" strategy The strategy to use for partitioning PDF/image. Uses a layout detection model if set to 'hi_res', otherwise partition simply extracts the text from the document and processes it. skip_infer_table_types The document types that you want to skip table extraction with. languages The languages present in the document, for use in partitioning and/or OCR. For partitioning image or pdf documents with Tesseract, you'll first need to install the appropriate Tesseract language pack. For other partitions, language is detected using naive Bayesian filter via `langdetect`. Multiple languages indicates text could be in either language. detect_language_per_element Detect language per element instead of at the document level. language_fallback Optional callable for short text (e.g. when detection defaults to English). Called with the text; return a list of ISO 639-3 codes or None to leave language unspecified. pdf_infer_table_structure Deprecated! Use `skip_infer_table_types` to opt out of table extraction for any document type. If True and strategy=hi_res, any Table Elements extracted from a PDF will include an additional metadata field, "text_as_html," where the value (string) is a just a transformation of the data into an HTML . The "text" field for a partitioned Table Element is always present, whether True or False. extract_images_in_pdf Only applicable if `strategy=hi_res`. If True, any detected images will be saved in the path specified by 'extract_image_block_output_dir' or stored as base64 encoded data within metadata fields. Deprecation Note: This parameter is marked for deprecation. Future versions will use 'extract_image_block_types' for broader extraction capabilities. extract_image_block_types Only applicable if `strategy=hi_res`. Images of the element type(s) specified in this list (e.g., ["Image", "Table"]) will be saved in the path specified by 'extract_image_block_output_dir' or stored as base64 encoded data within metadata fields. extract_image_block_to_payload Only applicable if `strategy=hi_res`. If True, images of the element type(s) defined in 'extract_image_block_types' will be encoded as base64 data and stored in two metadata fields: 'image_base64' and 'image_mime_type'. This parameter facilitates the inclusion of element data directly within the payload, especially for web-based applications or APIs. extract_image_block_output_dir Only applicable if `strategy=hi_res` and `extract_image_block_to_payload=False`. The filesystem path for saving images of the element type(s) specified in 'extract_image_block_types'. hi_res_model_name The layout detection model used when partitioning strategy is set to `hi_res`. model_name The layout detection model used when partitioning strategy is set to `hi_res`. To be deprecated in favor of `hi_res_model_name`. starting_page_number Indicates what page number should be assigned to the first page in the document. This information will be reflected in elements' metadata and can be be especially useful when partitioning a document that is part of a larger document. )rfilenamerr.z]The pdf_infer_table_structure kwarg is deprecated. Please use skip_infer_table_types instead.N)rrrr r!zUThe headers kwarg is set but the url kwarg is not. The headers kwarg will be ignored.) file_pathrrrmetadata_file_pathrinfer_table_structurec>UHnTURlTURlTb<[R"T5nU(a UR OSURlMdTR URlM U$)z)Add some metadata fields to each element.N)metadatar data_sourcerfrom_mime_type mime_typefiletype)elementselement out_filetyperr- file_typers u/var/www/eduai.edurigo.com/storigo/production/storigo_env/lib/python3.13/site-packages/unstructured/partition/auto.pyaugment_metadata#partition..augment_metadatassG#&G   +?G   (''66|D FRL,B,BX\  ),5,?,?  ) )r3rrr6r"r%r&r'r/r)r*r+r,r1image)r3rzDetected a JSON file that does not conform to the Unstructured schema. partition_json currently only processes serialized Unstructured output.zDetected an NDJSON file that does not conform to the Unstructured schema. partition_json currently only processes serialized Unstructured output.r&r'rr%r1r"r*r,)r= list[Element]returnrF)r setdefaultrwarningrfile_and_type_from_urlr seekpopdecide_table_extraction_PartitionerLoaderrPDFgetpartitioner_shortnameJSONr ValueErrorNDJSONr EMPTYcopydeepcopy)%r3rrrrrr r!r"r#r$r%r&r'r(r)r*r+r,r-r.r/r0r1kwargskwargs_infer_table_structurer6partitioner_loaderrB partition_pdfr=partition_imagepartition_jsonpartition_ndjsonpartitioning_kwargs partitionr@s% `` ` @rArarasVdT#6 )+<=   $IO]CI 0%!+  i b= NN5 $%0    ! $*::.Et#L  ( 3 % $  " % ,-  HLL *..y9   "7(C//=:"7&?+I+I!5  " ))&&9+J+Jg+U,00;" "7(C//=:"7&?+I+I!5  " ))HMM!"H@Z ,// :!I8I&I))HOO#$hBZ .11)<#KXKFK))HNN" --/9T56/@+,&. #3H/0'0 $2F./&. #7P34rDcvU(aURR5OSnUS:Xa U=(d X1;$X1;$)Nr)namerl)r<r#r(doc_types rArNrNCs; )1x}}""$dH5 )RH,RR  11rDc<\rSrSr%Sr0rS\S'S SjrS SjrSr g) rOiTaProvides uniform helpful error when a partitioner dependency is not installed. Used by `partition()` to encapsulate coping with the possibility the Python environment it is executing in may not have all dependencies installed for a particular partitioner. Provides `.get()` to access partitioners by file-type, which raises when one or more dependencies for that partitioner are not installed. The error message indicates what extra needs to be installed to enable that partitioner. This avoids an inconsistent variety of possibly puzzling exceptions arising from much deeper in the partitioner when access to the missing dependency is first attempted. zdict[FileType, Partitioner] _partitionerscUR(d[SUS35eXR;aURU5URU'URU$)zReturn partitioner for `file_type`. Raises when one or more package dependencies for that file-type have not been installed. Also raises when the file-type is not partitionable. z&Partitioning is not supported for the z file type.)is_partitionablerrr_load_partitioner)selfr@s rArQ_PartitionerLoader.getes` )),8 ;O  .. .,0,B,B9,MD  y )!!),,rDc"URH8n[U5(aM[URSURS35e UR (de[ R"UR5n[X1R5$)zBLoad the partitioner for `file_type` after verifying dependencies.zg() is not available because one or more dependencies are not installed. Use: pip install "unstructured[z:]" (including quotes) to install the required dependencies) importable_package_dependenciesr ImportErrorpartitioner_function_name extra_namert importlib import_modulepartitioner_module_qnamegetattr)rvr@pkg_namepartitioner_modules rAru$_PartitionerLoader._load_partitionerws"AAH$X..! ::;<22;2F2F1GH<=B))))&44Y5W5WX)+N+NOOrDrHN)r@rrGr) __name__ __module__ __qualname____firstlineno____doc__rr__annotations__rQru__static_attributes__rHrDrArOrOTs 24M.3-$PrDrO)N)4r3 Optional[str]rzOptional[IO[bytes]]rrrrrrrdict[str, str]r boolr! Optional[int]r"strr# list[str]r$rr%Optional[list[str]]r&rr'z.Optional[Callable[[str], Optional[list[str]]]]r(rr)rr*rr+rr,rr-zOptional[DataSourceMetadata]r.rr/rr0rr1intrYrrGrF) rrrrrrr rr!rrGztuple[io.BytesIO, FileType])r<zOptional[FileType]r#rr(rrGr)+r __future__rrWr}rgtypingrrrrrftyping_extensionsrunstructured.documents.elementsr r unstructured.file_utils.filetyper r r unstructured.file_utils.modelrunstructured.loggerrunstructured.partition.commonr$unstructured.partition.common.commonr"unstructured.partition.common.langr&unstructured.partition.utils.constantsrunstructured.utilsrlistrrAUTOrarKrNrOrHrDrArsX?"  ..'G 3&D<BD0!#tG}"45 Y5#N&!%""& %)%**(E#'%)(-HL&+"'5948+09='+'+ $ !3N&N& N& N&  N&  N&N&N&#N&N&&N&!N&#N&"&N&FN& $!N&" #N&$ 3%N&&%2'N&(%))N&*7+N&,%-N&.%/N&01N&23N&45N&67N&f#' %)    #  ! (2 2%2 $2 2"2P2PrD