ZiyCSSKJrJrJr SSKJr SSKJr SSKJr SSKJ r J r SSK J r SSK Jr SSKrSS KJrJrJrJr SSKr\R,"S 5r"S S \5r"S S\5rSS\ S\\S4S\4SjjrS\\\4S\\S44Sjr"SS\5r"SS\5r"SS\5r S\!S\"4Sjr#Sr$g))UnionListTuple) TypedDict) Generator) Transformer)v_argsDiscard)Tree)TokenN)log_htmlrtf_strippinglog_htmlrtf_statelog_htmlrtf_evaluated_chars is_logger_onRTFDEc^\rSrSrSr\"SS9S\4Sj5r\S\ S\ \ S 44S j5r S r g ) StripNonVisibleRTFGroups zVisits each Token in provided RTF Trees and strips out any RTF groups which are non-visible when de-encapsulated into HTML. T)treercURn[U5S:XagUSnS/n[U[5(aURU;aU$S/n[U[5(aURU;ag/SQnUR U5nXv;ag/nSn [U[5(aO[UR5S:wa6URSn [U [ 5(aU RS:XaSn Sn U SLa?US n [U [ 5(a%U RS :XaU n U RU;aU$gU$) aTransformer which aggressively seeks out possible non-visible RTF groups and replaces them with empty strings. NOTE: Currently deleting all groups that don't have an htmltag. Please file an issue if you find one that should be included in de-encapsulated HTML. I will refine what gets deleted and what is converted based on identified needs for greater functionality or specific issues which need to be addressed. Args: tree: A .rtf group (Tree object) which needs its contents decoded. r htmltag_groupmhtmltag_group)s\contexts \colortbls\fonttblN STAR_ESCAPET CONTROLWORD) childrenlen isinstancer dataget_first_controlwordr typevalue) selfrr first_childknown_control_groups known_non_visible_control_groupsnon_visible_control_words first_controlunderstood_commandsis_star_escaped first_item control_word first_tokens l/var/www/eduai.edurigo.com/question_generate/ques_gen_env/lib/python3.13/site-packages/RTFDE/transformers.pygroupStripNonVisibleRTFGroups.group$sF== x=A qk /0 k4 ( (#77 ,<+=( k4 ( (#CC%P!228<  5 *, {D ) ) %% &! +$--a0J*e,,??m3&*O d ""1+K+u--##}4#.L#))-@@#  rrreturnNcvUH"nURS:XaURs $M$ g![a M4f=f)zExtracts the first control word from a .rtf group. Args: children: A list of child objects within a .rtf group Returns: The first controlword found in a group. Returns None if no controls words are found. rN)r"r#AttributeError)ris r/r!.StripNonVisibleRTFGroups.get_first_controlwordZsFA 66]*77N+ "  s * 88)__name__ __module__ __qualname____firstlineno____doc__r r r0 staticmethodrrstrr!__static_attributes__r7rr/rr sR 3$33js4xrrc|\rSrSrSrS\S\4SjrS\S\4Sjr S\S\4Sjr S \S\4S jr S \S\4S jr S\S\4S jr S\S\4SjrS\S\4SjrS\4SjrS \S\4SjrS\S\4SjrS\S\4SjrS\S\4SjrS\S\4SjrS\S\4SjrS\S\4SjrS\S\4SjrS\S\4SjrS\S\4SjrS\S\4SjrSrg )! RTFCleanerlzVisits each Token in provided RTF Trees. Converts all tokens that need converting. Deletes all tokens that shouldn't be visible. And, joins all strings that are left into one final string. argsr2c$SRU5$)zJoins the .rtf object's string representations together at highest level object `start`. This is the final string combination. rjoinr$rCs r/startRTFCleaner.startpsxx~rstringc6URb UR$g)z*Convert a string object into a raw string.rr#r$rJs r/STRINGRTFCleaner.STRINGvs << #<< rcUR$NrLrMs r/ SPACE_SAVERTFCleaner.SPACE_SAVE|s ||rstringsc$SRU5$)zGConvert all string objects withing a string group into a single string.rrEr$rTs r/rJRTFCleaner.stringsxx  rgrpc/nUH4n[U[[55(aM#URU5 M6 SR U5$)z&Join the strings in all group objects.r)rr"r appendrF)r$rX _new_childrenr5s r/r0RTFCleaner.groupsD A!T']++$$Q'  xx &&rcZUVs/sH o"cMUPM nnSRU5$s snf)z_Join the all the strings in an .rtf object into a single string representation of the document.rrE)r$rCr5s r/documentRTFCleaner.documents)14a41xx~2s((cg)zDelete all open parens.rr7rGs r/ OPENPARENRTFCleaner.OPENPARENrcg)zDelete all closed parens.rr7rGs r/ CLOSEPARENRTFCleaner.CLOSEPARENrcrc[$)zProcess MHTMLTAG groups Currently discarding because they don't need to be processed. Returns: Always returns a discard object.r rGs r/rRTFCleaner.mhtmltag_groups rc$SRU5$)zYHTMLTAG processing. Takes any string values within an HTMLTAG and returns them. rrErVs r/rRTFCleaner.htmltag_groups xx  rhtmltagcg)zDelete all HTMLTAG objectsrr7)r$rls r/HTMLTAGRTFCleaner.HTMLTAGrcrcharcg)zDelete all star escape objectsrr7)r$rps r/rRTFCleaner.STAR_ESCAPEsrsymbolsc$SRU5$)z7Join all visible symbols from in control symbol groups.rrE)r$rss r/control_symbolRTFCleaner.control_symbolsxx  rc"SR5$)z8Convert non-breaking spaces into visible representation. encoderGs r/NONBREAKING_SPACERTFCleaner.NONBREAKING_SPACE!!rc"SR5$)z9Convert non-breaking hyphens into visible representation.­ryrGs r/NONBREAKING_HYPHENRTFCleaner.NONBREAKING_HYPHENr}rc"SR5$)z8Convert hyphen control char into visible representation.u‧ryrGs r/OPTIONAL_HYPHENRTFCleaner.OPTIONAL_HYPHENr}rcg)aConvert a formula character into an empty string. If we are attempting to represent formula characters the scope for this library has grown too inclusive. This was only used by Word 5.1 for the Macintosh as the beginning delimiter for a string of formula typesetting commands.rr7rGs r/FORMULA_CHARACTERRTFCleaner.FORMULA_CHARACTERrcg)zProcess index subentry items Discard index sub-entries. Because, we don't care about indexes when de-encapsulating at this time.rr7rGs r/INDEX_SUBENTRYRTFCleaner.INDEX_SUBENTRYrrcRSSSS.nURURS5nUbU$g)zzConvert encoded chars which are mis-categorized as control symbols into their respective chars. Delete all the other ones.{}\)s\{s\}s\\Nr)getr#)r$rCrs replacements r/ CONTROLSYMBOLRTFCleaner.CONTROLSYMBOLs9  kk$**d3  " rc 2SSSSRS5SRS5SRS5SRS5SRS5S RS5S RS5S . nURURS 5nUbU$g )zConvert encoded chars which are mis-categorized as control words into their respective chars. Delete all the other ones.   u‘zutf-8u’u“u”u•u–u—) s\pars\tabs\lines\lquotes\rquotes \ldblquotes \rdblquotes\bullets\endashs\emdashNr)rzrr#)r$rCwordsrs r/rRTFCleaner.CONTROLWORDs!1!1$OOG4$OOG4!1!1!1  ii D1  " rr7N)r8r9r:r;r<rbytesrHr rNrRrJr0r^rarerrrnrrur{rrrrrrr?r7rr/rArAls|$5 Uu 5!d!u! ''%'Te eu4!T!e!u% !d!u!"e"" "u"" "E"e" e 5U % E %rrAr current_stater2c#B# UcS/nOU/nURHnSn[U[5(a[X2S5HnUv M M5[ U5nUbUR U5 Uv MZUSSLdMd[ S5SLa [U5 Uv M g7f)aGet a list of Tokens which should be suppressed by HTMLRTF control words. NOTE: This de-encapsulation supports the HTMLRTF control word within nested groups. The state of the HTMLRTF control word transfers when entering groups and is restored when exiting groups, as specified in [MSFT-RTF]. Returns: A list of Tokens which should be suppressed by HTMLRTF control words. NFTRTFDE.HTMLRTF_Stripping_logger)rrr get_stripped_HTMLRTF_valuestoggle_htmlrtfrZrr )rr htmlrtf_stackchild is_htmlrtftoggles r/rrs &  eT " "5e2=NO P(.J%$$Z0 r"d* @ATI)%0 s A7B="Brc:[U[5(aURS:XavURR 5R 5n[ U5S:a$USS:Xa[S5SLa [SU5 g[S5SLa [S U5 gg ) zIdentify if htmlrtf is being turned on or off. Returns: Bool representing if htmlrtf is being enabled or disabled. None if object is not an HTMLRTF token. HTMLRTFrr0z$RTFDE.HTMLRTF_Stripping_state_loggerTOFFFONN) rr r"r#decodestriprrr)r htmlrtfstrs r/rrs % :: "++-335JJ!# 2#(= FG4O%e4BCtK!B/ rcH^\rSrSrSrS\\4U4SjjrS\4SjrSr U=r $)DeleteTokensFromTreei$aSRemoves a series of tokens from a Tree. Parameters: tokens_to_delete: A list of tokens to delete from the Tree object. (sets self.to_delete) Attributes: to_delete: A list of tokens to delete from the Tree object. delete_start_pos: The starting position for all the identified tokens. Used to identify which tokens to delete. tokens_to_deletec>[TU]5 XlURVs1sHo"RiM snUlgs snf)zSetup attributes including token start_pos tracking. Args: tokens_to_delete: A list of tokens to delete from the Tree object. (sets self.to_delete) N)super__init__ to_delete start_posdelete_start_pos)r$rr5 __class__s r/rDeleteTokensFromTree.__init__/s5 )6:nn Enn E EsAtokenc[S5SLaUb [U5 [U[5(aURUR ;a[ UR5Hup#URUR:XdM!URUR:XdM=URUR:XdMY[S5SLa [U5 URRU5 [s $ U$)zDiscard any identified tokens. Args: token: All tokens within the transformed tree. Returns: Returns all non-identified tokens. Returns Discard objects for any identified tokens. z&RTFDE.HTMLRTF_Stripping_logger_VERBOSETr) rrrr rr enumeraterend_posr#r popr )r$renr5s r/__default_token__&DeleteTokensFromTree.__default_token__9s @ AT I +E2 eU # #$"7"77&t~~6EB u6 U]]25;;.'(HITQ1%8**2.&7 r)rr) r8r9r:r;r<rr rrr? __classcell__)rs@r/rr$s+FeFurrcB\rSrSrSrS\4SjrS\4SjrS\4SjrSr g) StripUnusedSpecialCharactersiQaStrip all unused tokens which lark has extracted from the RTF. These tokens are largely artifacts of the RTF format. We have to do this because we use the "keep_all_tokens" option in our lark parser. It's better to be explicit then to allow for ambiguity because of the grammar. rc[$zERemove RTF braces. Returns: Always returns a discard object.rhr$rs r/_LBRACE$StripUnusedSpecialCharacters._LBRACEY rc[$rrhrs r/_RBRACE$StripUnusedSpecialCharacters._RBRACE`rrc[$)zRemove spaces which are not a part of the content These are mostly spaces used to separate control words from the content they precede. Returns: Always returns a discard object. rhrs r/ _SPACE_DELETE*StripUnusedSpecialCharacters._SPACE_DELETEgs rr7N) r8r9r:r;r<r rrrr?r7rr/rrQs+UU5rrc&\rSrSrSrS\4SjrSrg)StripControlWordsirzGVisits each control word and strips the whitespace from around it. rcTURURR5S9nU$)zStrips the whitespace from around a provided control word. Args: token: A CONTROLWORD token to strip whitespace from. rL)updater#r)r$rtoks r/rStripControlWords.CONTROLWORDvs& ll!2!2!4l5 rr7N)r8r9r:r;r<r rr?r7rr/rrrsrrraw_rtfcL/nSn[R"X 5Hbn[US5nUR5SnXXT-UR 5UR5SXT-US.nUR U5 Md SnSnUHn XpXS- nU SnM XpUS - nXq4$) aExtracts binary objects from a rtf file. Parameters: raw_rtf: (bytes): It's the raw RTF file as bytes. Returns: A tuple containing (new_raw, found_bytes) new_raw: (bytes) A bytes object where any binary data has been removed. found_bytes: (list) List of dictionaries containing binary data extracted from the rtf file. Each dictionary includes the data extracted, where it was extracted from in the original rtf file and where it can be inserted back into the stripped output. Description of found_bytes dictionaries: "bytes": (bytes) The binary data contained which was extracted. "ctrl_char": (tuple) Tuple containing the binary control word and its numeric parameter "start_pos": (int) The position (in the original raw rtf data) where the binary control word started. "bin_start_pos": (int) The position (in the original raw rtf data) where the binary data starts. "end_pos": (int) The position (in the original raw rtf data) where the binary data ends. Here is an example of what this looks like (by displaying the printable representation so you can see the bytes and then splitting the dict keys on new lines to make it readable.) >> print(repr(found_bytes)) "{'bytes': b'\xf4UP\x13\xdb\xe4\xe6CO\xa8\x16\x10\x8b\n\xfbA\x9d\xc5\xd1C', 'ctrl_char': (b'\\bin', b'20'), 'start_pos': 56, 'end_pos': 83, 'bin_start_pos': 63}" s(\\bin)([0-9]+)[ ]?rr)r ctrl_charrr bin_start_posrrrN)refinditerintspangroupsrZ) r found_bytes byte_finder matchitemparamrbyte_objnew_raw start_buffer new_bytess r/strip_binary_objectsrs8K)K[[6 IaL!!(, $=3FG!*!1!1!3!*!1!!4,2%2  8$7GL <+(>?? + ! |}%%G  !!rcRUS:Xa URSR5S5nU$)Nhtmlrxs )replacerz)content content_types r/transform_based_on_content_typers(v//)"2"2"4i@ NrrQ)%typingrrrrcollections.abcr lark.visitorsrr r lark.treer lark.lexerr r RTFDE.utilsr rrrlogging getLoggerlogrrAboolrr>rrrrrtuplerrr7rr/rs &%%%) kk  J{JXGGRd5d;KW`:%c *uT$Y/?$+;+Z;B  /"%/"E/"dr