qi' ZdZddlmZddlmZddlmZgdZeGddeZdgZ y) z Processor class for EVOLLA. ) BatchFeature)ProcessorMixin)auto_docstring)aa_seqfoldseekmsac eZdZdfd ZddZ ddefdZe ddee e zdzdeee ee zdzdedzdedzfd Z d Z d Z d Z d ZxZS)EvollaProcessorNc | td| tdt| ||d|j_||_||_y)a protein_tokenizer (`EsmTokenizer`): An instance of [`EsmTokenizer`]. The protein tokenizer is a required input. protein_max_length (`int`, *optional*, defaults to 1024): The maximum length of the sequence to be generated. text_max_length (`int`, *optional*, defaults to 512): The maximum length of the text to be generated. Nz+You need to specify an `protein_tokenizer`.z"You need to specify a `tokenizer`.z<|reserved_special_token_0|>) ValueErrorsuper__init__ tokenizer pad_tokenprotein_max_lengthtext_max_length)selfprotein_tokenizerrrrkwargs __class__s ^/opt/pipecat/venv/lib/python3.12/site-packages/transformers/models/evolla/processing_evolla.pyrzEvollaProcessor.__init__sU  $JK K  AB B *I6#A "4.c Pg}|D]}|jd}|jd}djt||Dcgc]&\}}|j|j z(c}}} |j | |j |dd|d} | Scc}}w)NrrptT)return_tensors truncation max_lengthpadding)getjoinzipupperlowerappendr) rproteinsr sa_sequencesproteinrrsf sa_sequence sa_tokenss rprocess_proteinsz EvollaProcessor.process_proteins2s  -G[[*F{{:.H''SQYEZ"[TQ1779qwwy#8"[\K    ,  - ** $K]gk+  #\s+B" rcg}|D]1}|jj|dd}|j|3|j|dddd|}|S)NFT)tokenizeadd_generation_promptrlongest)add_special_tokensrrrr)rapply_chat_templater%)rtextsrpromptsmessagesprompt prompt_inputss r process_textzEvollaProcessor.process_text?sw  #H^^77&*8F NN6 "  # $& ' rr& messages_listrc || td||n |j}||n |j}t|tr|g}t|t t frt|dt t fs|g}t|t t frtd|Ds tdt|t t fr6td|Ds$tddjtd|t|t t fr|D]}t|t t fstd t|d td |Ds td td |Dstd|Dsstd|ntdt|d |j||}|j||}t|d|d|d|ddS)a proteins (`Union[List[dict], dict]`): A list of dictionaries or a single dictionary containing the following keys: - `"aa_seq"` (`str`) -- The amino acid sequence of the protein. - `"foldseek"` (`str`) -- The foldseek string of the protein. messages_list (`Union[List[List[dict]], List[dict]]`): A list of lists of dictionaries or a list of dictionaries containing the following keys: - `"role"` (`str`) -- The role of the message. - `"content"` (`str`) -- The content of the message. protein_max_length (`int`, *optional*, defaults to 1024): The maximum length of the sequence to be generated. text_max_length (`int`, *optional*, defaults to 512): The maximum length of the text. Return: a dict with following keys: - `protein_input_ids` (`torch.Tensor` of shape `(batch_size, sequence_length)`) -- The input IDs for the protein sequence. - `protein_attention_mask` (`torch.Tensor` of shape `(batch_size, sequence_length)`) -- The attention mask for the protein sequence. - `text_input_ids` (`torch.Tensor` of shape `(batch_size, sequence_length)`) -- The input IDs for the text sequence. - `text_attention_mask` (`torch.Tensor` of shape `(batch_size, sequence_length)`) -- The attention mask for the text sequence. z3You need to specify `messages_list` and `proteins`.c3<K|]}t|tywN isinstancedict.0ps r z+EvollaProcessor.__call__..s:aST:a;N:azUThe proteins should be a list of dictionaries, but not all elements are dictionaries.c3\K|]$}td|jD&yw)c3,K|] }|tvywr>)PROTEIN_VALID_KEYS)rCks rrEz5EvollaProcessor.__call__...s:A'':sN)allkeysrBs rrEz+EvollaProcessor.__call__..s&; ?@C:: :; s*,z2There should be a list of dictionaries with keys: z, z for each protein.But got: z;Each messages in messages_list should be a list instead of .c3<K|]}t|tywr>r?rCms rrEz+EvollaProcessor.__call__..sA1:a.ArFzfEach message in messages_list should be a list of dictionaries, but not all elements are dictionaries.c3TK|] }t|jdk7"yw)N)lenrLrOs rrEz+EvollaProcessor.__call__..s.s+D=>CMfi%88Ds(*zlEach message in messages_list should be a list of dictionaries with two keys: 'role' and 'content'.But got: zFThe messages_list should be a list of lists of dictionaries, but it's input_idsattention_mask)protein_input_idsprotein_attention_maskrXrY)data)r rrr@rAlisttuplerKr!rI TypeErrortypeanyr-r9r) rr&r:rrrr6r, text_tokenss r__call__zEvollaProcessor.__call__Ws+>  }4RS S3E3Q/W[WnWn-<-H/dNbNb h % zH mdE] 3J}UVGWZ^`eYf"?   rc:|jj|i|Sr>)r batch_decoderargsrs rrezEvollaProcessor.batch_decodes*t~~**D;F;;rc:|jj|i|Sr>)rdecoderfs rrizEvollaProcessor.decodes$t~~$$d5f55rc:|jj|i|Sr>)rrerfs rprotein_batch_decodez$EvollaProcessor.protein_batch_decodes 2t%%22DCFCCrc:|jj|i|Sr>)rrirfs rprotein_decodezEvollaProcessor.protein_decodes ,t%%,,d=f==r)N)rn)ro)NNNN)__name__ __module__ __qualname__rr-intr9rr]rArcrerirkrm __classcell__)rs@rr r s/(  #0.2>B)-&* T t*t#d*T DJ'$t*4t;T  $J T t T T l<6D>rr N) __doc__feature_extraction_utilsrprocessing_utilsrutilsrrIr __all__rrr{sJ5$3[>n[>[>|  r