qiddZddlZddlmZddlZddlmZddlmZddlm Z ddl m Z dd l mZmZmZdd lmZdd lmZmZdd lmZdd lmZddlmZmZmZmZmZddl m!Z!m"Z"ddl#m$Z$ddl%m&Z&m'Z'm(Z(m)Z)ddl*m+Z+e)jXe-Z.dej^de0de0fdZ1GddejdZ3GddejdZ4 de'd34Gd5d6e;eZ?Gd7d8e;Z@Gd9d:e;eZAgd;ZBy)=zPyTorch Blenderbot model.N)Callable)nn)CrossEntropyLoss)initialization)ACT2FN)Cache DynamicCacheEncoderDecoderCache)GenerationMixin)create_bidirectional_maskcreate_causal_mask)FlashAttentionKwargs)GradientCheckpointingLayer)BaseModelOutput)BaseModelOutputWithPastAndCrossAttentions!CausalLMOutputWithCrossAttentionsSeq2SeqLMOutputSeq2SeqModelOutput)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)TransformersKwargsauto_docstringis_torchdynamo_compilinglogging)BlenderbotConfig input_ids pad_token_iddecoder_start_token_idc|j|j}|ddddfj|ddddf<||dddf<| td|j |dk(||S)z1 Shift input ids one token to the right. Nrrz1self.model.config.pad_token_id has to be defined.i) new_zerosshapeclone ValueError masked_fill_)rr r!shifted_input_idss d/opt/pipecat/venv/lib/python3.12/site-packages/transformers/models/blenderbot/modeling_blenderbot.pyshift_tokens_rightr+4s}"++IOO<(CRC0668ae4adLMM""#4#(HPUPZPZcgcncncucuLw|,,r,)rN) __name__ __module__ __qualname____doc__intr4r>SizeTensorrB __classcell__r6s@r*r.r.DsT8s838qu -$zz -CF -Z_ZfZfimZm - -r,r.c `eZdZdZd dededededzffd Zdejffd Z xZ S) BlenderbotScaledWordEmbeddingz\ This module overrides nn.Embeddings' forward by multiplying with embeddings scale. r/r0 padding_idx embed_scaleNc6t||||||_yr2)r3r4rQ)r5r/r0rPrQr6s r*r4z&BlenderbotScaledWordEmbedding.__init__^s  D&r,rc<t|||jzSr2)r3rBrQ)r5rr6s r*rBz%BlenderbotScaledWordEmbedding.forwardbswy)D,<,<<rKrBrLrMs@r*rOrOYsE's'3'S'_dgk_k'===r,rOmodulequerykeyvalueattention_maskscalingdropoutkwargsc ||jddz}tj||jdd|z}|||z}tj j |d}tj j|||j}tj||} | jddj} | |fS)Nr#r;rdimptrainingr) sizer>matmul transposer functionalsoftmaxr\rd contiguous) rVrWrXrYrZr[r\r] attn_weights attn_outputs r*eager_attention_forwardrmgs**R.D(<<s}}Q':;gEL!#n4 ==((2(>L==((6??([L,,|U3K''1-88:K  $$r,cZeZdZdZ ddedededededed edzd edzffd Z dd e jd e jdzde dzde jdzdede jdzde e dee je jdzee jdzffdZxZS)BlenderbotAttentionz=Multi-headed attention from 'Attention Is All You Need' paperN embed_dim num_headsr\ is_decoderbias is_causalconfig layer_idxc t |||_||_||_||z|_||_|j |z|jk7rtd|jd|d|j dz|_||_ ||_ ||_ |9|jr-tjd|jjdt!j"||||_t!j"||||_t!j"||||_t!j"||||_y)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).r_zInstantiating a decoder z without passing `layer_idx` is not recommended and will lead to errors during the forward call, if caching is used. Please make sure to provide a `layer_idx` when creating this class.rs)r3r4rprqr\head_dimrur'r[rrrtrvlogger warning_oncer6rErLineark_projv_projq_projout_proj) r5rprqr\rrrsrtrurvr6s r*r4zBlenderbotAttention.__init__s$ "" !Y.  MMI %$.. 8MdnnM]$YKr3 }}d* $""     *4>>+B+B*CD,,  ii 94@ ii 94@ ii 94@  )YTB r, hidden_stateskey_value_statespast_key_valuesrZoutput_attentionscache_positionr]returnc |du}|jdd\} } |r|jdn| } | | d|jf} | | d|jf} |j|j| j dd}d}|St |t rA|jj|j}|r |j}n|j}n|}|r|n|}|rK|I|rGj|jj}|j|jj}n|j|}|j!|}|j| j dd}|j| j dd}|T|s|nd}j#|||jd|i\}}|r)t |t rd|j|j<t%j&|j(j*t,}||||||f|j.sdn |j0|j2|d |\}}|j5| | dj7}|j9|}||fS) z#Input shape: Batch x Time x ChannelNr#rr;FrT)r\r[r)r%ryrviewrg isinstancer is_updatedgetrvcross_attention_cacheself_attention_cachelayerskeysvaluesr}r~updater get_interfaceru_attn_implementationrmrdr\r[reshaperjr)r5rrrrZrrr]is_cross_attentionrCtgt_lensrc_len q_input_shapekv_input_shape query_statesrcurr_past_key_valuescurrent_states key_states value_statesattention_interfacerlrks r*rBzBlenderbotAttention.forwards{ .T9%**3B/ W/A"((+wgr4==9 wDMM:7t{{=166 FPPQRTUV   &/+>?,77;;DNNK %+:+P+P(+:+O+O('6$-?)] /"=*-44T^^DIIJ/66t~~FMML^4J;;~6L(.9CCAqIJ,<,,n=GG1ML*7It+?+F+F dnn?OQ_>`,( L&*_FY*ZAEO..t~~>(?(M(M KK , ,.E) %8      % $}}C$,,LL/ %  % ! \"))#w;FFH mmK0 L((r,)rFTFNN)NNNFN)rErFrGrHrIrUboolrr4r>rKr rrtuplerBrLrMs@r*roros=G  *. $%C%C%C %C  %C  %C%C!4'%C:%CT15(,.2"'.2P)||P) ,,-P) P)  t+ P)  P) t+P)-.P) u||U\\D0% 2E2LL MP)r,roc |eZdZdeffd Z ddej dej dedej fdZxZ S) BlenderbotEncoderLayerrucht||j|_t |j|j |j ||_tj|j|_ |j|_ t|j|_|j|_tj |j|j"|_tj |j"|j|_tj|j|_y)N)rprqr\ru)r3r4d_modelrproencoder_attention_headsattention_dropout self_attnr LayerNormself_attn_layer_normr\ractivation_function activation_fnactivation_dropoutr|encoder_ffn_dimfc1fc2final_layer_normr5rur6s r*r4zBlenderbotEncoderLayer.__init__s ,nn44,,   %'LL$@!~~ #F$>$>?"(";";99T^^V-C-CD99V33T^^D " T^^ <r,rrZrrc|}|j|}|j|||\}}tjj ||j|j }||z}|}|j |}|j|j|}tjj ||j|j }|j|}tjj ||j|j }||z}|jtjk(rEtj|jjdz }tj || |}||fS)a Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`): attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. )rrZrrbi)minmax)rrrrhr\rdrrrrrr<r>float16finforclamp)r5rrZrresidualrk clamp_values r*rBzBlenderbotEncoderLayer.forwardsP!11-@ &*nn')/'5' # |  --mt||VZVcVc-d  =0  --m< **488M+BC  --mt?V?Vaeanan-o /  --mt||VZVcVc-d  =0   %-- /++m&9&9:>>EK!KK K<[YMl**r,)F) rErFrGrr4r>rKrrBrLrMs@r*rrsL=/=,#( %+||%+ %+ %+  %+r,rceZdZddededzffd Z ddejdejdzdejdzdejdzd edzd e dzd e dzd ejdzd ejfdZ xZ S)BlenderbotDecoderLayerNrurvc t||j|_t |j|j |j dd|||_|j|_t|j|_ |j|_ tj|j|_t |j|j |j d|||_tj|j|_tj$|j|j&|_tj$|j&|j|_tj|j|_y)NT)rprqr\rrrtrurv)r\rrrurv)r3r4rrprodecoder_attention_headsrrr\rrrrrrr encoder_attnencoder_attn_layer_normr|decoder_ffn_dimrrr)r5rurvr6s r*r4zBlenderbotDecoderLayer.__init__?s  ,nn44,, ~~ #F$>$>?"(";";$&LL$@!/ NN  * *,,  (*||DNN'C$99T^^V-C-CD99V33T^^D " T^^ <r,rrZencoder_hidden_statesencoder_attention_maskrr use_cacherrc ,|} |j|}|j|||||\}} tjj ||j|j }| |z}d} |g|} |j |}|j|||||\}} tjj ||j|j }| |z}|} |j|}|j|j|}tjj ||j|j }|j|}tjj ||j|j }| |z}|f} |r| | | fz } | S)a Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`): attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. encoder_hidden_states (`torch.FloatTensor`): cross attention input to the layer of shape `(batch, seq_len, embed_dim)` encoder_attention_mask (`torch.FloatTensor`): encoder attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. past_key_values (`Cache`): cached past key and value projection states output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence. It is used to update the cache in the correct position and to infer the complete sequence length. )rrrZrrrbN)rrrZrr) rrrrhr\rdrrrrrrr) r5rrZrrrrrrrself_attn_weightscross_attn_weightsoutputss r*rBzBlenderbotDecoderLayer.forward^s8!11-@ ,0>>'+)/) ,:, ( ( --mt||VZVcVc-d  =0 " ,$H 88GM040A0A+!65 /"3 1B1 -M-MM11-4< >Gr,r2)NNNNFTN) rErFrGrrIr4r>rKr rrBrLrMs@r*rr>s=/=C$J=D/3596:(,).!%.2H||H t+H %||d2 H !& t 3 H  H $;H$;H t+H Hr,rcReZdZUeed<dZdZdZdZdZ dZ fdZ e dZ xZS)BlenderbotPreTrainedModelrumodelTct||t|tr t j |j yyr2)r3 _init_weightsr"BlenderbotForConditionalGenerationinitzeros_final_logits_bias)r5rVr6s r*rz'BlenderbotPreTrainedModel._init_weightss3 f% f@ A KK00 1 Br,c|jj}tjgddddd|gg|j}|j |||d}|S)N)r r;r r;r=)rZrdecoder_input_ids)rur r>tensorr=ne)r5 pad_tokenr dummy_inputss r*rz&BlenderbotPreTrainedModel.dummy_inputssZKK,, LL"2Q2q)4L!MVZVaVab 'll95"!* r,)rErFrGr__annotations__base_model_prefixsupports_gradient_checkpointing_supports_flash_attn_supports_sdpa_supports_flex_attn_can_compile_fullgraphrpropertyrrLrMs@r*rrsE &*#N!2 r,rc<eZdZdZdeffd Z ddZxZS)BlenderbotEncoderz Transformer encoder consisting of *config.encoder_layers* self attention layers. Each layer is a [`BlenderbotEncoderLayer`]. Args: config: BlenderbotConfig embed_tokens (nn.Embedding): output embedding ruct|||j|_|j|_|j }|j |_|j|_ |jrtj|nd}t|j||j||_t!|j||_t%j&t)|j*Dcgc] }t-|c}|_t%j0|j |_d|_|j7ycc}w)NrTrQF)r3r4r\encoder_layerdrop layerdroprr rPmax_position_embeddingsmax_source_positionsscale_embeddingmathsqrtrO vocab_size embed_tokensr.embed_positionsr ModuleListrangeencoder_layersrrr layer_normgradient_checkpointing post_init)r5rurprQ_r6s r*r4zBlenderbotEncoder.__init__s  ~~ 11NN !..$*$B$B!.4.D.Ddii *# 9   y$*:*:   D  * *  mmUSYShShMi$j%;F%C$jk ,,v~~6&+#  %ks-Ec ||n|jj}||n|jj}||n|jj}| | t d|8|j |||j }|jd|d}n!||j dd}n t d||j|}|j|} || z} tjj| |j|j} t|j||}|rdnd} |rdnd} t|j D]b\} }|r| | fz} d}|jr&t#j$g}||j&krd }|rd }n|| || }|d } |sZ| |d fz} d|j)| } |r| | fz} |st+d| | | fDSt-| | | S)a8 Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. NzDYou cannot specify both input_ids and inputs_embeds at the same timer#z5You have to specify either input_ids or inputs_embedsrb)ru inputs_embedsrZFT)NN)rrrc3&K|] }|| ywr2r.0vs r* z,BlenderbotEncoder.forward..RseqWXWdeslast_hidden_stater attentions)ruroutput_hidden_statesuse_return_dictr'%warn_if_padding_and_no_attention_maskrerrrrrhr\rdr enumeraterr>randrrrr)r5rrZrrr return_dictr] input_shape embed_posrencoder_statesall_attentionsidx encoder_layerto_dropdropout_probability layer_outputss r*rBzBlenderbotEncoder.forwards0P2C1N-TXT_T_TqTq$8$D $++JjJj &1%cd d  "  6 6y. Q#..*K!r;r?;I  &',,.s3KTU U   --i8M((5 % 1  --mt||VZVcVc-d 2;;')  40d"+DKK"8 F C#!/=2B!BG}}&+jjn#&7"G , -!"&7! !.a 0  !/=3C2E!E- F2 6 +}.>>Ne]NN$See e+>Vd  r,)NNNNNN)rErFrGrHrr4rBrLrMs@r*rrs//8!m r,rcfeZdZdZdeffd Z ddejdzfdZxZ S)BlenderbotDecoderz Transformer decoder consisting of *config.decoder_layers* layers. Each layer is a [`BlenderbotDecoderLayer`] Args: config: BlenderbotConfig embed_tokens (nn.Embedding): output embedding ruc t|||j|_|j|_|j |_|j|_|jrtj|jnd}t|j|j|j ||_t!|j|j|_t%j&t)|j*Dcgc]}t-||c}|_t%j0|j|_d|_|j7ycc}w)NrTr)rvF)r3r4r\decoder_layerdroprr rPrmax_target_positionsrrrrrOrrr.rrrrdecoder_layersrrrrrr)r5rurQir6s r*r4zBlenderbotDecoder.__init__as  ~~ 11!..$*$B$B!393I3Idii/s 9   v~~t/?/?[  D  * * NN mmBGH]H]B^ _Q #Fa 8 _  ,,v~~6&+#  `s?ENrc ||n|jj}| | n|jj} ||n|jj}| | n|jj} |du|duz r t d|$|} | j }|jd|d}n-| |jdd}|dddddf} n t d||j| }|jr%|jr|rtjdd}|rd|b||jjr4tt!|jt!|jnt!|j}|jdd\}}||j#nd}| %t%j&|||z|j( } |1t+s'||z}t%j,|||j( }t/|tr |j0n|}t3|j||| | }t5|j||| }|j7||f|| }||z}t8j:j=||j<|j }| rdnd}|rdnd}|r|dnd}t?|j@D]k\}}| r||fz }|jr%t%jBg}||jDkr?||||||||| }|d}|sW||dfz }|c||dfz }m|jG|}| r||fz }| stId|||||fDStK|||||S)a~ Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) encoder_hidden_states (`torch.FloatTensor` of shape `(batch_size, encoder_sequence_length, hidden_size)`, *optional*): Sequence of hidden-states at the output of the last layer of the encoder. Used in the cross-attention of the decoder. encoder_attention_mask (`torch.LongTensor` of shape `(batch_size, encoder_sequence_length)`, *optional*): Mask to avoid performing cross-attention on padding tokens indices of encoder input_ids. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): It is a [`~cache_utils.Cache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache). Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `decoder_input_ids` of shape `(batch_size, sequence_length)`. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence. It is used to update the cache in the correct position and to infer the complete sequence length. NzTYou cannot specify both decoder_input_ids and decoder_inputs_embeds at the same timer#zEYou have to specify either decoder_input_ids or decoder_inputs_embedsz[`use_cache=True` is incompatible with gradient checkpointing`. Setting `use_cache=False`...F)rurr)rurrZrr)rurrZr)r9rbr)rrrrrrr;c3$K|]}|| ywr2rrs r*rz,BlenderbotDecoder.forward..4s=s)rrrrcross_attentions)&rurrrrr'r%rrerrrdrzr{is_encoder_decoderr r get_seq_lengthr>r?r=ronesrrrr rrrhr\r rr rrrr)r5rrZrrrrrrrr rr]inputr  batch_size seq_lengthr8mask_seq_lengthself_attn_cache causal_maskr9rall_hidden_statesall_self_attnsall_cross_attentionsr decoder_layerrrs r*rBzBlenderbotDecoder.forwardzsH2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKK(KTaThThN  !*B*D4zAO"ZZ OML`L`aN/+>?  0 0   );;'))+  ";;;'1"7 " ++  $&<>, & 4  --mt||VZVcVc-d #7BD0d&7rKrBrLrMs@r*rrXsR/6"#!.2E  t+E r,rceZdZdddZdeffd ZdZdZe dde jdzd e jdzd e jdzd e jdzd e e zdzd edzde jdzde jdzdedzdedzdedzdedzde jdzde e jezfdZxZS)BlenderbotModelz shared.weight)zencoder.embed_tokens.weightzdecoder.embed_tokens.weightrucJt|||j|j}}|jrt j |jnd}t||j|||_ t||_ t||_ |jy)NrTr)r3r4r rrrrrrOsharedrencoderrdecoderr)r5rurPrrQr6s r*r4zBlenderbotModel.__init__Is}  "("5"5v7H7HZ 393I3Idii/s 3JP[itu (0 (0  r,c|jSr2)r0r5s r*get_input_embeddingsz$BlenderbotModel.get_input_embeddingsUs {{r,c~||_|j|j_|j|j_yr2)r0r1rr2r5rYs r*set_input_embeddingsz$BlenderbotModel.set_input_embeddingsXs) $(KK !$(KK !r,NrrZrdecoder_attention_maskencoder_outputsrrdecoder_inputs_embedsrrrr rrc | | n|jj} | | n|jj} | | n|jj} | | n|jj} ||j |||| | | }nI| rGt |ts7t|dt|dkDr|dndt|dkDr|dnd}|j|||d|||| | | | |  }| s||zSt|j|j|j|j|j|j|j|jS) a| decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) Blenderbot uses the `bos_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). decoder_attention_mask (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. Example: ```python >>> from transformers import AutoTokenizer, BlenderbotModel >>> model = BlenderbotModel.from_pretrained("facebook/blenderbot-400M-distill") >>> tokenizer = AutoTokenizer.from_pretrained("facebook/blenderbot-400M-distill") >>> inputs = tokenizer("Studies have been shown that owning a dog is good for you", return_tensors="pt") >>> decoder_input_ids = tokenizer("Studies show that", return_tensors="pt").input_ids # Batch size 1 >>> outputs = model(input_ids=inputs.input_ids, decoder_input_ids=decoder_input_ids) >>> last_hidden_states = outputs.last_hidden_state >>> list(last_hidden_states.shape) [1, 6, 1280] ```N)rrZrrrr rrr;r rrZrrrrrrrr r)rrdecoder_hidden_statesdecoder_attentionsrencoder_last_hidden_staterencoder_attentions)rurrrrr1rrlenr2rrrrrr)r5rrZrr9r:rrr;rrrr rr]decoder_outputss r*rBzBlenderbotModel.forward]sd2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKK LongTensorrKrrr FloatTensorrrrBrLrMs@r*r.r.Bsw(7'6 / 0 .2.259:>:>(,-1:>!%)-,0#'.2d ##d*d  t+d !++d2 d !& 0 04 7 d 047 d d ||d*d  %0047d $;d  $;d #Tkd D[d  t+d u  !$6 6!d d r,r.z\ The Blenderbot Model with a language modeling head. Can be used for summarization. ) custom_introc!eZdZdZdgZddiZdeffd Z dded edzd e d e jffd Z ded dfd Z e ddej dzdej"dzdej dzdej dzdeezdzdedzdej"dzdej*dzdej dzde dzde dzde dzde dzdej"dzd eej*ezfdZxZS)rrrlm_head.weightzmodel.shared.weightrucxt||t||_|j dt j d|jjjftj|j|jjjd|_ |jy)NrrFrx)r3r4r.rregister_bufferr>zerosr0r/rr|rlm_headrrs r*r4z+BlenderbotForConditionalGeneration.__init__s  $V,  0%++q$**BSBSBbBb>c2deyy1B1B1Q1QX]^  r,Nnew_num_tokenspad_to_multiple_of mean_resizingrczt||||}|j|jjd|S)Nr)r3resize_token_embeddings_resize_final_logits_biasrAr%)r5rNrOrPnew_embeddingsr6s r*rRz:BlenderbotForConditionalGeneration.resize_token_embeddingss?8I[]jk &&~'<'<'B'B1'EFr,c6|jjd}||kr|jddd|f}nStjd||z f|jj}tj |j|gd}|j d|y)Nr#rrr`r)rr%r>rLr=catrK)r5rNold_num_tokensnew_bias extra_biass r*rSz>> from transformers import AutoTokenizer, BlenderbotForConditionalGeneration >>> mname = "facebook/blenderbot-400M-distill" >>> model = BlenderbotForConditionalGeneration.from_pretrained(mname) >>> tokenizer = AutoTokenizer.from_pretrained(mname) >>> UTTERANCE = "My friends are cool but they eat too many carbs." >>> print("Human: ", UTTERANCE) Human: My friends are cool but they eat too many carbs. >>> inputs = tokenizer([UTTERANCE], return_tensors="pt") >>> reply_ids = model.generate(**inputs) >>> print("Bot: ", tokenizer.batch_decode(reply_ids, skip_special_tokens=True)[0]) Bot: That's unfortunate. Are they trying to lose weight or are they just trying to be healthier? >>> REPLY = "I'm not sure" >>> print("Human: ", REPLY) Human: I'm not sure >>> NEXT_UTTERANCE = ( ... "My friends are cool but they eat too many carbs. That's unfortunate. " ... "Are they trying to lose weight or are they just trying to be healthier? " ... " I'm not sure." ... ) >>> inputs = tokenizer([NEXT_UTTERANCE], return_tensors="pt") >>> next_reply_ids = model.generate(**inputs) >>> print("Bot: ", tokenizer.batch_decode(next_reply_ids, skip_special_tokens=True)[0]) Bot: I see. Well, it's good that they're trying to change their eating habits. ``` NzJThe `use_cache` argument is changed to `False` since `labels` is provided.F) rZrr:r9rrr;rrrr rrr#r) losslogitsrr>r?rr@rrA)rurrzwarningr+r r!rrMrrrrrrr>r?rr@rrA)r5rrZrr9r:rrr;rZrrrr rr]r lm_logitsmasked_lm_lossloss_fctoutputs r*rBz*BlenderbotForConditionalGeneration.forwardsN&1%rErKrrr rFrrBrLrMs@r*rrs  ':&;#//ae!7:TzY] <<<.2.259:>:>(,-1:>*.!%)-,0#'.2u ##d*u  t+u !++d2 u !& 0 04 7 u 047 u u ||d*u  %0047u   4'u $;u  $;u #Tku D[u  t+u " u  !O 3#u u r,rc(eZdZdZfdZdZxZS)BlenderbotDecoderWrapperz This wrapper class is a helper class to correctly load pretrained checkpoints when the causal language model is used in combination with the [`EncoderDecoderModel`] framework. cdt||t||_|j yr2)r3r4rr2rrs r*r4z!BlenderbotDecoderWrapper.__init__js&  (0  r,c&|j|i|Sr2)r2)r5argsr]s r*rBz BlenderbotDecoderWrapper.forwardost||T,V,,r,)rErFrGrHr4rBrLrMs@r*rfrfds  -r,rfceZdZddiZfdZdZdZe ddejdzdejdzd ejdzd ejdzd e dzd ejdzd ejdzde dzde dzde dzde dzdejdzdeejzdeezfdZxZS)BlenderbotForCausalLMrIz!model.decoder.embed_tokens.weightcd|_d|_t| |t ||_t j|j|jd|_ |jy)NTFrx) rrr r3r4rfrrr| hidden_sizerrMrrs r*r4zBlenderbotForCausalLM.__init__ysX $)!  -f5 yy!3!3V5F5FUS  r,cB|jjjSr2rr2rr4s r*r5z*BlenderbotForCausalLM.get_input_embeddingsszz!!...r,c:||jj_yr2ror7s r*r8z*BlenderbotForCausalLM.set_input_embeddingss*/ 'r,NrrZrrrrrZrrrr rlogits_to_keeprc | | n|jj} | | n|jj} | | n|jj} |jj |||||||| | | |  }|d}t | tr t| dn| }|j|dd|ddf}d}|a|j|j}t}||jd|jj|jd}| s|f|ddz}||f|zS|St|||j |j"|j$|j&S)ah labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, BlenderbotForCausalLM >>> tokenizer = AutoTokenizer.from_pretrained("facebook/blenderbot-400M-distill") >>> model = BlenderbotForCausalLM.from_pretrained("facebook/blenderbot-400M-distill") >>> assert model.config.is_decoder, f"{model.__class__} has to be configured as a decoder." >>> inputs = tokenizer("Hello, my dog is cute", return_tensors="pt") >>> outputs = model(**inputs) >>> logits = outputs.logits >>> expected_shape = [1, inputs.input_ids.shape[-1], model.config.vocab_size] >>> list(logits.shape) == expected_shape True ```Nr=rr#r)r\r]rrrr)rurrrrr2rrIslicerMtor=rrrrrrrr)r5rrZrrrrrZrrrr rrqr]rr slice_indicesr]r\rarbs r*rBzBlenderbotForCausalLM.forwardsR2C1N-TXT_T_TqTq$8$D $++JjJj &1%SV8W~ot4]k mA}a,?@A  YYv}}-F')HFKKDKK,B,BCV[[QS_UDY,F'+'7D7V# CV C0#33!//))$55   r,) NNNNNNNNNNNNr)rErFrGrDr4r5r8rr>rErKrFr rrIrrrBrLrMs@r*rkrktse= /0.2.2:>;?(,26*.!%)-,0#'26-.S ##d*S  t+S  %0047 S !& 1 1D 8 S  S ((4/S   4'S $;S  $;S #TkS D[S ((4/S ell*S 2 2!S S r,rk)rkrr.r)Nr)CrHrcollections.abcrr>rtorch.nnrrr activationsr cache_utilsr r r generationr masking_utilsr rmodeling_flash_attention_utilsrmodeling_layersrmodeling_outputsrrrrrmodeling_utilsrrprocessing_utilsrutilsrrrrconfiguration_blenderbotr get_loggerrErzrKrIr+rdr.rOModulerUrmrorrrrrr.rrfrk__all__rr,r*rs $ %&!CC)JB9G& 7   H % %,, c [^  -2<<-* =BLL =(!% II% <<% % << % LL4' % T\ %%'(%:z)"))z)|8+78+xh7hV4Q 1Q hg 1g T /  D V )BOV  V t -8 - j 5j Z r,