qiBdZddlZddlmZddlZddlZddlmZddlm Z ddl m Z ddl mZdd lmZmZmZdd lmZdd lmZmZdd lmZdd lmZddlmZmZmZm Z m!Z!ddl"m#Z#m$Z$ddl%m&Z&ddl'm(Z(m)Z)m*Z*m+Z+ddl,m-Z-e+j\e/Z0dejbde2de2fdZ3GddejhZ5 d:dejldejbdejbdejbdejbdzde7dzd e7d!e&e(fd"Z8Gd#d$ejlZ9Gd%d&eZ:Gd'd(eZ;e)Gd)d*e$Z<Gd+d,e<Z=Gd-d.e<Z>e)Gd/d0e<Z?e)d12Gd3d4e )D92D> 9D> input_ids_shapepast_key_values_length position_idsc|F|dd\}}tj|||ztj|jj}t ||S)z3`input_ids_shape` is expected to be [bsz x seqlen].Nr:)r;device)rBarangelongr=rSr5forward)r7rOrPrQbszseq_lenr8s r*rVz+MarianSinusoidalPositionalEmbedding.forwardZs]  *2A.LC <<&(>(HPUPZPZcgcncncucuLw|,,r,N)rN) __name__ __module__ __qualname____doc__intr6rNrBno_gradSizeTensorrV __classcell__r8s@r*r.r.EsNEcE#ECRVJEbfE U]]_pt -$zz -CF -Z_ZfZfimZm -  - -r,r.modulequerykeyvalueattention_maskscalingdropoutkwargsc ||jddz}tj||jdd|z}|||z}tj j |d}tj j|||j}tj||} | jddj} | |fS)Nr#r:rrHptrainingr) sizerBmatmul transposer functionalsoftmaxrjrq contiguous) rdrerfrgrhrirjrk attn_weights attn_outputs r*eager_attention_forwardrzhs**R.D(<<s}}Q':;gEL!#n4 ==((2(>L==((6??([L,,|U3K''1-88:K  $$r,cZeZdZdZ ddedededededed edzd edzffd Z dd e jd e jdzde dzde jdzdede jdzde e dee je jdzee jdzffdZxZS)MarianAttentionz=Multi-headed attention from 'Attention Is All You Need' paperN embed_dim num_headsrj is_decoderbias is_causalconfig layer_idxc t |||_||_||_||z|_||_|j |z|jk7rtd|jd|d|j dz|_||_ ||_ ||_ |9|jr-tjd|jjdt!j"||||_t!j"||||_t!j"||||_t!j"||||_y)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).rmzInstantiating a decoder z without passing `layer_idx` is not recommended and will lead to errors during the forward call, if caching is used. Please make sure to provide a `layer_idx` when creating this class.r)r5r6r}r~rjhead_dimrr'rirrrlogger warning_oncer8rZrLineark_projv_projq_projout_proj) r7r}r~rjrrrrrr8s r*r6zMarianAttention.__init__s$ "" !Y.  MMI %$.. 8MdnnM]$YKr3 }}d* $""     *4>>+B+B*CD,,  ii 94@ ii 94@ ii 94@  )YTB r, hidden_stateskey_value_statespast_key_valuesrhoutput_attentionscache_positionrkr2c |du}|jdd\} } |r|jdn| } | | d|jf} | | d|jf} |j|j| j dd}d}|St |t rA|jj|j}|r |j}n|j}n|}|r|n|}|rK|I|rGj|jj}|j|jj}n|j|}|j!|}|j| j dd}|j| j dd}|T|s|nd}j#|||jd|i\}}|r)t |t rd|j|j<t%j&|j(j*t,}||||||f|j.sdn |j0|j2|d |\}}|j5| | dj7}|j9|}||fS) z#Input shape: Batch x Time x ChannelNr#rr:FrT)rjrir)r%rrviewrt isinstancer is_updatedgetrcross_attention_cacheself_attention_cachelayerskeysvaluesrrupdater get_interfacer_attn_implementationrzrqrjrireshaperwr)r7rrrrhrrrkis_cross_attentionrWtgt_lensrc_len q_input_shapekv_input_shape query_statesrcurr_past_key_valuescurrent_states key_states value_statesattention_interfaceryrxs r*rVzMarianAttention.forwards{ .T9%**3B/ W/A"((+wgr4==9 wDMM:7t{{=166 FPPQRTUV   &/+>?,77;;DNNK %+:+P+P(+:+O+O('6$-?)] /"=*-44T^^DIIJ/66t~~FMML^4J;;~6L(.9CCAqIJ,<,,n=GG1ML*7It+?+F+F dnn?OQ_>`,( L&*_FY*ZAEO..t~~>(?(M(M KK , ,.E) %8      % $}}C$,,LL/ %  % ! \"))#w;FFH mmK0 L((r,)rFTFNN)NNNFN)rZr[r\r]r^floatboolrr6rBrar rrtuplerVrbrcs@r*r|r|s=G  &* $%C%C%C %C  %C  %C%Ct#%C:%CT15(,.2"'.2P)||P) ,,-P) P)  t+ P)  P) t+P)-.P) u||U\\D0% 2E2LL MP)r,r|c eZdZd dededzffd Z d dejdejdedzde ejejdzffd Z xZ S) MarianEncoderLayerNrrcjt||j|_t |j|j |j |||_tj|j|_ |j|_ t|j|_|j|_tj |j|j"|_tj |j"|j|_tj|j|_y)N)r}r~rjrr)r5r6d_modelr}r|encoder_attention_headsattention_dropout self_attnr LayerNormself_attn_layer_normrjractivation_function activation_fnactivation_dropoutrencoder_ffn_dimfc1fc2final_layer_normr7rrr8s r*r6zMarianEncoderLayer.__init__s (nn44,,  %'LL$@!~~ #F$>$>?"(";";99T^^V-C-CD99V33T^^D " T^^ <r,rrhrr2cZ|}|j|||\}}tjj||j|j}||z}|j |}|}|j |j|}tjj||j|j}|j|}tjj||j|j}||z}|j|}|jtjk(rhtj|jsEtj |jj"dz }tj$|| |}|f}|r||fz }|S)a Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`): attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. )rrhrroi)minmax)rrrurjrqrrrrrrr;rBfloat16isfiniteallfinforclamp)r7rrhrresidualrx clamp_valueoutputss r*rVzMarianEncoderLayer.forwardsu!&*nn')/'5' # |  --mt||VZVcVc-d  =0 11-@  **488M+BC  --mt?V?Vaeanan-o /  --mt||VZVcVc-d  =0 --m<   %-- /}8U8Y8Y8[++m&9&9:>>EK!KK K<[YM "    &Gr,rY)F) rZr[r\rr^r6rBrDrrrVrbrcs@r*rrsu=|=d =.*/ *((*))* $; * u  %"3"3d":: ; *r,rcPeZdZddededzffd Z ddejdejdzdejdzdejdzd edzd e dzd e dzd ejdzd e eje ejejfdzffdZ xZ S)MarianDecoderLayerNrrc t||j|_t |j|j |j dd|||_|j|_t|j|_ |j|_ tj|j|_t |j|j |j d|||_tj|j|_tj$|j|j&|_tj$|j&|j|_tj|j|_y)NT)r}r~rjrrrr)rjrrr)r5r6rr}r|decoder_attention_headsrrrjrrrrrrr encoder_attnencoder_attn_layer_normrdecoder_ffn_dimrrrrs r*r6zMarianDecoderLayer.__init__Fs  (nn44,, ~~ #F$>$>?"(";";$&LL$@!+ NN  * *,,  (*||DNN'C$99T^^V-C-CD99V33T^^D " T^^ <r,rrhencoder_hidden_statesencoder_attention_maskrr use_cacherr2c .|} |j|||||\}} tjj||j|j}| |z}|j |}d} |h|} |j ||||||\}} tjj||j|j}| |z}|j|}|} |j|j|}tjj||j|j}|j|}tjj||j|j}| |z}|j|}|f} |r| | | fz } | S)a Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`): attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. encoder_hidden_states (`torch.FloatTensor`): cross attention input to the layer of shape `(batch, seq_len, embed_dim)` encoder_attention_mask (`torch.FloatTensor`): encoder attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. past_key_values (`Cache`): cached past key and value projection states output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence. It is used to update the cache in the correct position and to infer the complete sequence length. )rrrhrrroN)rrrhrrr) rrrurjrqrrrrrrrr) r7rrhrrrrrrrself_attn_weightscross_attn_weightsrs r*rVzMarianDecoderLayer.forwardes8!,0>>'+)/) ,:, ( ( --mt||VZVcVc-d  =0 11-@ " ,$H040A0A+!65 /"3- 1B1 -M-MM11-4< >Gr,rY)NNNNFTN)rZr[r\rr^r6rBrar rrrDrVrbrcs@r*rrEs=|=d =D/3596:(,).!%.2I||I t+I %||d2 I !& t 3 I  I $;I$;I t+I u  %(9(95;L;L(L"MPT"TT UIr,rczeZdZUeed<dZdZdZdZdZ dZ e jfdZ edZxZS)MarianPreTrainedModelrmodelTct||t|tr/t j |j |jyt|tr t j|jyyrY) r5 _init_weightsrr.initcopy_r=rN MarianMTModelzeros_final_logits_bias)r7rdr8s r*rz#MarianPreTrainedModel._init_weightssW f% fA B JJv}}f&:&:&< =  . KK00 1/r,c|jj}tjgddddd|gg|j}|j |||d}|S)N)r r:r r:rS)rhrdecoder_input_ids)rr rBtensorrSne)r7 pad_tokenr dummy_inputss r*rz"MarianPreTrainedModel.dummy_inputssZKK,, LL"2Q2q)4L!MVZVaVab 'll95"!* r,)rZr[r\r__annotations__base_model_prefixsupports_gradient_checkpointing_supports_flash_attn_supports_sdpa_supports_flex_attn_can_compile_fullgraphrBr_rpropertyrrbrcs@r*rrsZ &*#N!U]]_22r,rceZdZdZdeffd Z d dejdzdejdzdejdzde dzd e dzd e dzd e eje zfd Z xZS) MarianEncoderz Transformer encoder consisting of *config.encoder_layers* self attention layers. Each layer is a [`MarianEncoderLayer`]. Args: config: MarianConfig embed_tokens (nn.Embedding): output embedding rct|||j|_|j|_|j }|j |_|j|_ |jrtj|nd|_ tj|j ||j|_t%|j||j|_tj(t+|j,Dcgc] }t/|c}|_d|_|j5ycc}w)N?F)r5r6rjencoder_layerdrop layerdroprr r1max_position_embeddingsmax_source_positionsscale_embeddingmathsqrt embed_scaler Embedding vocab_size embed_tokensr.embed_positions ModuleListr@encoder_layersrrgradient_checkpointing post_init)r7rr}_r8s r*r6zMarianEncoder.__init__s  ~~ 11NN !..$*$B$B!393I3I499Y/sLL):):ItGWGWXB  * *It7G7G mmvOdOdIe$fA%7%?$fg &+#  %gsD;Nrrh inputs_embedsroutput_hidden_states return_dictr2c ||n|jj}||n|jj}||n|jj}| | t d|8|j |||j }|jd|d}n!||j dd}n t d||j||jz}|j|} || z} tjj| |j|j} t|j||}|rdnd} |rdnd} t!|j"D]b\} }|r| | fz} d}|jr&t%j&g}||j(krd }|rd }n|| || }|d } |sZ| |d fz} d|r| | fz} |st+d| | | fDSt-| | | S)a8 Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. NzDYou cannot specify both input_ids and inputs_embeds at the same timer#z5You have to specify either input_ids or inputs_embedsro)rr rhFT)NN)rrrc3&K|] }|| ywrYr.0vs r* z(MarianEncoder.forward..WseqWXWdeslast_hidden_stater attentions)rrr use_return_dictr'%warn_if_padding_and_no_attention_maskrrrrrrrrurjrqr enumeraterrBrandrrr)r7rrhr rr rrk input_shape embed_posrencoder_statesall_attentionsidx encoder_layerto_dropdropout_probability layer_outputss r*rVzMarianEncoder.forwards-P2C1N-TXT_T_TqTq$8$D $++JjJj &1%cd d  "  6 6y. Q#..*K!r;r?;I  &',,.s3KTU U   --i84;K;KKM((5 % 1  --mt||VZVcVc-d 2;;')  40d"+DKK"8 F C#!/=2B!BG}}&+jjn#&7"G , -!"&7! !.a 0  !/=3C2E!E- F0 +}.>>Ne]NN$See e+>Vd  r,)NNNNNN)rZr[r\r]rr6rB LongTensorrDrrrarrVrbrcs@r*rrs|0.22626)-,0#'j ##d*j ((4/j ((4/ j  $; j #Tk j D[j  u||  .j r,rcReZdZdZdeffd Z ddejdzdejdzdejdzdejdzd e dzd ejdzd e dzd e dzd e dzde dzdejdzde eje zfdZxZS) MarianDecoderz Transformer decoder consisting of *config.decoder_layers* layers. Each layer is a [`MarianDecoderLayer`] Args: config: MarianConfig embed_tokens (nn.Embedding): output embedding rc t|||j|_|j|_|j |_|j|_|jrtj|jnd|_ tj|j |j|j |_t%|j|j|j |_tj(t+|j,Dcgc]}t/||c}|_d|_|j5ycc}w)Nr)rF)r5r6rjdecoder_layerdroprr r1rmax_target_positionsrrrrrrrdecoder_vocab_sizerr.rrr@decoder_layersrrr r )r7rir8s r*r6zMarianDecoder.__init__fs  ~~ 11!..$*$B$B!8>8N8N499V^^4TWLL)B)BFNNTXTdTdeB  * *FNND|j }| rd nd}|rd nd}|r|d nd}tA|jBD]k\}}| r||fz }|j r%t'jDg}||jFkr?||||||||| }|d}|sW||dfz }|c||dfz }m| r||fz }| stId|||||fDStK|||||S)a~ Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) encoder_hidden_states (`torch.FloatTensor` of shape `(batch_size, encoder_sequence_length, hidden_size)`, *optional*): Sequence of hidden-states at the output of the last layer of the encoder. Used in the cross-attention of the decoder. encoder_attention_mask (`torch.LongTensor` of shape `(batch_size, encoder_sequence_length)`, *optional*): Mask to avoid performing cross-attention on padding tokens indices of encoder input_ids. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): It is a [`~cache_utils.Cache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache). Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `decoder_input_ids` of shape `(batch_size, sequence_length)`. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence. It is used to update the cache in the correct position and to infer the complete sequence length. NzZ`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...Fz:You must specify exactly one of input_ids or inputs_embedsr#)rrr)rr rhrr)rr rhr)rQror)rrrrrrr:c3$K|]}|| ywrYrrs r*rz(MarianDecoder.forward..0s=s)rrrrcross_attentions)&rrr rrr rqrrr'r%rrrrris_encoder_decoderr r get_seq_lengthrBrTrSronesrrrr rrrurjrrrrrr)r7rrhrrrr rrr rrrkinputr batch_size seq_lengthrPmask_seq_lengthself_attn_cache causal_maskrQrall_hidden_statesall_self_attnsall_cross_attentionsr! decoder_layerr$r%s r*rVzMarianDecoder.forwardysH2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKK(KTaThThN  !*B*D4zAO"ZZ OML`L`aN/+>?  0 0   );;'))+  ";;;'1"7 " ++  $&<>, & 4  --mt||VZVcVc-d #7BD0d&7:>(,26!%)-,0#'.2B ##d*B  t+B  %0047 B !& 0 04 7 B  B ((4/B $;B  $;B #TkB D[B  t+B  u|| H HB r,r(ceZdZddgZdeffd ZdZdZdZdZ d e d e jfd Z e dd ej d zdej"d zdej d zdej"d zdeej"ezd zded zdej*d zdej*d zded zded zded zded zdej"d zd efdZxZS) MarianModel$model.encoder.embed_positions.weight$model.decoder.embed_positions.weightrcPt|||j|j}}|jj r1t j||j||_ ddd|_ nd|_ t||_ t||_|jy)Nz shared.weight)zdecoder.embed_tokens.weightzencoder.embed_tokens.weight)r5r6r rr share_encoder_decoder_embeddingsrrrshared_tied_weights_keysrencoderr(decoderr )r7rr1rr8s r*r6zMarianModel.__init__Es  "("5"5v7H7HZ  ;; 7 7,,z6>>;ODK/>/>'D # '+D #$V, $V,  r,c>|jjSrY) get_encoderget_input_embeddingsr7s r*rKz MarianModel.get_input_embeddingsZs!6688r,c|jjr>||_|j|j_|j|j _y||j_yrY)rrDrErGrrHr7rgs r*set_input_embeddingsz MarianModel.set_input_embeddings^sB ;; 7 7DK(, DLL %(, DLL %(-DLL %r,c|jjr td|jj S)Nz`get_decoder_input_embeddings` should not be called if `config.share_encoder_decoder_embeddings` is `True`. Please use `get_input_embeddings` instead.)rrDr' get_decoderrKrLs r*get_decoder_input_embeddingsz(MarianModel.get_decoder_input_embeddingsfs< ;; 7 7H !6688r,ch|jjr td||j_y)Na `config.share_encoder_decoder_embeddings` is set to `True` meaning the decoder input embeddings are shared with the encoder. In order to set the decoder input embeddings, you should simply set the encoder input embeddings by calling `set_input_embeddings` with the appropriate embeddings.)rrDr'rHrrNs r*set_decoder_input_embeddingsz(MarianModel.set_decoder_input_embeddingsns0 ;; 7 7r  %* !r,new_num_tokensr2c|jjr td|j}|j ||}|j ||j}||S||j_|j|SNz`resize_decoder_token_embeddings` should not be called if `config.share_encoder_decoder_embeddings` is `True`. Please use `resize_token_embeddings` instead.)rrDr'rR_get_resized_embeddingsrTr, tie_weights)r7rUold_embeddingsnew_embeddings model_embedss r*resize_decoder_token_embeddingsz+MarianModel.resize_decoder_token_embeddingsws ;; 7 7K  ::<55nnU )).988:  ! *8 & r,Nrrhrdecoder_attention_maskencoder_outputsrr decoder_inputs_embedsrrr rrc v| | n|jj} | | n|jj} | | n|jj} ||j |||| | | }nI| rGt |t s7t |dt|dkDr|dndt|dkDr|dnd}|j|||d|||| | | | |  }| s||zSt|j|j|j|j|j|j|j|jS) a decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) Marian uses the `pad_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). decoder_attention_mask (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. Example: ```python >>> from transformers import AutoTokenizer, MarianModel >>> tokenizer = AutoTokenizer.from_pretrained("Helsinki-NLP/opus-mt-en-de") >>> model = MarianModel.from_pretrained("Helsinki-NLP/opus-mt-en-de") >>> inputs = tokenizer("Studies have been shown that owning a dog is good for you", return_tensors="pt") >>> decoder_inputs = tokenizer( ... " Studien haben gezeigt dass es hilfreich ist einen Hund zu besitzen", ... return_tensors="pt", ... add_special_tokens=False, ... ) >>> outputs = model(input_ids=inputs.input_ids, decoder_input_ids=decoder_inputs.input_ids) >>> last_hidden_states = outputs.last_hidden_state >>> list(last_hidden_states.shape) [1, 26, 512] ```N)rrhr rr rrrr:r rrhrrrr rrr rr)rrdecoder_hidden_statesdecoder_attentionsr1encoder_last_hidden_staterencoder_attentions)rrr rrGrrlenrHrrrrrr1)r7rrhrr^r_rr r`rrr rrrkdecoder_outputss r*rVzMarianModel.forwardsml2C1N-TXT_T_TqTq$8$D $++JjJj &1%s /.'# |*9.9*cbll0.2.2596:HL(,26:>!%)-,0#'.2h ##d*h  t+h !++d2 h !& t 3 h u||,>E h h ((4/h  %0047h $;h  $;h #Tkh D[h  t+h !h h r,r@zX The Marian Model with a language modeling head. Can be used for summarization. ) custom_introc veZdZdZgdZddgZddiZdeffd Z d$d e d e d zd e d e jffd Z d%d e d e jfdZdZd e d d fdZde jfdZe d&dej(d zdej*d zdej(d zdej*d zdeej*ezd zded zdej2d zdej2d zdej(d zde d zde d zde d zd e d zd!ej*d zd efd"Zdej*fd#ZxZS)'rr)rrArBrArBlm_head.weight!model.decoder.embed_tokens.weightrct||t||_|jj r dddd|_|j r |jn |j}|jdtjd|ftj|j|d|_|j!y)Nzmodel.shared.weight)rlrmz!model.encoder.embed_tokens.weightrrFr)r5r6r@rrrDrFrr,register_bufferrBzerosrrrlm_headr )r7rtarget_vocab_sizer8s r*r6zMarianMTModel.__init__ s   ( ;; 7 7"75J5J'D # 281X1XF--^d^w^w 0%++qBS>T2UVyy1BO  r,NrUpad_to_multiple_of mean_resizingr2cxt||||}|jjr|j ||SrY)r5resize_token_embeddingsrrD_resize_final_logits_bias)r7rUrsrtr[r8s r*rvz%MarianMTModel.resize_token_embeddingss;8I[]jk ;; 7 7  * *> :r,c|j}|j|||}|j||jjd}|j j r||j _|j j rY|jI|j js3|j}|j||}|j||jS)Nr) rKrXrOr=r%rrDr,get_output_embeddingstie_word_embeddings_get_resized_lm_headset_output_embeddings)r7rUrsargsrZr[ old_lm_head new_lm_heads r*_resize_token_embeddingsz&MarianMTModel._resize_token_embeddings$s22455nnVhi !!.1'..44Q7 ;; 7 7-;DKK * KK 8 8**,8KK33446K33KPK  & &{ 3((**r,c(|jjr td|jj }|j ||}|jj ||jI|jjs3|j}|j||}|j||jj }||S||j_ |j|j||SrW)rrDr'rrRrXrTryrzr{r|r,rYrw)r7rUrZr[r~rr\s r*r]z-MarianMTModel.resize_decoder_token_embeddings:s ;; 7 7K  @@B55nnU //?  % % ' 3DKK<[<[446K33KPK  & &{ 3zz>>@  ! *8 &  &&~6r,c6|jjd}||kr|jddd|f}nStjd||z f|jj}tj |j|gd}|j d|y)Nr#rrrnr)rr%rBrprScatro)r7rUold_num_tokensnew_bias extra_biass r*rwz'MarianMTModel._resize_final_logits_biasZs//55b9 ^ +--a..@AHa.)H%IRVRhRhRoRopJyy$"8"8*!E1MH 0(;r,r[c||_yrY)rq)r7r[s r*r|z#MarianMTModel.set_output_embeddingscs % r,rrhrr^r_rr r`labelsrrr rrc | | n|jj} | R| rtjdd} |7|5t | |jj |jj }|j||||||||| | | | | }|j|d|jz}d}| Ft}||jd|jj| jd}| s|f|ddz}||f|zS|St|||j|j|j |j"|j$|j&|j( S) u decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) Marian uses the `pad_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). decoder_attention_mask (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, MarianMTModel >>> src = "fr" # source language >>> trg = "en" # target language >>> model_name = f"Helsinki-NLP/opus-mt-{src}-{trg}" >>> model = MarianMTModel.from_pretrained(model_name) >>> tokenizer = AutoTokenizer.from_pretrained(model_name) >>> sample_text = "où est l'arrêt de bus ?" >>> batch = tokenizer([sample_text], return_tensors="pt") >>> generated_ids = model.generate(**batch) >>> tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0] "Where's the bus stop?" ``` NzJThe `use_cache` argument is changed to `False` since `labels` is provided.F) rhrr_r^rr r`rrr rrrr#r) losslogitsrrcrdr1rerrf)rrrwarningr+r r!rrqrrrr,rrrcrdr1rerrf)r7rrhrr^r_rr r`rrrr rrrkr lm_logitsmasked_lm_lossloss_fctoutputs r*rVzMarianMTModel.forwardfsv&1%*.!%)-,0#'.2i ##d*i  t+i !++d2 i !& t 3 i u||,>E i i ((4/i  %0047i   4'i $;i  $;i #Tki D[i  t+i " #i i VhELLhr,rc(eZdZdZfdZdZxZS)MarianDecoderWrapperz This wrapper class is a helper class to correctly load pretrained checkpoints when the causal language model is used in combination with the [`EncoderDecoderModel`] framework. cdt||t||_|j yrY)r5r6r(rHr r7rr8s r*r6zMarianDecoderWrapper.__init__s&  $V,  r,c&|j|i|SrY)rH)r7r}rks r*rVzMarianDecoderWrapper.forwardst||T,V,,r,)rZr[r\r]r6rVrbrcs@r*rrs  -r,rceZdZddiZfdZdZdZe ddejdzdejdzd ejdzd ejdzd e dzd ejdzd ejdzde dzde dzde dzde dzdejdzdeejzdeezfdZxZS)MarianForCausalLMrlrmcd|_d|_t| |t ||_t j|j|jd|_ |jy)NTFr) rr2r5r6rrrr hidden_sizerrqr rs r*r6zMarianForCausalLM.__init__sX $)!  )&1 yy!3!3V5F5FUS  r,cB|jjjSrYrrHrrLs r*rKz&MarianForCausalLM.get_input_embeddingsszz!!...r,c:||jj_yrYrrNs r*rOz&MarianForCausalLM.set_input_embeddingss*/ 'r,Nrrhrrrr rrrr rrlogits_to_keepr2c | | n|jj} | | n|jj} | | n|jj} |jj |||||||| | | |  }|d}t | tr t| dn| }|j|dd|ddf}d}|a|j|j}t}||jd|jj|jd}| s|f|ddz}||f|zS|St|||j |j"|j$|j&S)aT labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, MarianForCausalLM >>> tokenizer = AutoTokenizer.from_pretrained("Helsinki-NLP/opus-mt-fr-en") >>> model = MarianForCausalLM.from_pretrained("Helsinki-NLP/opus-mt-fr-en") >>> assert model.config.is_decoder, f"{model.__class__} has to be configured as a decoder." >>> inputs = tokenizer("Hello, my dog is cute", return_tensors="pt") >>> outputs = model(**inputs) >>> logits = outputs.logits >>> expected_shape = [1, inputs.input_ids.shape[-1], model.config.vocab_size] >>> list(logits.shape) == expected_shape True ```Nrbrr#r)rrrrrr1)rrr rrrHrr^slicerqtorSrrrrrrrr1)r7rrhrrrr rrrr rrrrkrr slice_indicesrrrrs r*rVzMarianForCausalLM.forwardsR2C1N-TXT_T_TqTq$8$D $++JjJj &1%SV8W~ot4]k mA}a,?@A  YYv}}-F')HFKKDKK,B,BCV[[QS_UDY,F'+'7D7V# CV C0#33!//))$55   r,) NNNNNNNNNNNNr)rZr[r\rFr6rKrOrrBr&rarDr rr^rrrVrbrcs@r*rrse= /0.2.2:>;?(,26*.!%)-,0#'26-.S ##d*S  t+S  %0047 S !& 1 1D 8 S  S ((4/S   4'S $;S  $;S #TkS D[S ((4/S ell*S 2 2!S S r,r)rr@rr)Nr)Dr]rcollections.abcrnumpyr>rBrtorch.nnrrr activationsr cache_utilsr r r generationr masking_utilsr rmodeling_flash_attention_utilsrmodeling_layersrmodeling_outputsrrrrrmodeling_utilsrrprocessing_utilsrutilsrrrrconfiguration_marianr get_loggerrZrrar^r+rr.Modulerrzr|rrrrr(r@rrr__all__rr,r*rsD $ %&!CC)JB9G& /   H % %,, c [^  -",,-R!% II% <<% % << % LL4' % T\ %%'(%:z)biiz)|>3>Di3iXO