qiJdZddlZddlZddlZddlmZddlmZmZmZddl m Z ddl m Z ddlmZmZmZdd lmZdd lmZmZdd lmZdd lmZmZmZmZmZmZm Z dd l!m"Z"ddl#m$Z$m%Z%m&Z&m'Z'm(Z(ddl)m*Z*e'jVe,Z-Gddej\Z/ ddl0m1Z1e1Z/e-jedGddej\Z6Gddej\Z7Gddej\Z8Gddej\Z9Gddej\Z:Gd d!ej\Z;Gd"d#eZ<Gd$d%ej\Z=e&Gd&d'e"Z>Gd(d)e>Z?e&Gd*d+e>Z@e&d,-Gd.d/e>eZAe&Gd0d1e>ZBe&d2-Gd3d4e>ZCe&Gd5d6e>ZDe&Gd7d8e>ZEgd9ZFy#e3$rY5e4$re-jkdYMwxYw):zPyTorch T5 model.N)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)initialization)ACT2FN)Cache DynamicCacheEncoderDecoderCache)GenerationMixin)create_bidirectional_maskcreate_causal_mask)GradientCheckpointingLayer)BaseModelOutput)BaseModelOutputWithPastAndCrossAttentionsSeq2SeqLMOutputSeq2SeqModelOutput#Seq2SeqQuestionAnsweringModelOutputSeq2SeqSequenceClassifierOutputTokenClassifierOutput)PreTrainedModel) DUMMY_INPUTS DUMMY_MASKauto_docstringloggingtorch_compilable_check)T5Configc&eZdZdfd ZdZxZS) T5LayerNormct|tjt j ||_||_y)zc Construct a layernorm module in the T5 style. No bias and no subtraction of mean. N)super__init__r Parametertorchonesweightvariance_epsilon)self hidden_sizeeps __class__s T/opt/pipecat/venv/lib/python3.12/site-packages/transformers/models/t5/modeling_t5.pyr$zT5LayerNorm.__init__/s1 ll5::k#:; #c|jtjjdj dd}|tj ||j zz}|jjtjtjfvr%|j|jj}|j|zS)NT)keepdim) tor&float32powmeanrsqrtr)r(dtypefloat16bfloat16)r* hidden_statesvariances r.forwardzT5LayerNorm.forward7s !##EMM266q9>>r4>P% Ht?T?T4T(UU  ;;   ? ?),,T[[->->?M{{]**r/)gư>)__name__ __module__ __qualname__r$r> __classcell__r-s@r.r!r!.s $ +r/r!) FusedRMSNormzODiscovered apex.normalization.FusedRMSNorm - will use it instead of T5LayerNormzBdiscovered apex but it failed to load, falling back to T5LayerNormc*eZdZdeffd ZdZxZS)T5DenseActDenseconfigc^t|tj|j|j d|_tj|j |jd|_tj|j|_ t|j|_ yNFbias)r#r$rLineard_modeld_ffwiwoDropout dropout_ratedropoutr dense_act_fnactr*rGr-s r.r$zT5DenseActDense.__init__Usn ))FNNFKKeD))FKKeDzz&"5"56 &--.r/c|j|}|j|}|j|}t|jj t jr|j|jj jk7r`|jj jt jk7r/|j|jj j}|j |}|SN) rOrUrS isinstancerPr(r&Tensorr9int8r4r*r<s r.r>zT5DenseActDense.forward\s . /  ]3 tww~~u|| 4##tww~~';';;$$ 2),,TWW^^-A-ABM . r/r?r@rArr$r>rBrCs@r.rFrFTs/x/ r/rFc*eZdZdeffd ZdZxZS)T5DenseGatedActDenserGct|tj|j|j d|_tj|j|j d|_tj|j |jd|_tj|j|_ t|j|_yrI)r#r$rrLrMrNwi_0wi_1rPrQrRrSr rTrUrVs r.r$zT5DenseGatedActDense.__init__ks IIfnnfkkF IIfnnfkkF ))FKKeDzz&"5"56 &--.r/c,|j|j|}|j|}||z}|j|}t |j j tjr|j|j j jk7r`|j j jtjk7r/|j|j j j}|j |}|SrX) rUrarbrSrYrPr(r&rZr9r[r4)r*r< hidden_gelu hidden_linears r.r>zT5DenseGatedActDense.forwardsshhtyy78  -0 #m3  ]3 tww~~u|| 4##tww~~';';;$$ 2),,TWW^^-A-ABM . r/r]rCs@r.r_r_js/x/r/r_c*eZdZdeffd ZdZxZS) T5LayerFFrGct||jrt||_nt ||_t |j|j|_ tj|j|_ y)Nr,)r#r$ is_gated_actr_DenseReluDenserFr!rMlayer_norm_epsilon layer_normrrQrRrSrVs r.r$zT5LayerFF.__init__s_    "6v">D "1&"9D %fnn&:S:STzz&"5"56 r/cr|j|}|j|}||j|z}|SrX)rmrkrS)r*r<forwarded_statess r.r>zT5LayerFF.forwards=??=9../?@% 5E(FF r/r]rCs@r.rgrgs7x7r/rgcfeZdZ ddededzffd Zed dZd dZ d dZ xZ S) T5AttentionNrG layer_idxct||j|_||_|j|_|j |_|j |_|j|_|j|_ |j|_ |j|jz|_ ||_|9|jr-tj!d|j"j$dt'j(|j |jd|_t'j(|j |jd|_t'j(|j |jd|_t'j(|j|j d|_|jr/t'j2|j|j|_d|_y)NzInstantiating a decoder z without passing `layer_idx` is not recommended and will to errors during the forward call, if caching is used. Please make sure to provide a `layer_idx` when creating this class.FrJ)r#r$ is_decoderhas_relative_attention_biasrelative_attention_num_bucketsrelative_attention_max_distancerMd_kvkey_value_proj_dim num_headsn_headsrRrS inner_dimrrlogger warning_oncer-r?rrLqkvo Embeddingrelative_attention_biasgradient_checkpointingr*rGrurrr-s r.r$zT5Attention.__init__si  +++F(.4.S.S+/5/U/U,~~ "(++'' ** (?(??"     *4>>+B+B*CD,,  4<<eD4<<eD4<<eD4>>4<=max_distance map to the same bucket. All relative positions <=-max_distance map to the same bucket. This should allow for more graceful generalization to longer sequences than the model has been trained on Args: relative_position: an int32 Tensor bidirectional: a boolean - whether the attention is bidirectional num_buckets: an integer max_distance: an integer Returns: a Tensor with the same shape as relative_position, containing int32 values in the range [0, num_buckets) rr1r) r4r&longabsmin zeros_likelogfloatmath full_likewhere)relative_position bidirectional num_buckets max_distancerelative_buckets max_exactis_smallrelative_position_if_larges r._relative_position_bucketz%T5Attention._relative_position_buckets(,  A K !2Q!6 : :5:: F T T  % *; < !&+N>NO`>a!b b  1$ $y0&/ II'--/); <hh|i/0 1Y& ( "UZZ. &" &+YY &8RT_bcTc(d& " EKK2CE_``r/c | |jjj}|.tj|tj |dddf}n|dddfj |}tj|tj |dddf}||z }|j||j |j|j}|j|} | jgdjd} | S)z%Compute binned relative position biasN)r9device)rrr)r1rrr) rr(rr&arangerr4rrtrvrwpermute unsqueeze) r* query_length key_lengthrcache_positioncontext_positionmemory_positionrrelative_position_bucketvaluess r. compute_biaszT5Attention.compute_biass >1188??F  !$||L SYZ[\^b[bc -ag699&A ,,zFSTXZ[T[\+.>>#'#A#A #.;;== $B$  --.FG *44Q7 r/c |jdd\} } |du} |j|} | j| d|j|jj dd} d}t |trA|jj|j}| r |j}n|j}n|}| r|n|}| rK|I|rG|j|jj}|j|jj}n|j!|}|j#|}|j| d|j|jj dd}|j| d|j|jj dd}|T| s| nd} |j%|||jd| i\}}| r)t |trd|j|j<t'j(| |j dd}||jd }||n| ddz}|j*sZt'j,d|j| |f|j.|j0 }|j2rE|j4r9d|_n1|j9|||j.| }|dddd| dddf}|#|ddddddd|jd f}||z}|}||z }t:j<j?|jAd jC|}t:j<jE||jD|j4 }t'j(||}|j ddjG}|j| d|jH}|jK|}||f}|r||fz}|S)z Self-attention (if key_value_states is None) or attention over source sentence (provided by key_value_states). Nr1r2rFrTr)rr9)rrdim)ptraining)&shaperviewr{ry transposerYr is_updatedgetrrcross_attention_cacheself_attention_cachelayerskeysrrrupdater&matmulruzerosrr9rr requires_gradrr functionalsoftmaxrtype_asrS contiguousr|r)r*r<maskkey_value_states position_biaspast_key_valuesr use_cacheoutput_attentionsr batch_size seq_lengthis_cross_attention query_statesrcurr_past_key_valuescurrent_states key_states value_statesscoresrreal_seq_length causal_maskposition_bias_masked attn_weights attn_outputoutputss r.r>zT5Attention.forwards""/!4!4Ra!8 J.T9vvm, #((RtG^G^_iijkmno  o': ;(3377GJ!'6'L'L$'6'K'K$#2 -?)] /"=*-44T^^DIIJ/66t~~FMML/J66.1L#RtG^G^_iijkmnoJ',,ZT\\4KbKbcmmnoqrsL*7It+?+F+F dnn?OQ_>`,( L&*_FY*ZAEO..t~~>lJ,@,@A,FG  #))"-J.:.FlN[]L^abLbO33 %  j*=fmm[a[g[g! ..4==26M/ $ 1 1#Z Vd!2! !.aZKL!.C D "1a,Bj.>.>r.B,B#BC - ; ,&&}},,V\\^,DLLVT }},,\T\\TXTaTa,b ll<> !++Aq1<<> !&&z2t~~F ff[)  . /Gr/FN)T )NN)NNNNNFFN) r?r@rArintr$ staticmethodrrr>rBrCs@r.rqrqsa%* $ , ,: ,D- - ^.br/rqc@eZdZddedzffd Z ddZxZS)T5LayerSelfAttentionNrrct|t||||_t |j |j |_tj|j|_ y)Nrurrri) r#r$rq SelfAttentionr!rMrlrmrrQrRrSrs r.r$zT5LayerSelfAttention.__init__fsT ( 0KW` &fnn&:S:STzz&"5"56 r/c |j|}|j|||||||} ||j| dz}|f| ddz} | S)N)rrrrrrrr)rmrrS) r*r<attention_maskrrrrrnormed_hidden_statesattention_outputrs r.r>zT5LayerSelfAttention.forwardnss $}=-- '+/). & 5Ea5H(II  "%5ab%99r/r)NNNFFNr?r@rArr$r>rBrCs@r.rres-7SSWZ7r/rcBeZdZddedzffd Z ddZxZS)T5LayerCrossAttentionNrrct|t|d||_t |j |j |_tj|j|_ y)NFrri) r#r$rqEncDecAttentionr!rMrlrmrrQrRrS)r*rGrrr-s r.r$zT5LayerCrossAttention.__init__sN *6u`ij%fnn&:S:STzz&"5"56 r/c |j|} |j| ||||||||  } ||j| dz} | f| ddz} | S)N)rrrrrrrrrr)rmrrS)r*r<rrrrrrrrrr layer_outputrs r.r>zT5LayerCrossAttention.forwardsx $}=// -'+%/)0  %t||4DQ4G'HH /$4QR$88r/rX)NNNFNFNrrCs@r.rrs/7#*7r/rcHeZdZddedzffd Z ddZxZS)T5BlockNrrcpt||j|_tj|_|j j t||||jr&|j j t|||j j t|y)Nr)rr) r#r$rtr ModuleListlayerappendrrrgrs r.r$zT5Block.__init__s  ++]]_   E`lu v  ?? JJ  3FiP Q )F+,r/c |jd|||||| | } | d}| dd} |jtjk(rtjtj |j tj|jjdz tj|jj}tj|| |}|jxr|du}|r|jd|||||| ddz|| }|d}|jtjk(rtjtj |j tj|jjdz tj|jj}tj|| |}| |ddz} |jd|}|jtjk(rtjtj |j tj|jjdz tj|jj}tj|| |}|f}|| zS)Nr)rrrrrrri)rmaxr2)rrrrrrr) rr9r&r:risinfanyfinforclamprt)r*r<rrencoder_hidden_statesencoder_attention_maskencoder_decoder_position_biasrrr return_dictrself_attention_outputsattention_outputs clamp_valuedo_cross_attentioncross_attention_outputsrs r.r>zT5Block.forwardsl"/A )'+/)" /q1 2126   %-- /++ M*..0 M//044t; M//044K "KK K<[YM!__R1Fd1R &3djjm!65; /+B/!3#"3 ' #4A6M""emm3#kkKK .224KK 3 34884?KK 3 3488 !& M |Q\ ] !24KAB4O O ' 2}5    %-- /++ M*..0 M//044t; M//044K "KK K<[YM " ' ' r/r) NNNNNNFFTNrrCs@r.rrs: -SSWZ -"#&*M r/rc`eZdZdZdeffd ZdejdejfdZxZ S)T5ClassificationHeadz-Head for sentence-level classification tasks.rGc(t|tj|j|j|_tj |j|_tj|j|j|_ y)N)r) r#r$rrLrMdenserQclassifier_dropoutrS num_labelsout_projrVs r.r$zT5ClassificationHead.__init__ sZ YYv~~v~~> zzF$=$=>  &..&2C2CD r/r<returnc|j|}|j|}tj|}|j|}|j |}|SrX)rSrr&tanhrr\s r.r>zT5ClassificationHead.forwardsN ]3  =1  =1  ]3  m4 r/) r?r@rA__doc__rr$r&rZr>rBrCs@r.rrs/7ExE U\\ellr/rcteZdZUeed<dZdZdZdgZdgZ e dZ e jdZdZy ) T5PreTrainedModelrG transformerTrrPcvtjt}tjt}|||d}|S)N)decoder_input_ids input_idsdecoder_attention_mask)r&tensorrr)r*r  input_mask dummy_inputss r.rzT5PreTrainedModel.dummy_inputs$s6LL. \\*- !*"&0 r/cr|jj}t|tr$t j |j |dzy t|ttttfrt j|jj d|dzt|drE|jjs/t j|jj d|dzt|drpt j|j j d||jj"dzzt j$|j j&y y t|t(rft|drYt j|j*j d|dzt j$|j*j&y y t|t,r9t j|j.j d||jj"dzzt|j.dr?|j.j&)t j$|j.j&t j|j0j d||jj"dzzt|j0drA|j0j&*t j$|j0j&y y y t|t2r9t j|j4j d||jj"dzzt|j4dr?|j4j&)t j$|j4j&t j|j6j d||jj8dzzt|j6drA|j6j&*t j$|j6j&y y y t|t:rt j|j<j d||jj"dzzt|j<dr?|j<j&)t j$|j<j&t j|j>j d||jj"dzzt|j>dr?|j>j&)t j$|j>j&t j|j6j d||jj8dzzt|j6drA|j6j&*t j$|j6j&y y y t|t@rP|jj"}|jjB}|jjD}t j|jFj d|||zdzzt j|jHj d||dzzt j|jJj d||dzzt j|jLj d|||zdzz|jNr3t j|jPj d||dzzy y y ) zInitialize the weightsg?g)r7stdlm_head qa_outputs classifierrKN))rGinitializer_factorrYr!init constant_r(T5ModelT5ForConditionalGenerationT5EncoderModelT5ForQuestionAnsweringnormal_sharedhasattrtie_word_embeddingsrrrMzeros_rKT5ForTokenClassificationrrrrrFrOrPrNr_rarbrqrxrzrrrrrur)r*modulefactorrMryr{s r. _init_weightszT5PreTrainedModel._init_weights/s6// fk * NN6==&3, 7   0.BX Y  LL--CVc\ Jvy)$++2Q2Q V^^22&3,Ov|, V..55CVPTP[P[PcPchlOlEmn F--223- 8 9v|, V..55CVc\R F--223- 4 5 LL,,3Ft{{GZGZ_cFc%>%@#qr'"$:&!  PQ Q&&'8D'@,O  r/N)r?r@rAr__annotations__base_model_prefixsupports_gradient_checkpointing_can_compile_fullgraph_no_split_modules_keep_in_fp32_modulespropertyrr&no_gradr'r1r/r.r r s` %&*#!" !F U]]_4p4pl!r/r cBeZdZfdZdZ ddZxZS)T5Stackc t||tj|j|j |_|j|_tjt|jDcgc]}t|t|dk(|c}|_ t|j |j|_tj"|j$|_|j)d|_ycc}w)NrrriF)r#r$rr vocab_sizerM embed_tokensrtrrange num_layersrboolblockr!rlfinal_layer_normrQrRrS post_initr)r*rGir-s r.r$zT5Stack.__init__}s  LL):):FNNK ++]]]bcictct]u vXYWVa1fQR S v  !,FNN@Y@Y Zzz&"5"56  &+# ws7!Dc||_yrX)r?r*new_embeddingss r.set_input_embeddingszT5Stack.set_input_embeddingss *r/c  X||n|jj}||n|jj}| | n|jj} | | n|jj} |$|"|j rdnd} t d| d| d|&|j}|jd|d}n8||jdd}n"|j rdnd} t d| d| d |jr%|jr|rtjd d }|(|j t d |j|}|\}}|d ur|j st d|d|j rf|rr|p|jjr5tt!|jt!|j}n%t!|j}n|j sd}||j#nd}| %t%j&|||z|j(} |jj r7t+|j||| t-|tr |j.n|}nt1|j||}d}|j r|t1|j|||}| rdnd}|rdnd}|r|j rdnd}d}d}|j3|}|j4D]`}| r||fz}||||||||||| |  }|d}|d}|j r | ||rdnd}|sB||dfz}|j sX||dfz}b|j7|}|j3|}| r||fz}| st9d|||||fDSt;|||||S)Ndecoder_zYou cannot specify both zinput_ids and zinputs_embeds at the same timer2zYou have to specify either z input_ids or inputs_embedszZ`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...Fzz"T5Stack.forward..#s = s)last_hidden_staterr< attentionscross_attentions)rGrroutput_hidden_statesuse_return_dictrtr,sizerrrr}r~r?is_encoder_decoderr r get_seq_lengthr&rrrrYrrrSrCrDtupler)r*r rrrrNrrrrVrrkwargserr_msg_prefix input_shaperrpast_key_values_lengthencoder_extended_attention_maskall_hidden_statesall_attentionsall_cross_attentionsrrr< layer_module layer_outputss r.r>zT5Stack.forwards4"+!6IDKK+/??ZN*>*:.HXXvw  "#..*K!r;r?;I  &',,.s3K+/??ZN:>:J-XfWggtuv v  & &4==##p"    ( !_`` --i8M!, J  ?? #LTFRg!hii ??_4;;11&9$DKK8,dkk:Z'O'3$++&FO#OETE`!?!?!Afg  !"\\&(>(KTaThThN ;; ! !/{{+--o/BC!0 D D$N7{{+-N +/' ??4@.G{{+5&; / +#7BD0d&7DOOrRV (,% ]3  JJ VL#$58H$H!(%/- /#"3'- M*!,M *!,M#8#D0=CTaZ[0\- !/=3C2E!E??+?=QRCSBU+U(= V@--m<  ]3  1]4D D  "#%"(    9+++%1   r/) NNNNNNNNNNN)r?r@rAr$rJr>rBrCs@r.r<r<|s6, + "#!d r/r<ceZdZdgZdddZdeffd ZdZdZe dd e jdzd e jdzd e jdzd e jdzd eee jdzdedzde j dzde j dzdedzdedzdedzdedzde jdzdee jezfdZxZS)rFdecoder.block.0.layer.1.EncDecAttention.relative_attention_bias.weight shared.weightencoder.embed_tokens.weightdecoder.embed_tokens.weightrGct||tj|j|j |_tj|}d|_ d|_ t||_ tj|}d|_ |j|_t||_|j!yNFT)r#r$rrr>rMr copydeepcopyrtrr<encodernum_decoder_layersrAdecoderrEr*rGencoder_configdecoder_configr-s r.r$zT5Model.__init__As  ll6#4#4fnnE v.$)!#( ~. v.$(!$*$=$=!~.  r/c|jSrXr r*s r.get_input_embeddingszT5Model.get_input_embeddingsR {{r/c~||_|jj||jj|yrXr rprJrrrHs r.rJzT5Model.set_input_embeddingsU-$  )).9 )).9r/Nr rr rencoder_outputsrrNdecoder_inputs_embedsrrrVrrrc F| | n|jj} | | n|jj} ||j|||| | | }nI| rGt |t s7t |dt |dkDr|dndt |dkDr|dnd}|d}|j||||||| | | | |  }| s||zSt|j|j|j|j|j|j|j|jS) aV input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. T5 is a model with relative position embeddings so you should be able to pad the inputs on both the right and the left. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for detail. [What are input IDs?](../glossary#input-ids) To know more on how to prepare `input_ids` for pretraining take a look a [T5 Training](./t5#training). decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) T5 uses the `pad_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). To know more on how to prepare `decoder_input_ids` for pretraining take a look at [T5 Training](./t5#training). decoder_attention_mask (`torch.BoolTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. Example: ```python >>> from transformers import AutoTokenizer, T5Model >>> tokenizer = AutoTokenizer.from_pretrained("google-t5/t5-small") >>> model = T5Model.from_pretrained("google-t5/t5-small") >>> input_ids = tokenizer( ... "Studies have been shown that owning a dog is good for you", return_tensors="pt" ... ).input_ids # Batch size 1 >>> decoder_input_ids = tokenizer("Studies show that", return_tensors="pt").input_ids # Batch size 1 >>> # preprocess: Prepend decoder_input_ids with start token which is pad token for T5Model. >>> # This is not needed for torch's T5ForConditionalGeneration as it does this internally using labels arg. >>> decoder_input_ids = model._shift_right(decoder_input_ids) >>> # forward pass >>> outputs = model(input_ids=input_ids, decoder_input_ids=decoder_input_ids) >>> last_hidden_states = outputs.last_hidden_state ```Nr rrNrrVrrrr1rSr<rT r rrNrrrrrrVrr)rSrdecoder_hidden_statesdecoder_attentionsrUencoder_last_hidden_staterencoder_attentions)rGrrWrprYrlenrrrrSrr<rTrU)r*r rr rr~rrNrrrrVrrr\r<decoder_outputss r.r>zT5Model.forwardZsRF"+!6IDKKrBrCs@r.rr7s Q*&(7'6 x": .23759:>BF(,-159!%)-,0#'26s ##d*s ))D0s !++d2 s !& 0 04 7 s uU%6%6784? s s ||d*s  %||d2s $;s  $;s #Tks D[s ((4/s u  !$6 6!s s r/rz: T5 Model with a `language modeling` head on top. ) custom_introc!eZdZdgZddddZdeffd ZdZdZe dd e jdzd e jdzd e jdzd e jdzd eee jdzdedzde jdzde jdzde jdzdedzdedzdedzdedzde jdzdee jezfdZde jfdZxZS)rrgrh)zlm_head.weightrjrkrGct|||j|_t j |j |j|_tj|}d|_ d|_ t||_ tj|}d|_ |j|_t||_t j"|j|j d|_|j'y)NFTrJ)r#r$rM model_dimrrr>r rnrortrr<rprqrArrrLrrErss r.r$z#T5ForConditionalGeneration.__init__s  ll6#4#4fnnE v.$)!#( ~. v.$(!$*$=$=!~. yy1B1BO  r/c|jSrXrwrxs r.ryz/T5ForConditionalGeneration.get_input_embeddingsrzr/c~||_|jj||jj|yrXr|rHs r.rJz/T5ForConditionalGeneration.set_input_embeddingsr}r/Nr rr rr~rrNrlabelsrrrVrrrc | | n|jj} | | n|jj} ||j|||| | | }nI| rGt |t s7t |dt |dkDr|dndt |dkDr|dnd}|d}| |||j| }|j||||||| | | | | }|d}|jjr||jdzz}|j|}d}| ^td }| j|j} ||jd |j!d | jd }| s|f|ddz|z}||f|zS|St#|||j$|j&|j(|j*|j,|j&|j( S) a input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. T5 is a model with relative position embeddings so you should be able to pad the inputs on both the right and the left. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for detail. [What are input IDs?](../glossary#input-ids) To know more on how to prepare `input_ids` for pretraining take a look a [T5 Training](./t5#training). decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) T5 uses the `pad_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). To know more on how to prepare `decoder_input_ids` for pretraining take a look at [T5 Training](./t5#training). decoder_attention_mask (`torch.BoolTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[-100, 0, ..., config.vocab_size - 1]`. All labels set to `-100` are ignored (masked), the loss is only computed for labels in `[0, ..., config.vocab_size]` Examples: ```python >>> from transformers import AutoTokenizer, T5ForConditionalGeneration >>> tokenizer = AutoTokenizer.from_pretrained("google-t5/t5-small") >>> model = T5ForConditionalGeneration.from_pretrained("google-t5/t5-small") >>> # training >>> input_ids = tokenizer("The walks in park", return_tensors="pt").input_ids >>> labels = tokenizer(" cute dog the ", return_tensors="pt").input_ids >>> outputs = model(input_ids=input_ids, labels=labels) >>> loss = outputs.loss >>> logits = outputs.logits >>> # inference >>> input_ids = tokenizer( ... "summarize: studies have shown that owning a dog is good for you", return_tensors="pt" ... ).input_ids # Batch size 1 >>> outputs = model.generate(input_ids) >>> print(tokenizer.decode(outputs[0], skip_special_tokens=True)) >>> # studies have shown that owning a dog is good for you. ```Nrrrr1rrrr) ignore_indexr2 losslogitsrrrrUrrr)rGrrWrprYrrr1rrscale_decoder_outputsrrrr4rrrXrrr<rTrUrS)r*r rr rr~rrNrrrrrVrrr\r<rsequence_output lm_logitsrloss_fctoutputs r.r>z"T5ForConditionalGeneration.forwards T"+!6IDKKrrBrCs@r.rrs Q*&*'6'6 x*: .23759:>=A(,26:>*.!%)-,0#'26O ##d*O ))D0O !++d2 O !& 0 04 7 O uU\\23d: O O ((4/O  %0047O   4'O $;O  $;O #TkO D[O ((4/O " u  !O 3#O O b)ELL)r/rceZdZddiZdgZdeffd ZdZdZe dd e jdzd e jdzd e jdzd e dzd e dzde dzdee jezfdZxZS)rrjrhrrrGct||tj|j|j |_|}d|_d|_t||_ |jy)NF) r#r$rrr>rMr rrYr<rprE)r*rGrtr-s r.r$zT5EncoderModel.__init__sY  ll6#4#4fnnE #( ,1)~.  r/c|jSrXrwrxs r.ryz#T5EncoderModel.get_input_embeddingsrzr/cH||_|jj|yrX)r rprJrHs r.rJz#T5EncoderModel.set_input_embeddingss$  )).9r/Nr rrNrrVrrc h||n|jj}|j||||||}|S)aI input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. T5 is a model with relative position embeddings so you should be able to pad the inputs on both the right and the left. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for detail. To know more on how to prepare `input_ids` for pretraining take a look a [T5 Training](./t5#training). Example: ```python >>> from transformers import AutoTokenizer, T5EncoderModel >>> tokenizer = AutoTokenizer.from_pretrained("google-t5/t5-small") >>> model = T5EncoderModel.from_pretrained("google-t5/t5-small") >>> input_ids = tokenizer( ... "Studies have been shown that owning a dog is good for you", return_tensors="pt" ... ).input_ids # Batch size 1 >>> outputs = model(input_ids=input_ids) >>> last_hidden_states = outputs.last_hidden_state ```r)rGrWrp) r*r rrNrrVrr\r~s r.r>zT5EncoderModel.forwardsHD&1%rBrCs@r.rrs7I*4& x :.23726)-,0#',##d*,))D0,((4/ ,  $; , #Tk ,D[, u  !O 3,,r/rz T5 model with a sequence classification/head on top (a linear layer on top of the pooled output) e.g. for GLUE tasks. c~eZdZdgZdeffd Ze ddejdzdejdzdejdzdejdzd e ejdzd ejdzd ejdzd ejdzd e dzde dzde dzde dzde ezfdZxZS)T5ForSequenceClassificationrgrGct||t||_t ||_|j yrX)r#r$rr rclassification_headrErVs r.r$z$T5ForSequenceClassification.__init__s5  "6?#7#?  r/Nr rr rr~rNrrrrrVrrc | | n|jj} |d} |$|"td|jj| || t d|j |}|j|||||||| | | |  }|d}|j|jjj|j}ttj|jdj!dk(d|j"\}}}||ddfj%|d |ddd ddf}|j'|}d}||j|j}|jj(|jj*dk(rd |j_nv|jj*dkDrL|j,tj.k(s|j,tj0k(rd |j_nd |j_|jj(d k(rSt3}|jj*dk(r&||j5|j5}n|||}n|jj(d k(rGt7}||j%d |jj*|j%d }n,|jj(d k(rt9}|||}| s|f|ddz}||f|zS|St;|||j<|j>|j@|jB|jD|jF|jH S)a input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. T5 is a model with relative position embeddings so you should be able to pad the inputs on both the right and the left. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for detail. [What are input IDs?](../glossary#input-ids) To know more on how to prepare `input_ids` for pretraining take a look a [T5 Training](./t5#training). decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) T5 uses the `pad_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). To know more on how to prepare `decoder_input_ids` for pretraining take a look at [T5 Training](./t5#training). decoder_attention_mask (`torch.BoolTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). NFz8Passing input embeddings is currently not supported for If no `decoder_input_ids` or `decoder_inputs_embeds` are passed, `input_ids` cannot be `None`. Please pass either `input_ids` or `decoder_input_ids` or `decoder_inputs_embeds`.) rr rr~rNrrrrVrrrz7All examples must have the same number of tokens.r2 regressionsingle_label_classificationmulti_label_classificationr)%rGrWNotImplementedErrorr-r?r,r1r eq eos_token_idr4rrr&unique_consecutivesumnumelrrr problem_typerr9rrrsqueezerrrrrrrUrrr)r*r rr rr~rNrrrrrVrr\rreos_maskr_r+sentence_representationrrrrs r.r>z#T5ForSequenceClassification.forwardsN`&1%>KbKbJcd   $)>)F  U !% 1 1) < "" )/#9+'"7/!5##  "!*<< 8 89<<_=S=ST  $ $X\\!_ 5 ; ; = B E &5%:%:" A{"1(A+">"C"CJPRT_"`abdfhiai"j))*AB  YYv}}-F{{''/;;))Q./;DKK,[[++a/V\\UZZ5OSYS_S_chclclSl/LDKK,/KDKK,{{''<7"9;;))Q.#FNN$4fnn6FGD#FF3D))-JJ+- B 0F0F GUWY))-II,./Y,F)-)9TGf$ Ev E.#33")"?"?&99$55&-&G&G")"?"?&99  r/) NNNNNNNNNNNN)r?r@rArrr$rr&rrZlistrrBr[rr>rBrCs@r.rrsU+s)s&x.2.259:>:>26:>*.!%)-,0#'A ##d*A  t+A !++d2 A !& 0 04 7 A e//047 A ((4/A  %0047A   4'A $;A  $;A #TkA D[A  0 0A A r/rceZdZdeffd Ze d dejdzdejdzdejdzdejdzdedzd edzd edzd e eje zfd Z xZ S)r$rGc,t|||j|_t||_t j |j|_t j|j|j|_ |jyrX) r#r$rrr rrQrrSrLr+rrErVs r.r$z!T5ForTokenClassification.__init__ssj   ++)&1zz&";";< ))F$6$68I8IJ r/Nr rrNrrrVrrc ||n|jj}|j||||||} | d} |j| } |j | } d} |z T5ForTokenClassification.forward~s4&1%rBrCs@r.r$r$qs x *..2-1&*)-,0#'6 <<$&6  t+6 ||d* 6  t# 6  $; 6 #Tk6 D[6  u|| 4 46 6 r/r$ceZdZdgZdddZdeffd ZdZdZe dd e jdzd e jdzd e jdzd e jdzd eee jdzde jdzde jdzde jdzde jdzdedzdedzdedzdedzdee jezfdZxZS)rrgrhrirGc$t|||j|_t j |j |j|_tj|}d|_ d|_ t||_ tj|}d|_ |j|_t||_|j"|_t j$|j&|j"|_|j+yrm)r#r$rMrrrr>r rnrortrr<rprqrArrrrLr+rrErss r.r$zT5ForQuestionAnswering.__init__s  ll6#4#4fnnE v.$)!#( ~. v.$(!$*$=$=!~.  ++))F$6$68I8IJ r/c|jSrXrwrxs r.ryz+T5ForQuestionAnswering.get_input_embeddingsrzr/c~||_|jj||jj|yrXr|rHs r.rJz+T5ForQuestionAnswering.set_input_embeddingsr}r/Nr rr rr~start_positions end_positionsrNrrrrVrrc | | n|jj} | | n|jj} ||d} | | | td|j |}| | n|jj} | | n|jj} ||j |||| | | }nI| rGt |ts7t|dt|dkDr|dndt|dkDr|dnd}|d}|j||| d||| | | | }|d}|j|}|jdd \}}|jd j}|jd j}d}||t|jdkDr*|jd j|j }t|jdkDr*|jd j|j }|jd}|j#d|}|j#d|}t%| }|||}|||}||zdz }| s||f|ddz|z}||f|zS|St'||||j(|j*|j,|j.|j0|j*|j, S)az input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. T5 is a model with relative position embeddings so you should be able to pad the inputs on both the right and the left. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for detail. [What are input IDs?](../glossary#input-ids) To know more on how to prepare `input_ids` for pretraining take a look a [T5 Training](./t5#training). decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) T5 uses the `pad_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). To know more on how to prepare `decoder_input_ids` for pretraining take a look at [T5 Training](./t5#training). decoder_attention_mask (`torch.BoolTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. NFrrrrr1r) r rrNrrrrrrVrr2rr) r start_logits end_logitsrrrrUrrr)rGrWrr,r1rprYrrrrrsplitrrrXr4rrrrrr<rTrUrS)r*r rr rr~rrrNrrrrVrr\r<rrrrr total_loss ignored_indexr start_lossend_lossrs r.r>zT5ForQuestionAnswering.forwards0\&1%)F  U !% 1 1) < !*!6IDKKrBrCs@r.rrs*r)s&'6'6 x,: .23759:>=A371526:>!%)-,0#'I ##d*I ))D0I !++d2 I !& 0 04 7 I uU\\23d: I ))D0I ''$.I ((4/I  %0047I $;I  $;I #TkI D[I u  !$G G!I I r/r)rrrr rrr$)Grrnrr&rtorch.nnrrrrMrr activationsr cache_utilsr r r generationr masking_utilsrrmodeling_layersrmodeling_outputsrrrrrrrmodeling_utilsrutilsrrrrrconfiguration_t5r get_loggerr?r}Moduler!apex.normalizationrDinfo ImportError ExceptionwarningrFr_rgrqrrrrr r<rrrrr$r__all__r:r/r.rsJ AA&!CC)J9.^^&   H %+"))+2 Y/K KKabbii,299: $I"))IX299D!BII!HZ (Z z299$^!^!^!Bx x vV V V r z)!2Oz)  z)zD&DDN M "3M  M `C 0C C Lo .o o d }0 Y NNWXYsG##H+HH