IuiadZddlZddlZddlZddlZddlZddlZddlmZm Z m Z ddl Z ddl Z dZdZdZdZd Zd Zd Zd Zd+d ZdZGddeZGddZGddeeZdZGddeeZGddeZdZ GddeZ!GddeZ"Gd d!eZ#Gd"d#eZ$d$Z%Gd%d&ejLZ'Gd'd(e'Z(Gd)d*e'Z)y#e$rdZYwxYw),zSpecifications declare the expected variables layout of CTranslate2 models that do not load a computation graph. The model converter should make sure that each required variable of the specification is set. N)DictListOptionalTF __optional)int8 int8_float32 int8_float16 int8_bfloat16int16float16bfloat16float32)rotary_scaling_long_factorrotary_scaling_short_factorc|s|S|d|SN/)scopenames N/opt/pipecat/venv/lib/python3.12/site-packages/ctranslate2/specs/model_spec.py _join_scoper)s  T ""c$|jdSr)splitrs r _split_scoper/s ;;s rcRt|}|dd|d}}dj||fS)Nr)rjoin)rkeysattrs r _parent_scoper$3s2  Ds)T"X4E 88E?D  rc zt|jjD]\}}|jdrt |tr1t |D]"\}}t ||t|d||fz$Yt |trt ||t||||t|||y)z Recursively visits a layer spec._%s_%drN) list__dict__items startswith isinstance enumerate visit_specr LayerSpec)specfnrrvalueielems rr.r.9sDMM//12 6 e ??3   eT "$U+ T44;ugq >Q+RS T y ) ub E4(@ A t[-u 5 6rc|s|St|}|D]} t||}|S#t$r0|jdd\}}t||t |}YLwxYw)Nr&)rgetattrAttributeErrorrsplitint)r0indexr"keyr#s r index_specr=Gsu   D3 34%D3 K 3**S!,KD%4&s5z2D 3s &6AAceZdZfdZxZS) FrozenMetac4t||i|}d|_|S)NT)super__call___frozen)selfargskwargsinstance __class__s rrBzFrozenMeta.__call__Us$7#T4V4r)__name__ __module__ __qualname__rB __classcell__rHs@rr?r?Ts rr?ceZdZfdZxZS) FrozenAttrcrt|drt||std|zt| ||y)NrCzAttribute %s does not exist)hasattrr8rA __setattr__)rDr<r2rHs rrRzFrozenAttr.__setattr__\s6 4 #GD#,> !>!DE E C'r)rIrJrKrRrLrMs@rrOrO[s ((rrOc zeZdZdZd dZ ddededeeejffdZ dZ d Z dd e eddfd Zd Zy)r/zPA layer specification declares the weights that should be set by the converters.returnNctgfd}|j|rtddjzy)zVerify that the required weights are set. Raises: ValueError: If a required weight is not set in the specification. cV|j|yt|tjr=|jtj k(r|j tj}nt|tr%tjdj|}nt|tr%tjdj|}nMt|tr=|tk7r4tj|jdtj}t|tjst|tj r t#|}n+t$r%t|t&j(r t+|}t-|d}t/|||y)Nrrutf-8)dtyper )appendr,npndarrayrXfloat64astyperfloattypeboolstrOPTIONAL frombufferencodergeneric NumpyVariabletorch_is_availabletorchTensorPyTorchVariablersetattr)r0rr2 attr_nameunset_attributess r_checkz"LayerSpec.validate.._checkms} ''-%,;;"**,!LL4EE5)+007E4((--e4E3'H$MM%,,w*?rwwOE%, 5"**0M%e,# 5%,,(G'.$T*2.I D)U +rz/Some required model attributes are not set: %s N)_visit ValueErrorr!)rDrnrms @rvalidatezLayerSpec.validateesH  ,4 F C)),-.  rprefixorderedcifd}|j||r%ttjdSS)a!Recursively returns the weights from this layer and its children. Arguments: prefix: Prefix to prepend to all variable names. ordered: If set, an ordered list is returned instead. Returns: Dictionary mapping variables name to value. cXt|tr |tk(ry|t|<yN)r,rarbr)r0rr2rsvars r _register_varz*LayerSpec.variables.._register_vars(%%%8*;-2C FD) *rc |dSNrr)xs rz%LayerSpec.variables..s !A$r)r<)rpr(sortedr*)rDrsrtryrxs ` @r variableszLayerSpec.variabless= 3 M" syy{?@ @ rc|jd}t|D]j\}}|D]`\}}||k(rt|\}}|jr,|j |s>|t vsGt ||}t|||jly)z4Find duplicate variables in spec and create aliases.TrtN)rreversedr$ is_scalarequalSKIP_CREATING_ALIASr=rk) rDrrr2 other_name other_valuerrlr0s r_alias_variableszLayerSpec._alias_variablessNN4N0 #I. KD%+4 ' K:%$1#6 y) K0!)<<&dE2DD)Z8  rc+tvr#tddjtfd}|j|y)z-Possibly quantizes the variable of the layer.Nz7 is not a valid quantization type. Accepted types are: , c t|tr|jryt|d}d}t |d|z}|j dv}|rN dk(r|j dj}tjdtjtj|z }||z}tj|}tj|tjtjj tjtjj"}|j%tj}t'|}t'|}np dvr|j dj}d}t)|j*dk(r+|j*}|j-|j*d d}tjtj|d }d ||d k(<d |z }|tj.|d z}tj|}|j%tj0}|r|j-|}t'|}t'|}nY d vrU|j }nC|rA dvr|j d}n+ dvr|j d}n dvr|j d}t3||||t3|d|z|yy)Nr z%s_scale)rr rr ri)rr r r rr6)axisg_@)r rr)r r r )rr r)rr r )r,VariablerrrQrXtonumpyrZramaxabsoluterintclipiinfor minmaxr]rflenshapereshape expand_dimsrrk) r0rr2r<scaleis_quantizableis_convertible old_shaper quantizations r _quantizez&LayerSpec._quantize.._quantizes{eX.%//2Ct$R(CE$T:+;D&+DO!DLER^^E155EGGENE!LL1E % i 8)%0E)%0E!%GG!HH\2E#>>!HHY/E!%BB!HHZ0E!%II!HHY/E D#u % j3.6!r)ACCEPTED_MODEL_TYPESrqr!rp)rDrrs ` rrzLayerSpec._quantizesG  # c(eZdZdZdZdZdZdZy) ModelConfigz$Base class for model configurations.c N|jD]\}}t|||y)z7Initializes the configuration with a set of parameters.N)r*rk)rDrFr<r2s r__init__zModelConfig.__init__'s' ,,. &JC D#u % &rc|jjDcic]\}}|jds||c}}Scc}}w)z*Returns the configuration as a dictionary.r&)r)r*r+rDr<r2s rto_dictzModelConfig.to_dict,sE#mm113 U>>#& J   sAc"||j|<yrw)r)rs r add_attributezModelConfig.add_attribute4s" crct|dd5}tj|j|dd|j ddddy#1swYyxYw) z'Saves the configuration as a JSON file.wrWencodingT)indent sort_keysroN)openjsondumprwrite)rDpath config_files r save_as_jsonzModelConfig.save_as_json7sS $g . $+ II      d # $ $ $s 9AAN)rIrJrKrrrrrrrrrr$s.&  # $rrc~eZdZdZdZedZedZedZdZ dde d e e d dfd Z d e d dfd Z dZy) ModelSpecz"The top level layer specification.c<|j|_i|_y)z$Initializes the model specification.N)get_default_config_config_filesrDs rrzModelSpec.__init__Fs..0  rct)z$The name of the model specification.NotImplementedErrorrs rrzModelSpec.nameK "##rcy)zThe model specification revision. This value is incremented each time the weights layout of the model is changed (e.g. a weight is renamed). r6rrs rrevisionzModelSpec.revisionPsrc|jS)zThe model configuration.)rrs rconfigzModelSpec.configYs||rcy)z5Returns the default configuration used by this model.Nrrs rrzModelSpec.get_default_config^srNrfilenamerTctjj|std|z|tjj |}||j vrtd|z||j |<y)z4Registers a file to be saved in the model directory.zFile %s does not existNz*A file with name %s was already registered)osrisfilerqbasenamer)rDrrs r register_filezModelSpec.register_filebshww~~d#5<= =  ww''-H t{{ "IHTU U $ Hr output_dirc|jtjj|d|j9|jj tjj|d|j jD]h\}}tjj||}tjj|rtd|ztj||jy)zwSaves this model on disk. Arguments: output_dir: Output directory where the model is saved. z model.binNz config.jsonz-File %s already exists in the model directory) _serializerrr!rrrr*exists RuntimeErrorshutilcopy)rDrrr destinations rsavezModelSpec.savels  Z=> << # LL % %bggll:}&M N"kk//1 +NHd'',,z8._write_stringsN FKKS[1_=> FMM'23 FKKQ/0rIrN)rr,rarYrrrrCURRENT_BINARY_VERSIONrrrrrrX num_bytesto_bytes) rDrraliasesvariablerrr2dimalias variable_namers @rrzModelSpec._serialize~s t4 +H(1+s+x(  *  + $  - 1 KK C)?@ A $)) $ KK C7 8 KK CY8 9( . ed# FKKS-=>? ;;7CKK C 567 FKK->u{{-KLM FKKU__->?@ ENN,- . KK CW6 7(/ -$}e$m, -) - - -s G2II$rw)rIrJrKrrpropertyrrrrrarrrrrrrrrCs, $$%#%#%$%+s+t+$ -rrc#K|jD]>\}}t|dk(r ||dft|D]\}}d||dzfz|f@yw)Nr6rr')r*rr-) vocabulariesr vocabularyr3vocabs r_flatten_vocabulariesrsn(..05j z?a  1 % %%j1 55q1u -u44 5 5sAAc PeZdZdZ d dedededeededef fd ZxZS) SequenceToSequenceModelConfigz.Configuration for sequence-to-sequence models. unk_token bos_token eos_tokendecoder_start_tokenadd_source_bosadd_source_eosc 2t|d||||||d|y)aInitializes the configuration for sequence-to-sequence models. Args: unk_token: The unknown token. bos_token: The start of sentence token. eos_token: The end of sentence token. decoder_start_token: The decoder start token. If ``None``, the token should be passed by the user in the target prefix. add_source_bos: If ``True``, ``bos_token`` will be automatically added to the source input. add_source_eos: If ``True``, ``eos_token`` will be automatically added to the source input. **kwargs: Additional configuration. )rrrrrrNrrAr) rDrrrrrrrFrHs rrz&SequenceToSequenceModelConfig.__init__s30   3))   r)r FF) rIrJrKrrarr`rrLrMs@rrrsb8!-2$$        &c]       rrceZdZdZfdZdZejdZejdZ de e ddfd Z de e ddfd Z d e ddfd Zdfd Zde ddffd ZxZS)SequenceToSequenceModelSpecz3Base specification for sequence to sequence models.c6t|ggd|_y)z7Initializes a sequence to sequence model specification.sourcetargetN)rAr _vocabulariesrDrHs rrz$SequenceToSequenceModelSpec.__init__s  rctSrw)rrs rrz.SequenceToSequenceModelSpec.get_default_configs ,..rct)z9Returns the source vocabulary size expected by the model.rrs rget_source_vocabulary_sizez6SequenceToSequenceModelSpec.get_source_vocabulary_sizerrct)z9Returns the target vocabulary size expected by the model.rrs rget_target_vocabulary_sizez6SequenceToSequenceModelSpec.get_target_vocabulary_sizerrtokensrTNc@|jdj|y)znRegisters a source vocabulary of tokens. Arguments: tokens: List of source tokens. rNrrYrDrs rregister_source_vocabularyz6SequenceToSequenceModelSpec.register_source_vocabulary 8$++F3rc@|jdj|y)znRegisters a target vocabulary of tokens. Arguments: tokens: List of target tokens. rNrrs rregister_target_vocabularyz6SequenceToSequenceModelSpec.register_target_vocabularyrrrc(|j|dy)zvRegisters a vocabulary mapping file. Arguments: path: Path to the vocabulary mapping file. zvmap.txtN)r)rDrs rregister_vocabulary_mappingz7SequenceToSequenceModelSpec.register_vocabulary_mappings 4,rc t||j|jd}|j D]\}}t |t s|g}|j|}t|t|k7r#td|t|t|fztt||D]?\}\}}t||k7std|j|t||fzy)NrzCIncorrect number of %s vocabularies: %d registered, but expected %dzK%s vocabulary %d has size %d but the model expected a vocabulary of size %d) rArrrrr*r,r(rrrqr-zip capitalize) rDvocabulary_sizesrsizesrr3r expected_sizerHs rrrz$SequenceToSequenceModelSpec.validates  557557  ,113 KD%eT*--d3L< CJ. YS.E ;<3z3SequenceToSequenceModelSpec.save..sTZz-a00Tssharedrz %s_vocabulary) dictrrr(valuesallr*_save_vocabularyrAr)rDrrrrr+rHs @rrz SequenceToSequenceModelSpec.saves1$2D2DEF  3 3 56 TCST T$&6q&9:L(..0 ILD& Z4)? H I  Z rr)rIrJrKrrrabcabstractmethodrrrrarrr!rrrrLrMs@rr r s= / $$ $$4c4t44c4t4---4 !s !t ! !rr c8eZdZdZ ddededeffd ZxZS)LanguageModelConfigz"Configuration for language models.rrrc ,t|d|||d|y)aInitializes the configuration for language models. Args: unk_token: The unknown token. bos_token: The start of sentence token. eos_token: The end of sentence token. **kwargs: Additional configuration. )rrrNrr)rDrrrrFrHs rrzLanguageModelConfig.__init__+s*     r)rr r )rIrJrKrrarrLrMs@rr5r5(s7,!       rr5ceZdZdZfdZdZejdZde e ddfdZ d fd Z d e ddffd Z xZS) LanguageModelSpecz'Base specification for language models.c0t|g|_y)z+Initializes a language model specification.N)rAr _vocabularyrs rrzLanguageModelSpec.__init__Es rctSrw)r5rs rrz$LanguageModelSpec.get_default_configJs "$$rct)z2Returns the vocabulary size expected by the model.rrs rget_vocabulary_sizez%LanguageModelSpec.get_vocabulary_sizeMrrrrTNc$t||_y)zbRegisters the vocabulary of tokens. Arguments: tokens: List of tokens. N)r(r:rs rregister_vocabularyz%LanguageModelSpec.register_vocabularyRs  <rct||j}t|j|k7r#t dt|j|fzy)NzEVocabulary has size %d but the model expected a vocabulary of size %d)rArrr=rr:rq)rDexpected_vocabulary_sizerHs rrrzLanguageModelSpec.validateZs` #'#;#;#= t $< <Wt''(*BCD  =rrcRt|d|jt| |y)Nr)r1r:rAr)rDrrHs rrzLanguageModelSpec.saveds#\43C3CD  Z rr)rIrJrKrrrr2r3r=rrar?rrrrLrMs@rr8r8Bs^1 % $$($s)((!s!t!!rr8ctjj|d|z}t|dd5}t j ||ddddy#1swYyxYw)Nz%s.jsonrrWrr)r)rrr!rrr)rrrvocabulary_pathvocabulary_files rr1r1lsNggll:y4/?@O osW 55 &/!4555s AAceZdZdZeej deefdZ de fdZ eej de fdZ de ddfdZej dej fdZde fd Zej defd Zej defd Zej de ddfd Zej de fd Zy)rz(Abstract base class for model variables.rTctrwrrs rrzVariable.shapevrrc2t|jdk(Sr{)rrrs rrzVariable.is_scalar{s4::!##rctrwrrs rrXzVariable.dtype~rrrXcF||jk(r|S|j|Srw)rX_torDrXs rrz Variable.tos! DJJ Kxxrctrwrrs rrzVariable.numpy !##rcTt|t|uxr|j|Srw)r__equalrDothers rrzVariable.equals#DzT%[(?T[[-??rctrwrrs rrzVariable.num_bytesrNrctrwrrs rrzVariable.to_bytesrNrctrwrrLs rrKz Variable._torNrctrwrrQs rrPzVariable._equalrNrN)rIrJrKrrr2r3rr:rr`rrarXrrZr[rrrbytesrrKrPrrrrrss62 $tCy$$$4$$s$$   $rzz$$@d@ $3$$ $%$$ $$$$ $t$$rrceZdZdZdZedeefdZede fdZ de jfdZ defdZdefdZd e defd Zdefd Zy ) rfz Model variable as a Numpy array.c||_yrwarray)rDr[s rrzNumpyVariable.__init__s  rrTc.|jjSrw)r[rrs rrzNumpyVariable.shapeszzrcB|jjjSrw)r[rXrrs rrXzNumpyVariable.dtypeszz$$$rc|jSrwrZrs rrzNumpyVariable.numpys zzrc.|jjSrw)r[nbytesrs rrzNumpyVariable.num_bytesszz   rc6|jjSrw)r[tobytesrs rrzNumpyVariable.to_bytesszz!!##rrXc|dk(r?ts tdtj|jj |St j|}|jj||_|S)Nrz5Converting to bfloat16 requires torch to be installed) rgrrj from_numpyr[rrZrXr]rLs rrKzNumpyVariable._tosg J %"K#--djj9<rs8  ''  T# ! 6 ((s jsl  $* $>[- [-|5# K# LU!)U!p + 4'! '!T5*$sww*$Z+ H+ \1Dh1DssC##C-,C-