qi ddlmZddlmZddlmZmZer ddlZddlm Z eje Z GddeZ Gdd ejjZGd d e j"Zdd eedzfd Zy)) ConversionOps)should_convert_module)is_torch_availableloggingNc reZdZdZ ddeeeejfdedzdeeejffdZ y) EetqQuantizec||_yN) hf_quantizer)selfr s P/opt/pipecat/venv/lib/python3.12/site-packages/transformers/integrations/eetq.py__init__zEetqQuantize.__init__s (N input_dictfull_layer_namereturnc jt|jd\}}|d}|j}tj|j j }tj|tjd\}}|j|}|j|}|||d|iS)NrF_scales) tupleitemsdevicetorcht contiguouscpueetq_kernels_hub quant_weightsint8to) r rrkwargs_value value_device int8_weightscaless rconvertzEetqQuantize.converts))+,Q/5a|| ggen//1557 .<<[%**V[\ V!nn\2 <(0A.I6RRrr ) __name__ __module__ __qualname__rdictstrlistrTensorr'rrr r sU)X\ SsD$667 SJMPT* S c5<<  Srr c.eZdZeddZedZy)EetqLinearMMFunctionNcr|j||||tj|||}|||z}|S|}|Sr )save_for_backwardr w8_a16_gemm)ctxxweightr&biasoutputs rforwardzEetqLinearMMFunction.forward0sK a6!--a@"&"2$ 9? rc|j\}}}}tj|jdj |j j |j }tj|||}|jdr?|jdj|jddjd}dddfS)Nr) saved_tensorsreyeshaper rdtyperr4needs_input_gradsqueezematmul transpose unsqueeze)r5 grad_outputinputr7r&r8identity grad_inputs rbackwardzEetqLinearMMFunction.backward8s&)&7&7#vvt99V\\!_-00?BB5;;O"--hG    "$,,Q/66v7G7G17MNXXYZ[J4t++rr )r(r)r* staticmethodr:rJr/rrr1r1/s( , ,rr1c>eZdZejdffd ZdZxZS) EetqLinearFct|tjt j ||f|d|_tjt j |tj|_|r>tjt j |tj|_ yd|_ y)N)r@F) requires_grad) superrnn Parameterremptyr7float16 weight_scalesr8)r in_features out_featuresr@r8 __class__s rrzEetqLinear.__init__Hs{ ll5;; \/JRW#Xhmn \\%++|EMM*Z[  U[[,u}}%UVDIDIrcrtj||j|j|j}|Sr )r1applyr7rUr8)r rGr9s rr:zEetqLinear.forwardQs.%++E4;;@R@RTXT]T]^ r)r(r)r*rrrr: __classcell__)rXs@rrMrMGs8= rrMmodules_to_not_convertcddlm}|dad}|rinddi}|jD]\}}t ||st j d5t|tjrCt|j|jfd|jdui|}|j||d }ddd|stj!d |S#1swYxYw) a A helper function to replace all `torch.nn.Linear` modules by `EetqLinear` modules. Parameters: model (`torch.nn.Module`): Input model or `torch.nn.Module` as the function is run recursively. modules_to_not_convert (`list[`str`]`, *optional*, defaults to `None`): Names of the modules to not convert in `EetqLinear`. In practice we keep the `lm_head` in full precision for numerical stability reasons. r<) get_kernelz#kernels-community/quantization-eetqFr@Nmetar8TzYou are loading your model using eetq but no linear modules were found in your model. Please double check your model architecture, or submit an issue on github if you think this is a bug.) hub_kernelsr^r named_modulesrrr isinstancerQLinearrMrVrWr8 set_submoduleloggerwarning) modelr\ pre_quantizedr^has_been_replaced module_kwargs module_namemodule new_modules rreplace_with_eetq_linearrnVs(""GH'Bgt_M$224 ) V$[2HI  \\& ! )&")),'&&(;(;BH++UYBY]j ##K<$(!  ) ) )   L ) )s ACC )NF)core_model_loadingrquantizers.quantizers_utilsrutilsrrrtorch.nnrQ get_loggerr(rer autogradFunctionr1ModulerMr-r,rnr/rrrws{/?/   H %S=S(,5>>22,0  %DI