qi dZddlmZddlmZmZer ddlZddlmZeje Z ddddd ddd ddd ddd ddd ddddd d dd Z dZ ddeezdzdefdZy)z;AWQ (Activation aware Weight Quantization) integration file)should_convert_module)is_torch_availableloggingNactc_fc)rlayer_before_act dense_h_to_4hup_projfc_in gelu_impl) starcoder2RefinedWebModelfalconmptgptjgpt_neox gpt_bigcodebloomcZddlm}|tvr|S|jD]\}}t|d}t|d}||k(rYt ||rMt |t|d}|j }tj|} ||| |j|<t||} |S)Nr)ScaledActivationrr ) &gptqmodel.quantization.awq.modules.actrAWQ_SCALES_MAPPINGSnamed_childrenhasattrgetattr out_featurestorchones_modulesreplace_quantization_scales) model model_typernamemoduleact_namelayer_before_act_namer size scale_like_s O/opt/pipecat/venv/lib/python3.12/site-packages/transformers/integrations/awq.pyr!r!'sG,, ,,.< f&z259 3J ?@R S 8 /D E&u.A*.MN`.ab #00DD)J#3FJ#GENN4 ' ;< L device_mapreturnc ddlm}ddlm}||j|j dd|j |j||j|jd }|jD]\}}t||stjd5t|tj r||j|j"|j$|j |j&|j(|j*du|j,jd } | j/d|j1|| d} ddd st2j5d |S#1swYxYw) a Public method that replaces the linear layers of the given model with awq quantized layers. Args: model (`torch.nn.Module`): The model to convert, can be any `torch.nn.Module` instance. quantization_config (`AwqConfig`): The quantization config object that contains the quantization parameters. modules_to_not_convert (`list[str]`, *optional*, defaults to `None`): A list of nn.Linear weights to not convert. If a parameter path is in the list (e.g. `lm_head.weight`), the corresponding module will not be converted. device_map (`Union[str, dict]`, *optional*, defaults to `None`): The device map that maps the parameters to the device r)METHOD)hf_select_quant_linear_v2F) bits group_sizedesc_actsymformatbackendr- quant_method zero_pointpackmetaNT) r2r5r4r3 in_featuresrbiasdevregister_bufferszYou are loading your model using eetq but no linear modules were found in your model. Please double check your model architecture, or submit an issue on github if you think this is a bug.)gptqmodel.quantizationr0gptqmodel.utils.importerr1r2r3r6r7AWQr9 named_modulesrrdevice isinstancennLinearr5r4r<rr=weightrequires_grad_ set_submoduleloggerwarning) r"modules_to_not_convertquantization_configr-r0r1 target_cls module_namer% new_modulehas_been_replaceds r+replace_with_awq_linearrS8sT(.B* % %&11 "))#++ZZ&11  J %224) V$[2HI  \\& ! )&")),',11+//0992== & 2 2!'!4!4D0 ,,%)  ))%0##K<$(! ) ))(   L1 ) )s B-E##E- )NNN)__doc__quantizers.quantizers_utilsrutilsrrrtorch.nnrF get_logger__name__rKrr!strdictboolrSr,r+r^s>?/   H % V<$/JAi 8w 7?C f= o F & $( ?d T! ?  ?r,