qi dZddlmZddlmZmZmZer ddlZddlm Z eje Z dde edzfdZy) z7SpQR (Sparse-Quantized Representation) integration file)should_convert_module)is_spqr_availableis_torch_availableloggingNmodules_to_not_convertcptrddlm}d}|jD]\}}t ||st j d5t|tjr|j}j|j|j|j|j|j ||d||d||d||d }|j"|j%d|j'||d }d d d |st(j+d |S#1swYxYw) a] Public method that replaces the Linear layers of the given model with SPQR quantized layers. Args: model (`torch.nn.Module`): The model to convert, can be any `torch.nn.Module` instance. modules_to_not_convert (`list[str]`, *optional*, defaults to `None`): A list of nn.Linear weights to not convert. If a parameter path is in the list (e.g. `lm_head.weight`), the corresponding module will not be converted. quantization_config (`SpQRConfig`): The quantization config object that contains the quantization parameters. r)QuantizedLinearFmetaz.dense_weights.shapez.row_offsets.shapez.col_vals.shapez.in_perm.shape) rowscolsbitsbeta1beta2dense_weights_shaperow_offsets_shapecol_vals_shape in_perm_shapeTNzYou are loading your model using eetq but no linear modules were found in your model. Please double check your model architecture, or submit an issue on github if you think this is a bug.)r spqr_quantr named_modulesrtorchdevice isinstancennLinearshapescreate_placehodler out_features in_featuresrrr_modulesrequires_grad_ set_submoduleloggerwarning) modelrquantization_configr has_been_replaced module_namemoduler new_modules P/opt/pipecat/venv/lib/python3.12/site-packages/transformers/integrations/spqr.pyreplace_with_spqr_linearr,sJ.$224) V$[2HI  \\& ! )&")),,33,??,,++,11-33-33(.+>R/S(T&, }r7sH:?BB   H %/DI