qi56VUddlmZddlmZmZmZmZddlmZddl m Z ddl m Z m Z ddlmZddlmZmZmZej(eZidid id id id id ididididididididididididiiZeeeee ffed<edeeefdeeeffdZedeedeefdZdeeeefeefdeeeefeeffdZGd d!ZGd"d#eZGd$d%eZGd&d'eZ ed()d*eded+fd,Z!d-ed.edefd/Z"y()0) lru_cache)AnyOptionalUnionoverload) constants)InferenceProviderMapping) MimeBytesRequestParameters)ChatCompletionInputMessage)build_hf_headers get_tokenloggingcerebrascohereclarifaizfal-aiz fireworks-aigroqz hf-inference hyperbolicnebiusnscalenvidiaovhcloud replicate sambanovascalewaytogether wavespeedzzai-org!HARDCODED_MODEL_INFERENCE_MAPPINGobjreturncyNrs ^/opt/pipecat/venv/lib/python3.12/site-packages/huggingface_hub/inference/_providers/_common.py filter_noner&.s8;cyr"r#r$s r%r&r&0s.1r'cft|trEi}|jD].\}}| t|ttfr t |}|||<0|St|tr1|Dcgc]%}t|ttfr t |n|'c}St dt |cc}w)NzExpected dict or list, got ) isinstancedictitemslistr& ValueErrortype)rcleanedkvs r%r&r&4s#t"$IIK DAqy!dD\*NGAJ   #tNQR*Qt "= A1DRR 249+> ??Ss**B.ceZdZdZdedededdfdZddd ed eeefd ed eed eedeeeefde fdZ d de e efdee defdZ d eedefdZd eedefdZd eeefdeeeefdeedeeeffdZd ed edeeeffdZd ededefdZd edefdZded edefdZd ed ededeefdZd ed ededeedeef dZy)!TaskProviderHelperz.Base class for task-specific provider helpers.providerbase_urltaskr Nc.||_||_||_yr")r5r7r6)selfr5r6r7s r%__init__zTaskProviderHelper.__init__Hs     r') extra_payloadinputs parametersheadersmodelapi_keyr;c|j|}|j|}|j||}|j||j}|j |||} | t | t|xsi} |j||||} | | td| | td|j|| | } t||j|j| | | S)z Prepare the request to be sent to the provider. Each step (api_key, model, headers, url, payload) can be customized in subclasses. )provider_mapping_infoz8Both payload and data cannot be set in the same request.z2Either payload or data must be set in the request.)urlr7r?jsondatar>) _prepare_api_key_prepare_mapping_info_prepare_headers _prepare_url provider_id_prepare_payload_as_dictrecursive_merger&_prepare_payload_as_bytesr._normalize_headersr r7) r9r<r=r>r?r@r;rBrCpayloadrEnormalized_headerss r%prepare_requestz"TaskProviderHelper.prepare_requestMs ''0!% : :5 A''9)>)J)JK// Zo/p  %g{=;NB/OPG--fjBWYfg  4#3WX X ?t|QR R"44WgtL '33&   r'responserequest_paramsc|S)z Return the response in the expected format. Override this method in subclasses for customized response handling.r#)r9rRrSs r% get_responsezTaskProviderHelper.get_responses r'cT| t}|td|jd|S)zZReturn the API key to use for the request. Usually not overwritten in subclasses.z)You must provide an api_key to work with z$ API or log in with `hf auth login`.)rr.r5r9r@s r%rFz#TaskProviderHelper._prepare_api_keys: ?kG ?;DMM?Jno r'c |td|jdtj|jij|rt|j|Sd}t |D]}|j|jk(s|}n|td|d|jd|j |j k7r6td|d|j d|jd|j d |j d k(r&tjd|d |jd |j d k(r&tjd |d|jd|S)zbReturn the mapped model ID to use for the request. Usually not overwritten in subclasses.Nz+Please provide an HF model ID supported by .zModel z is not supported by provider z is not supported for task z and provider z. Supported task: stagingz! is in staging mode for provider z. Meant for test purposes only.errorz,Our latest automated health check on model 'z' for provider 'z<' did not complete successfully. Inference call might fail.) r.r5rget!_fetch_inference_provider_mappingr7statusloggerwarning)r9r?provider_mappingmappings r%rGz(TaskProviderHelper._prepare_mapping_infos =J4==/YZ[\ \ - 0 0 C G G N4T]]CEJ J8? G4==0#*    #veW,J4==/YZ[\ \  DII -:499+^TXTaTaSbc##3#8#8"9<   " "i / NN@Ono   " "g - NN>ugEUVZVcVcUde--  r'rOrEc|jDcic]\}}| |j|}}}|jd&||j|j|d<|S|d|d<|Scc}}w)zzNormalize the headers to use for the request. Override this method in subclasses for customized headers. z content-typezapplication/json)r,lowerr\ mime_type)r9r>rOrEkeyvaluerPs r%rNz%TaskProviderHelper._normalize_headerssDK==?hZS%V[Vgciik50hh  ! !. 1 9DNN$>59^^">2"!$5G">2!! is A.A.c"it||S)zwReturn the headers to use for the request. Override this method in subclasses for customized headers. )token)r )r9r>r@s r%rHz#TaskProviderHelper._prepare_headerss >"1=W==r' mapped_modelc|j|}|j||}|jdd|jdS)zVReturn the URL to use for the request. Usually not overwritten in subclasses./)_prepare_base_url_prepare_routerstriplstrip)r9r@rjr6routes r%rIzTaskProviderHelper._prepare_urlsK))'2##L'://#&'qc):(;<>s>tCH~>=C=s=s= ! ! !3'+D\ $    8   ~  )  r'r4c neZdZdZdedeffd ZdededefdZd eee e fd e d e de e fd Z xZS) BaseConversationalTaskz Base class for conversational (chat completion) tasks. The schema follows the OpenAI API format defined here: https://platform.openai.com/docs/api-reference/chat r5r6c*t|||dy)Nconversationalr5r6r7superr:r9r5r6 __class__s r%r:zBaseConversationalTask.__init__s (XDTUr'rjr@r cy)Nz/v1/chat/completionsr#r{s r%rnz%BaseConversationalTask._prepare_routes%r'r<r=rBc<td|i|d|jiS)Nmessagesr?r&rJr}s r%rKz/BaseConversationalTask._prepare_payload_as_dicts' Ji*igG\GhGhijjr')rrrrrr:rnr-rr+r r rrK __classcell__rs@r%rrs| VVV&3&&&kU4!;;<=kk 8 k $ kr'rcHeZdZdZfdZdedefdZdeedefdZ xZ S)AutoRouterConversationalTaska Auto-router for conversational tasks. We let the Hugging Face router select the best provider for the model, based on availability and user preferences. This is a special case since the selection is done server-side (avoid 1 API call to fetch provider mapping). c(t|ddy)Nautozhttps://router.huggingface.co)r5r6r)r9rs r%r:z%AutoRouterConversationalTask.__init__%s &3RSr'r@r cR|jds td|jS)rsrtz>Cannot select auto-router when using non-Hugging Face API key.)rur.r6rWs r%rmz.AutoRouterConversationalTask._prepare_base_url(s( !!%(]^ ^== r'r?c<| tdtd||ddS)z In auto-router, we don't need to fetch provider mapping info. We just return a dummy mapping info with provider_id set to the HF model ID. zPlease provide an HF model ID.rliver)r5 hf_model_id providerIdr^r7)r.r )r9r?s r%rGz2AutoRouterConversationalTask._prepare_mapping_info2s1 ==> >'!   r') rrrrr:rrmrr rGrrs@r%rrs9T!!! 8C= =U r'rc ^eZdZdZdedeffd ZdededefdZd ed ed e de efd Z xZ S) BaseTextGenerationTaskz Base class for text-generation (completion) tasks. The schema follows the OpenAI API format defined here: https://platform.openai.com/docs/api-reference/completions r5r6c*t|||dy)Nztext-generationrrrs r%r:zBaseTextGenerationTask.__init__Is (XDUVr'rjr@r cy)Nz/v1/completionsr#r{s r%rnz%BaseTextGenerationTask._prepare_routeLs r'r<r=rBc<td|i|d|jiS)Npromptr?rr}s r%rKz/BaseTextGenerationTask._prepare_payload_as_dictOs'Hfg gGEZEfEfghhr') rrrrrr:rnrr+r rrKrrs@r%rrCsf WWW!3!!!ii'+iD\i $ir'rN)maxsizer?r c|ddlm}|j|dg}|j}|t d||S)z; Fetch provider mappings for a model from the Hub. r)HfApiinferenceProviderMapping)expandz$No provider mapping found for model )huggingface_hub.hf_apir model_infoinference_provider_mappingr.)r?rrvras r%r]r]UsM - 7  e-G,H  ID66?wGHH r'dict1dict2c i||jDcic]>\}}|||vr2t||trt|trt|||n|@c}}Scc}}wr")r,r*r+rL)rrrfrgs r%rLrLcsx    $kkm  U u E#J!=*UTXBY!sU3    sAA)# functoolsrtypingrrrrhuggingface_hubrrr !huggingface_hub.inference._commonr r :huggingface_hub.inference._generated.types.chat_completionr huggingface_hub.utilsr rr get_loggerrr_rr+r__annotations__r&r-r4rrrr]rLr#r'r%rsR11%;JaFF   H % UU bUU bUBU BUBU"U  b!U" b#U$ b%U&'U()U*+U,-U./U01U2r3U!4T#7O2O-P(P#Q: ;T#s(^;S#X; ; 1T#Y1491 1@U4S>4945@%S#XPTUXPY@Y:Z@"@@Fk/k*# #9# Li/i$ 4 S T:T5U   4   r'