qidvdZddlZddlZddlZddlmZddlmZmZddl m Z m Z m Z m Z mZddlZddlmZddlmZmZmZmZmZddlmZmZdd lmZmZdd lmZdd l m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&dd l'm(Z(dd l)m*Z*ddl+m,Z,m-Z-ddl.m/Z/ddl0m1Z1m2Z2ddl3mZ4ddl3m5Z5m6Z6ddl7m8Z8eGdde5Z9Gdde2Z:y)zMBase LLM service implementation for services that use the AsyncOpenAI client.N)asynccontextmanager) dataclassfield)AnyDictListMappingOptional)logger) NOT_GIVENAPITimeoutError AsyncOpenAI AsyncStreamDefaultAsyncHttpxClient)ChatCompletionChunkChatCompletionMessageParam) BaseModelField)OpenAILLMInvocationParams)FrameLLMContextFrameLLMFullResponseEndFrameLLMFullResponseStartFrameLLMMessagesFrame LLMTextFrame) LLMTokenUsage) LLMContext)OpenAILLMContextOpenAILLMContextFrame)FrameDirection)FunctionCallFromLLM LLMServicer ) LLMSettings _NotGiven) traced_llmc6eZdZUdZedZeezed<y)OpenAILLMSettingsz~Settings for BaseOpenAILLMService. Parameters: max_completion_tokens: Maximum completion tokens to generate. ctSN) _NOT_GIVENR/opt/pipecat/venv/lib/python3.12/site-packages/pipecat/services/openai/base_llm.pyzOpenAILLMSettings.:s:r-default_factorymax_completion_tokensN) __name__ __module__ __qualname____doc__rr2intr%__annotations__r,r-r.r(r(2s .3CU-V3?Vr-r(ceZdZUdZeZeed<GddeZdddddddddddd d e e d e e e e fd e e d e ed e ede e de e ffdZ d$dZde fdZde fdZdZdedeefdZdedefdZ d%deezde ede e de e fdZdedeefdZdedeefdZedeezfd Z d!e!d"e"ffd# Z#xZ$S)&BaseOpenAILLMServiceaBase class for all services that use the AsyncOpenAI client. This service consumes OpenAILLMContextFrame or LLMContextFrame frames, which contain a reference to an OpenAILLMContext or LLMContext object. The context defines what is sent to the LLM for completion, including user, assistant, and system messages, as well as tool choices and function call configurations. _settingsceZdZUdZedddZeeed<edddZ eeed<ed d Z ee ed <ed ddZ eeed<edd Z ee ed<edddZeeed<edd Zee ed<edd Zee ed<edZeeed<eeZeeeefed<y) BaseOpenAILLMService.InputParamsaInput parameters for OpenAI model configuration. .. deprecated:: 0.0.105 Use ``settings=OpenAILLMSettings(...)`` instead of ``params=InputParams(...)``. Parameters: frequency_penalty: Penalty for frequent tokens (-2.0 to 2.0). presence_penalty: Penalty for new tokens (-2.0 to 2.0). seed: Random seed for deterministic outputs. temperature: Sampling temperature (0.0 to 2.0). top_k: Top-k sampling parameter (currently ignored by OpenAI). top_p: Top-p (nucleus) sampling parameter (0.0 to 1.0). max_tokens: Maximum tokens in response (deprecated, use max_completion_tokens). max_completion_tokens: Maximum completion tokens to generate. service_tier: Service tier to use (e.g., "auto", "flex", "priority"). extra: Additional model-specific parameters. ctSr*r#r,r-r.r/z)BaseOpenAILLMService.InputParams._Ir-gg@)r1gelefrequency_penaltyctSr*r#r,r-r.r/z)BaseOpenAILLMService.InputParams.br?r-presence_penaltyctSr*r#r,r-r.r/z)BaseOpenAILLMService.InputParams.dsIr-r)r1r@seedctSr*r#r,r-r.r/z)BaseOpenAILLMService.InputParams.esYr-g temperatureN)defaultr@top_kctSr*r#r,r-r.r/z)BaseOpenAILLMService.InputParams.isyr-g?top_pctSr*r#r,r-r.r/z)BaseOpenAILLMService.InputParams.js)r- max_tokensctSr*r#r,r-r.r/z)BaseOpenAILLMService.InputParams.ksYr-r2ctSr*r#r,r-r.r/z)BaseOpenAILLMService.InputParams.lsIr-r0 service_tierextra)r3r4r5r6rrBr floatr8rDrFr7rHrJrLrOr2rRstrdictrSrrr,r-r. InputParamsr=Js &.3-$3. 8E? -2-$3- (5/ $4E!LhsmL',=NSV[^'_ Xe__ %Ta8x}8!&7HSUX!YxY$):KPQ$R HSMR/4EV[\/]x}]&+> , , LLD6!=dnn>_>_=`a b -r-c dt||||ttjddd|S)aCreate an AsyncOpenAI client instance. Args: api_key: OpenAI API key. base_url: Custom base URL for the API. organization: OpenAI organization ID. project: OpenAI project ID. default_headers: Additional HTTP headers. **kwargs: Additional client configuration arguments. Returns: Configured AsyncOpenAI client instance. diN)max_keepalive_connectionsmax_connectionskeepalive_expiry)limits)rYrZr[r\ http_clientr])rrhttpxLimits)rrrYrZr[r\r]rss r.roz"BaseOpenAILLMService.create_clients>,%/||.14Z^ ,  r-returncy)zCheck if this service can generate processing metrics. Returns: True, as OpenAI service supports metrics generation. Tr,rrs r.can_generate_metricsz)BaseOpenAILLMService.can_generate_metricss r-full_model_namec||_y)zvSet the full AI model name. Args: full_model_name: The full name of the AI model to use. Nrn)rrrs r.set_full_model_namez(BaseOpenAILLMService.set_full_model_names !0r-c|jS)zqGet the current full model name. Returns: The full name of the AI model being used. rrs r.get_full_model_namez(BaseOpenAILLMService.get_full_model_names $$$r-params_from_contextcXK|j|}|jrZ tj|jj j jdi||jd{}|S|jj j jdi|d{}|S7@#ttjf$rVtj|d|jj j jdi|d{7}|cYSwxYw7zw)acGet streaming chat completions from OpenAI API with optional timeout and retry. Args: params_from_context: Parameters, derived from the LLM context, to use for the chat completion. Contains messages, tools, and tool choice. Returns: Async stream of chat completion chunks. )timeoutNz): Retrying chat completion due to timeoutr,) build_chat_completion_paramsrmasynciowait_forrpchat completionscreaterlr TimeoutErrorr rq)rrrr^chunkss r.get_chat_completionsz)BaseOpenAILLMService.get_chat_completions s 223FG  ! ! &//8DLL%%1188B6BDLdLd  @4<<,,88??I&IIFM$W%9%9:  v%NOPCt||00<<CCMfMMM   JsTD*AB63B44B694D*-D(.D*4B66A$D%DD%"D*$D%%D*c |jjdddi|jj|jj|jj|jj |jj |jj|jj|j |jntd }|j||j|jj|jjrc|jdg}|r/|djddk(rtj |dd|jjd g|z|d<|S) aBuild parameters for chat completion request. Subclasses can override this to customize parameters for different providers. Args: params_from_context: Parameters, derived from the LLM context, to use for the chat completion. Contains messages, tools, and tool choice. Returns: Dictionary of parameters for the chat completion request. T include_usage) rXstreamstream_optionsrBrDrFrHrLrOr2rRmessagesrrolesystem\: Both system_instruction and a system message in context are set. Using system_instruction.rcontent)r;rXrBrDrFrHrLrOr2rkr updaterSrcgetr warning)rrrr^rs r.rz1BaseOpenAILLMService.build_chat_completion_params*s:^^)).5!%!A!A $ ? ?NN''>>55^^))..33%)^^%I%I262D2D2PD..V_    )* dnn**+ >> , ,zz*b1HHQKOOF3x?f11 "dnn.O.OP""F:  r-contextrOrccKt|tr"|j}|j|}n,t |j |j |j}|j|}d|d<|jdd|O|jdg}|r/|djdd k(rtj|d d |d g|z|d<|d |vr||d <n||d <|jjjj di|d{}|j"dj$j&S7'w)alRun a one-shot, out-of-band (i.e. out-of-pipeline) inference with the given LLM context. Args: context: The LLM context containing conversation history. max_tokens: Optional maximum number of tokens to generate. If provided, overrides the service's default max_tokens/max_completion_tokens setting. system_instruction: Optional system instruction to use for this inference. If provided, overrides any system instruction in the context. Returns: The LLM's response as a string, or None if no response is generated. rtools tool_choiceFrrNrrrrrrr2rOr,)rgrget_llm_adapterget_llm_invocation_paramsrrrrrpoprr rrprrrchoicesmessager) rrrrOrcadapterinvocation_paramsr^rresponses r. run_inferencez"BaseOpenAILLMService.run_inferenceXs]$ gz ***,G;B;\;\< !: ))GL_L_!  223DE!x #T*  )zz*b1HHQKOOF3x?f11,4@R"S!TW_!_F:   !&&02<./'1|$>**66==GGG"**222HsDEE(EcKtj|d|j|j}|D]x}|j ddk(s|dj }t j|jd}|j dd}d|d d d d |id g|d<|d=|d=zt||j|j}|j|d{}|S7w)Nz,: Generating chat from LLM-specific context mime_typez image/jpegdatazutf-8rrftext)typer image_urlurlzdata:image/jpeg;base64,)rrr) r rqget_messages_for_logging get_messagesrreadbase64 b64encodedecoderrrr) rrrrr raw_bytes encoded_imagerr^rs r.)_stream_chat_completions_specific_contextz>BaseOpenAILLMService._stream_chat_completions_specific_contexts  f@AaAaAc@d e 6=5I5I5K  )G{{;'<7#FO002 & 0 0 ; B B7 K {{9b1#T2 +&+/F}o-V%W& "FOK( )"+W]]@S@S 0088 9sAC7BC7.C5/C7cK|j}tj|d|j||j |}|j |d{}|S7w)Nz): Generating chat from universal context )rr rqrrr)rrrrr^rs r.*_stream_chat_completions_universal_contextz?BaseOpenAILLMService._stream_chat_completions_universal_contextsm&&( f=g>^>^_f>g=h i -4,M,Mg,V0088 9sAA*!A("A*c Kg}g}g}d}d}d}d}|jd{t|tr|j|n|j |d{} t d} | | 4d{} | 23d{} | j r| j jr | j jjnd} | j jr | j jjnd}t| j j| j j| j j| |}|j|d{| j r8|j#| j k7r|j%| j | j&t)| j&dk(rT|j+d{| j&dj,s| j&dj,j.r| j&dj,j.d}|j0|k7r>|j3||j3||j3|d}d}d}|dz }|j4r;|j4j6r%||j4j6z }|j8}|j4sr|j4j:s||j4j:z }| j&dj,j<r<|j?| j&dj,j<d{tA| j&dj,ds)| j&dj,jBjEds]|jGtI| j&dj,jBdd{777777M776dddd{7n#1d{7swYnxYw|r|r|j3||j3||j3|g}tK|||D]9\}}}tMjN|}|j3tQ||||;|jS|d{7yyyw) Nrrfc K|j} |t|dr|jd{t|dr|jd{yt|dr|jd{yy7O7-7 #t|dr|jd{7t|dr|jd{7wt|dr|jd{7wwxYww)Nacloseclose) __aiter__hasattrrr)r chunk_iters r._closingz7BaseOpenAILLMService._process_context.._closings))+J *  :x0$++---67+ ,,.((VX. --/))/ .)) :x0$++---67+ ,,.((VX. --/))/smDBDB#DB $DB D D D D.B1/$DC%D9C<:DD) prompt_tokenscompletion_tokens total_tokenscache_read_input_tokensreasoning_tokensrNaudio transcript)r tool_call_id function_name arguments)*start_ttfb_metricsrgrrrrusageprompt_tokens_details cached_tokenscompletion_tokens_detailsrrrrrstart_llm_usage_metricsrXrrrlenstop_ttfb_metricsdelta tool_callsindexappendfunctionnameidrr_push_llm_textrrr push_framerzipjsonloadsr!run_function_calls)rrrfunctions_listarguments_list tool_id_listfunc_idxrrr chunk_streamrrchunkrrtokens tool_callfunction_callstool_ids r._process_contextz%BaseOpenAILLMService._process_contextsx    %%''' '#34  : :7 C@@I   *  *L)D dD dZ)C dC de;;!;;<< 99GG!"!;;@@ ==NN!% +&+kk&?&?*/++*G*G%*[[%=%=0=)9 F66v>>>;;4#;#;#=#L,,U[[9==(C ,>!,C,,...}}Q'--==#))44!& a 0 6 6 A A! DI (2&--m<&--i8$++L9(* $& ')  A  ))i.@.@.E.E%););)@)@@ '0||  ))i.@.@.J.J!Y%7%7%A%AA ]]1%++33--emmA.>.D.D.L.LMMMU]]1-33W=%--PQBRBXBXB^B^BbBb C//,u}}Q7G7M7M7S7ST`7a*bcccE ( 6D dC d&?/BNdG *D dD dD dD dD dT Y  ! !- 0  ! !) ,    -N58 6 1 y'!JJy1 %%' '%,&3"+  )).9 9 9+'=s"TP'9TP*T;P-<T?QP=P0P= CQ'P3(BQ)P6*DQ4Q A3Q?P9)Q+2QAQ P;!Q'T*T-T0P=3Q6Q9Q;Q=Q> T Q  TQ"Q Q"BT;S><Tframe directioncKt|||d{d}t|tr |j}ngt|t r |j}nJt|t r tj|j}n|j||d{|r |jtd{|jd{|j|d{|j'd{|jt)d{yy777u7_7H#tj$r@}|j!dd{7|j#d|d{7Yd}~d}~wt$$r)}|j#d||d{7Yd}~d}~wwxYw77#|j'd{7|jt)d{7wxYww)aLProcess frames for LLM completion requests. Handles OpenAILLMContextFrame, LLMContextFrame, LLMMessagesFrame, and LLMUpdateSettingsFrame to trigger LLM completions and manage settings. Args: frame: The frame to process. direction: The direction of frame processing. Non_completion_timeoutzLLM completion timeout) error_msg exceptionzError during completion: )ri process_framergrrrrr from_messagesrrrstart_processing_metricsrr}TimeoutException_call_event_handler push_error Exceptionstop_processing_metricsr)rrrrrerus r.rz"BaseOpenAILLMService.process_frameJsg#E9555 e2 3mmG  /mmG / 0'44U^^DG//%3 3 3  Aoo&?&ABBB33555++G44422444oo&=&?@@@ ! 6 4C54)) W../FGGGoo0HTUoVVV ^oo2KA30O[\o]]] ^5@22444oo&=&?@@@sHD2BHD5H%D=D7D=D9D=2D;3D=7H G H+G,H5H7D=9D=;D==GF $E'%F FF G GF;0F31F;6G;GGHHHG!H=H>HH)NNNNN)NN)%r3r4r5r6r(Settingsr8rrWr rUr rTboolrjrorrrrrrrrVrrrr7rrrr&rrr r __classcell__)rus@r.r:r:=s!H  #Fi#FP $7;&*(,04.1+0`c}`c"'#s("34`csm`c%`c,-`c%UO`c#4.`cH ! Fd030%#< ( )<,@Y,^b,b%),0 83..83SM83%SM 83 # 83t '  ( ) D !  ( ) F:.>.KF:F:P'A'A>'A'Ar-r:);r6rrr contextlibr dataclassesrrtypingrrrr r r}logurur openair r rrropenai.types.chatrrpydanticrr)pipecat.adapters.services.open_ai_adapterrpipecat.frames.framesrrrrrrpipecat.metrics.metricsr*pipecat.processors.aggregators.llm_contextr1pipecat.processors.aggregators.openai_llm_contextrr"pipecat.processors.frame_processorr pipecat.services.llm_servicer!r"pipecat.services.settingsr+r$r%(pipecat.utils.tracing.service_decoratorsr&r(r:r,r-r.rsT *(55 N%O2A>H=<? W W WtA:tAr-