qi?dZddlZddlZddlZddlmZddlmZmZm Z m Z m Z m Z m Z mZmZddlmZddlmZddlmZmZddlmZdd lmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,dd l-m.Z.m/Z/dd l0m1Z1m2Z2dd l3m4Z4dd l5m6Z6ddl7m8Z8ddl9m:Z:ddl;mZ>m?Z?e dgedfZ@Gdde ZAeGddZBeGddZCeGddZDGdder?str__annotations__rrr)r%r2r8r@r:r0r0\s; sCx    **//r@cZeZdZUdZeeed<edzed<eed<eed<dZ ee ed<y) FunctionCallRegistryItemaRepresents an entry in the function call registry. This is what the user registers when calling register_function. Parameters: function_name: The name of the function (None for catch-all handler). handler: The handler for processing function call parameters. cancel_on_interruption: Whether to cancel the call on interruption. timeout_secs: Optional per-tool timeout in seconds. Overrides the global ``function_call_timeout_secs`` for this specific function. rBrhandlercancel_on_interruptionhandler_deprecatedN timeout_secs) r<r=r>r?r rIrJFunctionCallHandlerboolrPfloatr8r@r:rLrLqs9 C= #: ::  $(L(5/(r@rLcheZdZUdZeed<eed<eed<eeefed<e e zed<dZ e e ed<y) FunctionCallRunnerItemaInternal function call entry for the function call runner. The runner executes function calls in order. Parameters: registry_item: The registry item containing handler information. function_name: The name of the function. tool_call_id: A unique identifier for the function call. arguments: The arguments for the function. context: The LLM context. run_llm: Optional flag to control LLM execution after function call. registry_itemrBrCrDrGNrun_llm)r<r=r>r?rLrJrIrrr)r%rWr rRr8r@r:rUrUsC ,+sCx   **"GXd^"r@rUc,eZdZUdZeed<eZee ed< dHde de d e effd Z d e fd Zd ed efdZ dIdeezde ede ed e efdZeeddededed efdZdeffd Zdeffd Zdeffd ZdZ ded e!eefffd Z"de#de$ffd Z%e$jLfde#de$ffd Z'd!efd"Z(d#e)fd$Z*de+fd%Z,de+fd&Z-de+d e.eeffd'Z/ dJddd(d)e ed*ed+e d,e e fd-Z0ddd(d*e1d+e d,e e fd.Z2d)e efd/Z3d*efd0Z4d)efd1Z5d2e6e7fd3Z8dddddd4d5ed)e ed6e ed7e ed8e ed9e e f d:Z9d;Z:d<Z;d=ZZ=d?e6e>fd@Z?d?e6e>fdAZ@deezd)efdBZAdCe>fdDZBd)e efdEZCdFeDjfdGZFxZGS)KrEacBase class for all LLM services. Handles function calling registration and execution with support for both parallel and sequential execution modes. Provides event handlers for completion timeouts and function call lifecycle events. The service supports the following event handlers: - on_completion_timeout: Called when an LLM completion timeout occurs - on_function_calls_started: Called when function calls are received and execution is about to start Example:: @task.event_handler("on_completion_timeout") async def on_completion_timeout(service): logger.warning("LLM completion timed out") @task.event_handler("on_function_calls_started") async def on_function_calls_started(service, function_calls): logger.info(f"Starting {len(function_calls)} function calls") _settings adapter_classT$@Nrun_in_parallelfunction_call_timeout_secssettingsc :t|dd|xs ti|||_||_d|_d|_i|_|j|_ i|_ i|_ d|_ d|_ d|_|jd|jdy)aInitialize the LLM service. Args: run_in_parallel: Whether to run function calls in parallel or sequentially. Defaults to True. function_call_timeout_secs: Timeout in seconds for deferred function calls. Defaults to 10.0 seconds. settings: The runtime-updatable settings for the LLM service. **kwargs: Additional arguments passed to the parent AIService. r^FNon_function_calls_startedon_completion_timeoutr8)super__init__r,_run_in_parallel_function_call_timeout_secs_filter_incomplete_user_turns_base_system_instruction_start_callbacksrZ_adapter _functions_function_call_tasks_sequential_runner_task _skip_tts _summary_task_register_event_handler)r9r\r]r^kwargs __class__s r:rczLLMService.__init__s$  }    !0+E(38*7;% "**, IKZ\!?C$)-59 $$%@A $$%<=r@r6c|jS)zuGet the LLM adapter instance. Returns: The adapter instance used for LLM communication. )rir9s r:get_llm_adapterzLLMService.get_llm_adapters }}r@messagec@|jj|S)zCreate an LLM-specific message (as opposed to a standard message) for use in an LLMContext. Args: message: The message content. Returns: A LLMSpecificMessage instance. )rtcreate_llm_specific_message)r9rus r:rwz&LLMService.create_llm_specific_messages##%AA'JJr@rG max_tokenssystem_instructioncNKtd|jjw)aRun a one-shot, out-of-band (i.e. out-of-pipeline) inference with the given LLM context. Must be implemented by subclasses. Args: context: The LLM context containing conversation history. max_tokens: Optional maximum number of tokens to generate. If provided, overrides the service's default max_tokens/max_completion_tokens setting. system_instruction: Optional system instruction to use for this inference. If provided, overrides any system instruction in the context. Returns: The LLM's response as a string, or None if no response is generated. z!run_inference() not supported by )NotImplementedErrorrqr<)r9rGrxrys r: run_inferencezLLMService.run_inferences&("$EdnnF]F]E^"_``s#%) user_paramsassistant_paramsr}r~ctj5tjdtjdtddddy#1swYyxYw)aCreate a context aggregator for managing LLM conversation context. Must be implemented by subclasses. Args: context: The LLM context to create an aggregator for. user_params: Parameters for user message aggregation. assistant_params: Parameters for assistant message aggregation. Returns: A context aggregator instance. .. deprecated:: 0.0.99 `create_context_aggregator()` is deprecated and will be removed in a future version. Use the universal `LLMContext` and `LLMContextAggregatorPair` instead. See `OpenAILLMContext` docstring for migration guide. alwayszcreate_context_aggregator() is deprecated and will be removed in a future version. Use the universal LLMContext and LLMContextAggregatorPair directly instead. See OpenAILLMContext docstring for migration guide. stacklevelN)warningscatch_warnings simplefilterwarnDeprecationWarning)r9rGr}r~s r:create_context_aggregatorz$LLMService.create_context_aggregator sR0 $ $ &   ! !( + MMF#      s 2AAframecKt||d{|js|jd{yy7*7w)zRStart the LLM service. Args: frame: The start frame. N)rbstartrd_create_sequential_runner_taskr9rrqs r:rzLLMService.start/sD gmE"""$$557 7 7% # 7sAA#AAAAcKt||d{|js|jd{|j d{y7A77 w)zOStop the LLM service. Args: frame: The end frame. N)rbstoprd_cancel_sequential_runner_task_cancel_summary_taskrs r:rzLLMService.stop9sU gl5!!!$$557 7 7''))) " 7)1AA#AAAAAAAcKt||d{|js|jd{|j d{y7A77 w)zTCancel the LLM service. Args: frame: The cancel frame. N)rbcancelrdrrrs r:rzLLMService.cancelDsU gnU###$$557 7 7''))) $ 7)rc|j}|jj}|r|d||j_y||j_y)zCompose system_instruction by appending turn completion instructions. Combines the base system instruction with turn completion instructions and writes the result to ``self._settings.system_instruction``. z N)rg_user_turn_completion_configcompletion_instructionsrYry)r9basers r:_compose_system_instructionz&LLMService._compose_system_instructionOsI ,,"&"C"C"["[ 37&=T+ 6 4 93 5skFE6(FE9A!F"E<#%F2E>:A FF *F/F 0F9F<F>FF FcKt|tttfr|j|j|_t |||d{y7w)zPushes a frame. Args: frame: The frame to push. direction: The direction of frame pushing. N)rr!r rrmrrb push_frame)r9rrrqs r:rzLLMService.push_framesF el,EG^_ `~~)!%g  222sA AAAtextcK|jr|j|d{y|jt|d{y7(7w)a Push LLM text, using turn completion detection if enabled. This helper method simplifies text pushing in LLM implementations by handling the conditional logic for turn completion internally. Args: text: The text content from the LLM to push. N)rf_push_turn_textrr!)r9rs r:_push_llm_textzLLMService._push_llm_textsE  - -&&t, , ,//,t"45 5 5 - 5s!!AA "AAAA_cK|jjD]+\}}|js|j|d{-y7wN)rjitemsrN_cancel_function_call)r9rrBentrys r:rz LLMService._handle_interruptionssI$(OO$9$9$; @ M5++00??? @?s-AAA AcKtj|d|j|j|j ||_yw)aHandle context summarization request from aggregator. Processes a summarization request by generating a compressed summary of conversation history. Uses the adapter to format the summary according to the provider's requirements. Broadcasts the result back to the aggregator for context reconstruction. Args: frame: The summary request frame containing context and parameters. z#: Processing summarization request N)r debug request_id create_task_generate_summary_taskrn)r9rs r:rz"LLMService._handle_summary_requestsG  v@AQAQ@RST"--d.I.I%.PQsA A cKd}d}d}|jxst} tj|j ||d{\}}|jt|j||| d{d|_ y7;#tj $r"|j d|dd{7Yjt$r+}d|}|j || d{7Yd}~d}~wwxYw7xw) zBackground task to generate summary without blocking the pipeline. Args: frame: The summary request frame containing context and parameters. N)timeoutz&Context summarization timed out after s) error_msgz"Error generating context summary: ) exception)rsummarylast_summarized_indexerror) summarization_timeoutr.asynciowait_for_generate_summary TimeoutError push_error Exceptionbroadcast_framerrrn)r9rr last_indexrres r:rz!LLMService._generate_summary_tasks  --N1N 6(/(8(8&&u-)# GZ"" (''", #   "%### a//.TU\T]]^,_/` ` ` 68Ktj|j|j}|js#t j |dtdt j |dt|jd|jd|jtj|j}tdd|dg } |j||j|j d {}|s td|j%}t j&|dt|dt|jd||jfS7i#t$r$td |j j"d wxYww)a@Generate a compressed summary of conversation context. Uses the message selection logic to identify which messages to summarize, formats them as a transcript, and invokes the LLM to generate a concise summary. The summary is formatted according to the LLM provider's requirements using the adapter. Args: frame: The summary request frame containing context and configuration. Returns: Tuple of (formatted summary message, last_summarized_index). Raises: RuntimeError: If there are no messages to summarize, the service doesn't support run_inference(), or the LLM returns an empty summary. Note: Requires the service to implement run_inference() method for synchronous LLM calls. z: No messages to summarizezNo messages to summarizez: Generating summary for z messages (index 0 to z), target_context_tokens=userzConversation history: )rolecontent)messages)rxryNz LLM service z! does not implement run_inferencezLLM returned empty summaryz: Generated summary of z characters for z messages)r/get_messages_to_summarizerGmin_messages_to_keeprr r RuntimeErrorlenrtarget_context_tokensformat_messages_for_summaryr%r|summarization_promptr{rqr<stripr)r9rr5 transcriptsummary_context summary_texts r:rzLLMService._generate_summarys.-FF MM555  LLD6!;< =9: : f-c&//.B-CD!7789%%*%@%@$A C 1LLV__] $%4KJ<2XYZ  !%!3!3 66#(#=#="4"L;< <#))+  f+C ,=+>?v'(  3 V9999' # t~~6677XY  s1CF+E-E+E-A%F+E---FF)rNrPrBrMrNrPc tj|}t|jdkD}|rLt j 5t j dt jdtdddt||||||j|<|r\t j 5t j dt jdtddd||j|<yy#1swYxYw#1swY&xYw)aRegister a function handler for LLM function calls. Args: function_name: The name of the function to handle. Use None to handle all function calls with a catch-all handler. handler: The function handler. Should accept a single FunctionCallParams parameter. start_callback: Legacy callback function (deprecated). Put initialization code at the top of your handler instead. .. deprecated:: 0.0.59 The `start_callback` parameter is deprecated and will be removed in a future version. cancel_on_interruption: Whether to cancel this function call when an interruption occurs. Defaults to True. timeout_secs: Optional per-tool timeout in seconds. Overrides the global ``function_call_timeout_secs`` for this specific function. Defaults to None, which uses the global timeout. rzFunction calls with parameters `(function_name, tool_call_id, arguments, llm, context, result_callback)` are deprecated, use a single `FunctionCallParams` parameter instead.NrBrMrNrOrPzhParameter 'start_callback' is deprecated, just put your code on top of the actual function call instead.) inspect signaturer parametersrrrrrrLrjrh)r9rBrMstart_callbackrNrPrrOs r:register_functionzLLMService.register_function;s8%%g.  !5!56: ((* %%h/ D& *B'#91% *  & ((* %%h/ ~& 4BD ! !- 0 %  &  s0C8/0D8DD c|t|}t|j||d||j|j<y)aRegister a direct function handler for LLM function calls. Direct functions have their metadata automatically extracted from their signature and docstring, eliminating the need for accompanying configurations (as FunctionSchemas or in provider-specific formats). Args: handler: The direct function to register. Must follow DirectFunction protocol. cancel_on_interruption: Whether to cancel this function call when an interruption occurs. Defaults to True. timeout_secs: Optional per-tool timeout in seconds. Overrides the global ``function_call_timeout_secs`` for this specific function. Defaults to None, which uses the global timeout. FrN)rrLnamerj)r9rMrNrPwrappers r:register_direct_functionz#LLMService.register_direct_functionvs8*(0(@!,,#9$% )  %r@cV|j|=||jvr|j|=yy)zRemove a registered function handler. Args: function_name: The name of the function handler to remove. N)rjrhr9rBs r:unregister_functionzLLMService.unregister_functions1 OOM * D11 1%%m4 2r@cHt|}|j|j=y)zRemove a registered direct function handler. Args: handler: The direct function handler to remove. N)rrjr)r9rMrs r:unregister_direct_functionz%LLMService.unregister_direct_functions (0 OOGLL )r@ctd|jjvry||jjvS)zCheck if a function handler is registered. Args: function_name: The name of the function to check. Returns: True if the function is registered or if a catch-all handler (None) is registered. NT)rjkeysrs r: has_functionzLLMService.has_functions4 4??'') ) 4 4 666r@function_callsc Kt|dk(ry|jd|d{|jt|d{g}|D]}|j|j j vr|j |j}nRd|j j vr|j d}n&tj|d|jd|jt||j|j|j|j|jr|j|d{y|j!|d{y7F7*7%7 w)a#Execute a sequence of function calls from the LLM. Triggers the on_function_calls_started event and executes functions either in parallel or sequentially based on the run_in_parallel setting. Args: function_calls: The function calls to execute. rNr`)rz is calling 'z', but it's not registered.)rVrBrCrDrG)r_call_event_handlerrrrBrjrr warningappendrUrCrDrGrd_run_parallel_function_calls_run_sequential_function_calls)r9r runner_items function_callitems r:run_function_callszLLMService.run_function_callss] ~ ! # &&'BNSSS""#<^"\\\ + M**doo.B.B.DD}'B'BC--//t,fM-*E*E)FFab   &"&"/"="=!.!;!;+55)11   *  33LA A A55lC C C; T\2 B CsF%E7E-E7E0DE7 E3E7'E5(E70E73E75E7)rBrC text_content video_sourceruser_idrCrrrc .Ktj5tjdtjdtddd|j t ||d|||tjd{y#1swYAxYw7w)aRequest an image from a user. Pushes a UserImageRequestFrame upstream to request an image from the specified user. The user image can then be processed by the LLM. Use this function from a function call if you want the LLM to process the image. If you expect the image to be processed by a vision service, you might want to push a UserImageRequestFrame upstream directly. .. deprecated:: 0.0.92 This method is deprecated, push a `UserImageRequestFrame` instead. Args: user_id: The ID of the user to request an image from. function_name: Optional function name associated with the request. tool_call_id: Optional tool call ID associated with the request. text_content: Optional text content/context for the image request. video_source: Optional video source identifier. timeout: Optional timeout for the requested image to be added to the LLM context. rzUMethod `request_image_frame()` is deprecated, push a `UserImageRequestFrame` instead.NT)rrappend_to_contextrBrCrG) rrrrrrr$r*UPSTREAM)r9rrBrCrrrs r:request_image_framezLLMService.request_image_frames> $ $ &   ! !( + MMg"   oo !!"&+)$   # #     s(B0B:BBBB BcK|js>tj|_|j |j |_yywr)rlrQueue_sequential_runner_queuer_sequential_runner_handlerrss r:rz)LLMService._create_sequential_runner_tasks>++,3MMOD )+/+;+;D<[<[<]+^D (,sA Ac~K|jr+|j|jd{d|_yy7 wr)rl cancel_taskrss r:rz)LLMService._cancel_sequential_runner_tasks9  ' '""4#?#?@ @ @+/D ( ( @ +=;=c~K|jr+|j|jd{d|_yy7 wr)rnrrss r:rzLLMService._cancel_summary_tasks9   ""4#5#56 6 6!%D   6rcK |jjd{}|j|j|}||j|<|d{|j|=i7K7wr)r getr_run_function_callrk)r9 runner_itemtasks r:r z%LLMService._sequential_runner_handler"sp $ = = A A CCK##D$;$;K$HID.9D % %d +JJ))$/C s!A1A-8A1A/A1/A1rcKg}|D]]}|j|j|}|j|||j|<|j |j _ywr)rrrrkadd_done_callback_function_call_task_finished)r9rtasksrrs r:rz'LLMService._run_parallel_function_calls,sg' FK##D$;$;K$HID LL .9D % %d +  " "4#D#D E  FsA&A(cdK|D]%}|jj|d{'y7wr)r put)r9rrs r:rz)LLMService._run_sequential_function_calls4s4' AK//33K@ @ @ A @s $0.0cK||jjvr |j||||d{yd|jjvr|jd|||d{Sy7A7wr)rhr)r9rGrBs r:_call_start_functionzLLMService._call_start_function9s D11668 86$'' 6}dGT T T T**//1 14..t4]D'RR R2 URs!7A?A;;A?5A=6A?=A?rc Kjjjvrjjn-djjvrjdnytjdjdj dj jjjd{jtjj j jd{ddddtdttffdfd }j| t!j"d d{t%j&t(rgj&j+j t-jj j j  d{nj.rGj'jj j jd{nRt-jj j j }j'|d{r+j7sj9d{yyy777_777?#t0$rS}d jd|}tj2d|j5||dd{7Yd}~d}~wwxYw7u#r,j7sj9d{7wwwxYww)Nz Calling function [:z] with arguments )rBrCrDrNr3r5r4c Kr)jsjd{jtjj j |j|d{y7Q7w)N)rBrCrDr5rWr4)donerrrrBrCrDrW)r5r4rr9 timeout_tasks r:function_call_result_callbackzDLLMService._run_function_call..function_call_result_callback_s L$5$5$7&&|444&&')77(55%//#++%'  5 s"'BA<A B6A>7B>Bc TK j jn j}tj|d{t j dj djd|ddd{y7J7#tj$rwxYww)N Function call [rz] timed out after z seconds. You can increase this timeout by passing `timeout_secs` to `register_function()`, or set a global default via `function_call_timeout_secs` on the LLM constructor.) rPrersleepr rrBrCCancelledError)effective_timeoutr#rrr9s r:timeout_handlerz6LLMService._run_function_call..timeout_handlerss ((4%%99" mm$5666f,[-F-F,GqIaIaHbbtvGuHHhi 4D999 7 :))  sAB( >??;#<#<=D T__))+ +??4(D  f' (A(A'B!KD\D\C]]nozpEpEoF G '' (;(;[=V=VWWW "" '%33$11!++#'#>#> #   04 RV  (01M(N ( "''(9: - 5--" " "$,,(=>ll))$..-&1&?&?%0%=%="-"7"7 + 3 3(E *   **,,#11#00#--#++5 0&1&?&?%0%=%="-"7"7 + 3 3(E F,,v... L$5$5$7&&|444%8|I X  l # "/ U=k>W>W=XX[\][^_M LLD6=/2 3//MQe/T T T U 5L$5$5$7&&|444%8|sCM>KA M>&K'AM>*K*K!A?K*K$AK*K&AK*(K()K*-&M>M M>M>!K*$K*&K*(K** M3AM6L97M<M MM M> 'M;2M53M;;M>c FKt}|jjD]\}}|jj|k(s |j}|j }t j|d|d|d|rE|j|j|j|d{|j||jt||d{t j|d|d|d|D]}|j|y7n7?w)Nz Cancelling function call [rz]...)rBrCr%z] has been cancelled)setrkrrVrBrCr rremove_done_callbackrraddrr)r9rBcancelled_tasksrrrrCs r:rz LLMService._cancel_function_calls)%!%!:!:!@!@!B a D+((66-G"00*77  v%@a ~UYZ[--d.O.OP**4000#''-**+4l+ v%5dV1\NJ^_`' a,$ 4D  - -d 3 41s+AD!A'D!.D/0D!D >D!D!rc<||jvr|j|=yyr)rk)r9rs r:rz'LLMService._function_call_task_finisheds# 4,, ,))$/ -r@)Tr[N)NNr)Hr<r=r>r?r,rJrrZr rrRrSr rcrtrr&rwr%r)intrIr|r(r'rr#rrrrrrdictrrr*r DOWNSTREAMrrrrrrrtuplerrrrrrrr rrrrrrr rUrrrrrrTaskr __classcell__)rqs@r:rErEst.+;M4':!%,0*. &>&>%*&>;' &>P K3 K;M K%),0 a..aSMa%SM a # a40G/H9U9W ! !! - ! 7 ! ! F88 * * *+ * H*K*DcN*X66>6>JXIbIb 3e 3 3 6 6@->@ R3PR "2O"BA:-JA:uUXZ]U]A:N 9B (,(,9B}9B9B !% 9Buo9B~(,(,   !%  uo  <5#5*#* 7# 7)Dx@S7T)D^(,&*&*&*#'0 0  } 0 sm 0 sm 0 sm0 %0 d_ 0 & 0FxH^?_FAJ`AaA S'*4SEHSr54Jr5h4#4600r@rE)Fr?rrr dataclassesrtypingrrrrrr r r r logurur !pipecat.adapters.base_llm_adapterr(pipecat.adapters.schemas.direct_functionrr)pipecat.adapters.services.open_ai_adapterrpipecat.frames.framesrrrrrrrrrrrrrrr r!r"r#r$*pipecat.processors.aggregators.llm_contextr%r&+pipecat.processors.aggregators.llm_responser'r(1pipecat.processors.aggregators.openai_llm_contextr)"pipecat.processors.frame_processorr*pipecat.services.ai_servicer+pipecat.services.settingsr,(pipecat.turns.user_turn_completion_mixinr-/pipecat.utils.context.llm_context_summarizationr.r/rQr2r0rLrUrEr8r@r:rKsT!   <ZF*O=11V  45yFG  $ 00 0( )) )( ## #,u 02Iu 0r@