qi@7jdZddlZej5ejdejddddddlZddlZddlmZddl m Z m Z m Z m Z mZmZmZddlmZddlmZdd lmZmZd d lmZd d lmZmZGd deZGddeZGddeZy#1swYmxYw)a)OpenAI Realtime LLM context and aggregator implementations. .. deprecated:: 0.0.92 OpenAI Realtime no longer uses types from this module under the hood. It now uses ``LLMContext`` and ``LLMContextAggregatorPair``. Using the new patterns should allow you to not need types from this module. BEFORE:: # Setup context = OpenAILLMContext(messages, tools) context_aggregator = llm.create_context_aggregator(context) # Context aggregator type context_aggregator: OpenAIContextAggregatorPair # Context frame type frame: OpenAILLMContextFrame # Context type context: OpenAIRealtimeLLMContext # or context: OpenAILLMContext AFTER:: # Setup context = LLMContext(messages, tools) context_aggregator = LLMContextAggregatorPair(context) # Context aggregator type context_aggregator: LLMContextAggregatorPair # Context frame type frame: LLMContextFrame # Context type context: LLMContext NalwaysaTypes in pipecat.services.openai.realtime.llm (or pipecat.services.openai_realtime.llm) are deprecated. OpenAI Realtime no longer uses types from this module under the hood. It now uses `LLMContext` and `LLMContextAggregatorPair`. Using the new patterns should allow you to not need types from this module. BEFORE: ``` # Setup context = OpenAILLMContext(messages, tools) context_aggregator = llm.create_context_aggregator(context) # Context aggregator type context_aggregator: OpenAIContextAggregatorPair # Context frame type frame: OpenAILLMContextFrame # Context type context: OpenAIRealtimeLLMContext # or context: OpenAILLMContext ``` AFTER: ``` # Setup context = LLMContext(messages, tools) context_aggregator = LLMContextAggregatorPair(context) # Context aggregator type context_aggregator: LLMContextAggregatorPair # Context frame type frame: LLMContextFrame # Context type context: LLMContext ``` )logger)FrameFunctionCallResultFrameInterimTranscriptionFrameLLMMessagesUpdateFrameLLMSetToolsFrame LLMTextFrameTranscriptionFrame)OpenAILLMContext)FrameDirection) OpenAIAssistantContextAggregatorOpenAIUserContextAggregator)events)RealtimeFunctionCallResultFrameRealtimeMessagesUpdateFramecVeZdZdZd fd ZdZededdfdZdZ dZ d Z xZ S) OpenAIRealtimeLLMContextaOpenAI Realtime LLM context with session management and message conversion. Extends the standard OpenAI LLM context to support real-time session properties, instruction management, and conversion between standard message formats and realtime conversation items. .. deprecated:: 0.0.99 `OpenAIRealtimeLLMContext` is deprecated and will be removed in a future version. Use the universal `LLMContext` and `LLMContextAggregatorPair` instead. See `OpenAILLMContext` docstring for migration guide. c Jt|d||d||jy)aInitialize the OpenAIRealtimeLLMContext. Args: messages: Initial conversation messages. Defaults to None. tools: Available function tools. Defaults to None. **kwargs: Additional arguments passed to parent OpenAILLMContext. )messagestoolsN)super__init__&_OpenAIRealtimeLLMContext__setup_local)selfrrkwargs __class__s Z/opt/pipecat/venv/lib/python3.12/site-packages/pipecat/services/openai/realtime/context.pyrz!OpenAIRealtimeLLMContext.__init__{s( B(%B6B c.d|_d|_d|_y)NT)llm_needs_settings_updatellm_needs_initial_messages_session_instructionsrs r __setup_localz&OpenAIRealtimeLLMContext.__setup_locals)-&*.'%'"r!objreturnc|t|tr+t|tst|_|j |S)zUpgrade a standard OpenAI LLM context to a realtime context. Args: obj: The OpenAILLMContext instance to upgrade. Returns: The upgraded OpenAIRealtimeLLMContext instance. ) isinstancer rrr)r)s r upgrade_to_realtimez,OpenAIRealtimeLLMContext.upgrade_to_realtimes1 c+ ,ZE]5^4CM     r!c|jddk(r|jd}t|jdtrnd}|jdD]X}|jddk(r|d|jdzz }/tjd|jdd |Zt j dd t jd | g S|jddk(rM|jdr<|jdd}t j d|d|dd|ddStjd|y)zConvert a standard message format to a realtime conversation item. Args: message: The standard message dictionary to convert. Returns: A ConversationItem instance for the realtime API. roleusercontentr#typetext z+Unhandled content type in context message: z - message input_textr2r3r/r2r1 assistant tool_callsr function_callidfunctionname arguments)r2call_idr>r?z1Unhandled message type in from_standard_message: N)getr,listrerrorrConversationItem ItemContent)rr5r1ctcs r from_standard_messagez.OpenAIRealtimeLLMContext.from_standard_messagesL ;;v & (kk),G'++i0$7 Y/AuuV}.3v#66 I!%%PV-X[\c[de **++GLM  ;;v + -'++l2K\*1-B**$4 ^F+Z.5    H RSr!c F|jsgStj|j}|djddk(rnd|_|j d}|jd}t |tr||_n)t |tr|djd|_|sgSt|dk(r,|djddk(r|j|dgSd }d }dd d d j|tj|d|gdgdgS)a@Get conversation items for initializing the realtime session history. Converts the context's messages to a format suitable for the realtime API, handling system instructions and conversation history packaging. Returns: List of conversation items for session initialization. rr/systemTr1r3rr0z This is a previously saved conversation. Please treat this conversation history as a starting point for the current conversation.aS This is the end of the previously saved conversation. Please continue the conversation from here. If the last message is a user instruction or question, act on that instruction or answer the question. If the last message is an assistant response, simple say that you are ready to continue the conversation.r5r6z )indentr7r8)rcopydeepcopyrAr$popr,strr&rBlenrHjoinjsondumps)rrrJr1 intro_text trailing_texts r %get_messages_for_initializing_historyz>OpenAIRealtimeLLMContext.get_messages_for_initializing_historys$}}I==/ A;??6 "h .-1D *\\!_Fjj+G'3'-4*GT*-4QZ^^F-C*  x=A (1+//&"9V"C..x{;< <8 3 !!- & 'HQ)GW!   r!cfdd|jdjdgd}|j|y)zAdd a user content item as a standard message to the context. Args: item: The conversation item to add as a user message. r0r3rr7)r/r1N)r1 transcript add_message)ritemr5s r add_user_content_item_as_messagez9OpenAIRealtimeLLMContext.add_user_content_item_as_messages7!'a1K1KLM  !r!)NN) __name__ __module__ __qualname____doc__rr staticmethodr r-rHrWr\ __classcell__rs@r rrnsH   !1 6P  "!TF< | "r!rcJeZdZdZej fdedeffd ZdZxZ S)#OpenAIRealtimeUserContextAggregatora]User context aggregator for OpenAI Realtime API. Handles user input frames and generates appropriate context updates for the realtime conversation, including message updates and tool settings. .. deprecated:: 0.0.99 `OpenAIRealtimeUserContextAggregator` is deprecated and will be removed in a future version. Use the universal `LLMContext` and `LLMContextAggregatorPair` instead. See `OpenAILLMContext` docstring for migration guide. Args: context: The OpenAI realtime LLM context. **kwargs: Additional arguments passed to parent aggregator. frame directioncKt|||d{t|tr-|j t |j d{t|tr|j ||d{yy7m727 w)zProcess incoming frames and handle realtime-specific frame types. Args: frame: The frame to process. direction: The direction of frame flow in the pipeline. N)context)r process_framer,r push_framer_contextr rrfrgrs r rjz1OpenAIRealtimeUserContextAggregator.process_frame sg#E9555 e3 4//"=dmm"TU U U e- .//%3 3 3 / 6 V 4s3B BB ?B B  B c Kyw)zPush user input aggregation. Currently ignores all user input coming into the pipeline as realtime audio input is handled directly by the service. Nrr's r push_aggregationz4OpenAIRealtimeUserContextAggregator.push_aggregation5s  s) r]r^r_r`r DOWNSTREAMrrjrorbrcs@r reres. $9G8Q8Q44'54*  r!rec<eZdZdZdedeffd Zdeffd ZxZ S)(OpenAIRealtimeAssistantContextAggregatoraZAssistant context aggregator for OpenAI Realtime API. Handles assistant output frames from the realtime service, filtering out duplicate text frames and managing function call results. .. deprecated:: 0.0.99 `OpenAIRealtimeAssistantContextAggregator` is deprecated and will be removed in a future version. Use the universal `LLMContext` and `LLMContextAggregatorPair` instead. See `OpenAILLMContext` docstring for migration guide. Args: context: The OpenAI realtime LLM context. **kwargs: Additional arguments passed to parent aggregator. rfrgczKt|tttfst|||d{yy7w)zProcess assistant frames, filtering out duplicate text content. Args: frame: The frame to process. direction: The direction of frame flow in the pipeline. N)r,r r rrrjrms r rjz6OpenAIRealtimeAssistantContextAggregator.process_frameYs:%,0BD]!^_''y9 9 9` 9s /;9;cKt||d{|jt|tj d{y777w)zHandle function call result and notify the realtime service. Args: frame: The function call result frame to handle. N) result_frame)rhandle_function_call_resultrkrr UPSTREAM)rrfrs r rvzDOpenAIRealtimeAssistantContextAggregator.handle_function_call_resultcsO g1%888 oo + ?AXAX   9  s!AA1AA AA) r]r^r_r`rr rjrrvrbrcs@r rrrrAs- .::>:  7N   r!rr) r`warningscatch_warnings simplefilterwarnrMrSlogururpipecat.frames.framesrrrrr r r 1pipecat.processors.aggregators.openai_llm_contextr "pipecat.processors.frame_processorr pipecat.services.openai.llmrrr#rframesrrrrerrrr!r rs&PX"H(#HMM  "H O= P]"/]"@0 *E0 f/ /O/ a""s %B))B2