qi-*dZddlZddlZddlmZddlmZmZmZm Z m Z m Z m Z ddl mZddlmZddlmZmZdd lmZdd lmZmZGd d eZGd deZGddeZy)z;OpenAI Realtime LLM context and aggregator implementations.N)logger)FrameFunctionCallResultFrameInterimTranscriptionFrameLLMMessagesUpdateFrameLLMSetToolsFrame LLMTextFrameTranscriptionFrame)OpenAILLMContext)FrameDirection) OpenAIAssistantContextAggregatorOpenAIUserContextAggregator)events)RealtimeFunctionCallResultFrameRealtimeMessagesUpdateFramecVeZdZdZd fd ZdZededdfdZdZ dZ d Z xZ S) OpenAIRealtimeLLMContextaOpenAI Realtime LLM context with session management and message conversion. Extends the standard OpenAI LLM context to support real-time session properties, instruction management, and conversion between standard message formats and realtime conversation items. c Jt|d||d||jy)aInitialize the OpenAIRealtimeLLMContext. Args: messages: Initial conversation messages. Defaults to None. tools: Available function tools. Defaults to None. **kwargs: Additional arguments passed to parent OpenAILLMContext. )messagestoolsN)super__init__&_OpenAIRealtimeLLMContext__setup_local)selfrrkwargs __class__s _/opt/pipecat/venv/lib/python3.12/site-packages/pipecat/services/openai_realtime_beta/context.pyrz!OpenAIRealtimeLLMContext.__init__*s( B(%B6B c.d|_d|_d|_y)NT)llm_needs_settings_updatellm_needs_initial_messages_session_instructionsrs r __setup_localz&OpenAIRealtimeLLMContext.__setup_local5s)-&*.'%'"r objreturnc|t|tr+t|tst|_|j |S)zUpgrade a standard OpenAI LLM context to a realtime context. Args: obj: The OpenAILLMContext instance to upgrade. Returns: The upgraded OpenAIRealtimeLLMContext instance. ) isinstancer rrr)r(s rupgrade_to_realtimez,OpenAIRealtimeLLMContext.upgrade_to_realtime<s1 c+ ,ZE]5^4CM     r c|jddk(r|jd}t|jdtrnd}|jdD]X}|jddk(r|d|jdzz }/tjd|jdd |Zt j dd t jd | g S|jddk(rM|jdr<|jdd}t j d|d|dd|ddStjd|y)zConvert a standard message format to a realtime conversation item. Args: message: The standard message dictionary to convert. Returns: A ConversationItem instance for the realtime API. roleusercontentr"typetext z+Unhandled content type in context message: z - message input_textr1r2r.r1r0 assistant tool_callsr function_callidfunctionname arguments)r1call_idr=r>z1Unhandled message type in from_standard_message: N)getr+listrerrorrConversationItem ItemContent)rr4r0ctcs rfrom_standard_messagez.OpenAIRealtimeLLMContext.from_standard_messageNsL ;;v & (kk),G'++i0$7 Y/AuuV}.3v#66 I!%%PV-X[\c[de **++GLM  ;;v + -'++l2K\*1-B**$4 ^F+Z.5    H RSr c F|jsgStj|j}|djddk(rnd|_|j d}|jd}t |tr||_n)t |tr|djd|_|sgSt|dk(r,|djddk(r|j|dgSd }d }dd d d j|tj|d|gdgdgS)a@Get conversation items for initializing the realtime session history. Converts the context's messages to a format suitable for the realtime API, handling system instructions and conversation history packaging. Returns: List of conversation items for session initialization. rr.systemTr0r2rr/z This is a previously saved conversation. Please treat this conversation history as a starting point for the current conversation.aS This is the end of the previously saved conversation. Please continue the conversation from here. If the last message is a user instruction or question, act on that instruction or answer the question. If the last message is an assistant response, simple say that you are ready to continue the conversation.r4r5z )indentr6r7)rcopydeepcopyr@r#popr+strr%rAlenrGjoinjsondumps)rrrIr0 intro_text trailing_texts r%get_messages_for_initializing_historyz>OpenAIRealtimeLLMContext.get_messages_for_initializing_historyqs$}}I==/ A;??6 "h .-1D *\\!_Fjj+G'3'-4*GT*-4QZ^^F-C*  x=A (1+//&"9V"C..x{;< <8 3 !!- & 'HQ)GW!   r cfdd|jdjdgd}|j|y)zAdd a user content item as a standard message to the context. Args: item: The conversation item to add as a user message. r/r2rr6)r.r0N)r0 transcript add_message)ritemr4s r add_user_content_item_as_messagez9OpenAIRealtimeLLMContext.add_user_content_item_as_messages7!'a1K1KLM  !r )NN) __name__ __module__ __qualname____doc__rr staticmethodr r,rGrVr[ __classcell__rs@rrr"sH  !1 6P  "!TF< | "r rcJeZdZdZej fdedeffd ZdZxZ S)#OpenAIRealtimeUserContextAggregatoraOUser context aggregator for OpenAI Realtime API. Handles user input frames and generates appropriate context updates for the realtime conversation, including message updates and tool settings. Args: context: The OpenAI realtime LLM context. **kwargs: Additional arguments passed to parent aggregator. frame directioncKt|||d{t|tr-|j t |j d{t|tr|j ||d{yy7m727 w)zProcess incoming frames and handle realtime-specific frame types. Args: frame: The frame to process. direction: The direction of frame flow in the pipeline. N)context)r process_framer+r push_framer_contextrrrerfrs rriz1OpenAIRealtimeUserContextAggregator.process_framesg#E9555 e3 4//"=dmm"TU U U e- .//%3 3 3 / 6 V 4s3B BB ?B B  B c Kyw)zPush user input aggregation. Currently ignores all user input coming into the pipeline as realtime audio input is handled directly by the service. Nrr&s rpush_aggregationz4OpenAIRealtimeUserContextAggregator.push_aggregations  s) r\r]r^r_r DOWNSTREAMrrirnrarbs@rrdrds.9G8Q8Q44'54*  r rdc<eZdZdZdedeffd Zdeffd ZxZ S)(OpenAIRealtimeAssistantContextAggregatoraGAssistant context aggregator for OpenAI Realtime API. Handles assistant output frames from the realtime service, filtering out duplicate text frames and managing function call results. Args: context: The OpenAI realtime LLM context. **kwargs: Additional arguments passed to parent aggregator. rerfczKt|tttfst|||d{yy7w)zProcess assistant frames, filtering out duplicate text content. Args: frame: The frame to process. direction: The direction of frame flow in the pipeline. N)r+r r rrrirls rriz6OpenAIRealtimeAssistantContextAggregator.process_frames:%,0BD]!^_''y9 9 9` 9s /;9;cKt||d{|jt|tj d{y777w)zHandle function call result and notify the realtime service. Args: frame: The function call result frame to handle. N) result_frame)rhandle_function_call_resultrjrr UPSTREAM)rrers rruzDOpenAIRealtimeAssistantContextAggregator.handle_function_call_resultsO g1%888 oo + ?AXAX   9  s!AA1AA AA) r\r]r^r_rr rirrurarbs@rrqrqs- ::>:  7N   r rq)r_rLrRlogururpipecat.frames.framesrrrrrr r 1pipecat.processors.aggregators.openai_llm_contextr "pipecat.processors.frame_processorr pipecat.services.openai.llmr rr"rframesrrrrdrqrr rr}sdB O= PW"/W"t) *E) X( /O( r