qi`dZddlZddlmZmZddlmZmZmZerddl m Z ddl m Z ddl mZmZdZd Zd Zd Zd Zd ZdZeGddZeGddZeGddZeGddZGddZy)zUtility for context summarization in LLM services. This module provides reusable functionality for automatically compressing conversation context when token limits are reached, enabling efficient long-running conversations. N) dataclassfield) TYPE_CHECKINGListOptional) LLMService)logger) LLMContextLLMSpecificMessageg^@ i皙?daYou are summarizing a conversation between a user and an AI assistant. Your task: 1. Create a concise summary that preserves: - Key facts, decisions, and agreements - Important context needed to continue the conversation - User preferences and requirements mentioned - Any unresolved questions or action items 2. Format: - Use clear, factual statements - Group related information - Prioritize information likely to be referenced later - Keep the summary concise to fit within the specified token budget 3. Omit: - Greetings and small talk - Redundant information - Tangential discussions that were resolved The conversation transcript follows. Generate only the summary, no other text.ceZdZUdZdZeed<dZeed<dZe e ed<dZ e ed <dZ e d ed <e Zeed <d Zede fdZy)LLMContextSummaryConfigaJConfiguration for summary generation parameters. Contains settings that control how a summary is generated. Used by both automatic and manual summarization modes. Parameters: target_context_tokens: Maximum token size for the generated summary. This value is passed directly to the LLM as the max_tokens parameter when generating the summary. Should be sized appropriately to allow the summary plus recent preserved messages to fit within reasonable context limits. min_messages_after_summary: Number of recent messages to preserve uncompressed after each summarization. These messages maintain immediate conversational context. summarization_prompt: Custom prompt for the LLM to use when generating summaries. If None, uses DEFAULT_SUMMARIZATION_PROMPT. summary_message_template: Template for formatting the summary when injected into context. Must contain ``{summary}`` as a placeholder for the generated summary text. Allows applications to wrap the summary in custom delimiters (e.g., XML tags) so that system prompts can distinguish summaries from live conversation. llm: Optional separate LLM service for generating summaries. When set, summarization requests are sent to this service instead of the pipeline's primary LLM. Useful for routing summarization to a cheaper/faster model (e.g., Gemini Flash) while keeping an expensive model for conversation. If None, uses the pipeline LLM. summarization_timeout: Maximum time in seconds to wait for the LLM to generate a summary. If the call exceeds this timeout, summarization is aborted with an error and future summarizations are unblocked. ptarget_context_tokensr min_messages_after_summaryNsummarization_promptConversation summary: {summary}summary_message_templaterllmsummarization_timeoutcl|jdkr td|jdkr tdy)"Validate configuration parameters.r&target_context_tokens must be positivez/min_messages_after_summary must be non-negativeN)r ValueErrorrselfs a/opt/pipecat/venv/lib/python3.12/site-packages/pipecat/utils/context/llm_context_summarization.py __post_init__z%LLMContextSummaryConfig.__post_init__as;  % % *EF F  * *Q .NO O /returnc*|jxstSzGet the summarization prompt to use. Returns: The custom prompt if set, otherwise the default summarization prompt. rDEFAULT_SUMMARIZATION_PROMPTrs r summary_promptz&LLMContextSummaryConfig.summary_prompth((H,HHr")__name__ __module__ __qualname____doc__rint__annotations__rrrstrrrDEFAULT_SUMMARIZATION_TIMEOUTrfloatr!propertyr(r"r rr9sw>"&3%&''*.(3-.$EcE"&C, &#@5@PIIIr"rc\eZdZUdZdZeeed<dZeeed<e e Z e ed<dZ y ) !LLMAutoContextSummarizationConfiguConfiguration for automatic context summarization. Controls when conversation context is automatically compressed and how that summary is generated. Summarization is triggered when either the token limit or the unsummarized message count threshold is exceeded. At least one of ``max_context_tokens`` and ``max_unsummarized_messages`` must be set. Set the other to ``None`` to disable that threshold. Parameters: max_context_tokens: Maximum allowed context size in tokens. When this limit is reached, summarization is triggered to compress the context. The tokens are calculated using the industry-standard approximation of 1 token ≈ 4 characters. Set to ``None`` to disable token-based triggering. max_unsummarized_messages: Maximum number of new messages that can accumulate since the last summary before triggering a new summarization. This ensures regular compression even if token limits are not reached. Set to ``None`` to disable message-count triggering. summary_config: Configuration for summary generation parameters (prompt, token budget, messages to keep). If not provided, uses default ``LLMContextSummaryConfig`` values. @max_context_tokensmax_unsummarized_messages)default_factorysummary_configc|j|j td|j|jdkr td|j|jdkr td|jL|jj|jkDr(t |jdz|j_yyy)rNLAt least one of max_context_tokens and max_unsummarized_messages must be setr#max_context_tokens must be positive,max_unsummarized_messages must be at least 1r)r8r:rr<rr.rs r r!z/LLMAutoContextSummarizationConfig.__post_init__s  " " *t/M/M/U^   " " .43J3Ja3OBC C  ) ) 5$:X:X[\:\KL L  # # /##99D2)- ,/1x}1.3D[.\N+\[r"r6ceZdZUdZdZeeed<dZeed<dZ eeed<dZ eed <d Z ee ed <d Z e ed <d Zeded<eZeed<dZede fdZdefdZy )LLMContextSummarizationConfigaConfiguration for context summarization behavior. .. deprecated:: 0.0.104 Use :class:`LLMAutoContextSummarizationConfig` with a nested :class:`LLMContextSummaryConfig` instead:: LLMAutoContextSummarizationConfig( max_context_tokens=8000, max_unsummarized_messages=20, summary_config=LLMContextSummaryConfig( target_context_tokens=6000, min_messages_after_summary=4, ), ) Parameters: max_context_tokens: Maximum allowed context size in tokens. Set to ``None`` to disable token-based triggering. target_context_tokens: Maximum token size for the generated summary. max_unsummarized_messages: Maximum new messages before triggering summarization. Set to ``None`` to disable message-count triggering. min_messages_after_summary: Number of recent messages to preserve. summarization_prompt: Custom prompt for summary generation. r7r8rrr9r:r rNrrrrrrctjdtd|j|j t d|j|jdkr t d|j dkr t d|j6|j |jkDrt|jd z|_|j|jd kr t d |jdkr t d y) rzLLMContextSummarizationConfig is deprecated. Use LLMAutoContextSummarizationConfig with a nested LLMContextSummaryConfig instead.) stacklevelNr>rr?rrr@rAz+min_messages_after_summary must be positive) warningswarnDeprecationWarningr8r:rrr.rrs r r!z+LLMContextSummarizationConfig.__post_init__s  c    " " *t/M/M/U^   " " .43J3Ja3OBC C  % % *EF F  # # /**T-D-DD*-T-D-Ds-J)KD &  ) ) 5$:X:X[\:\KL L  * *Q .JK K /r"r#c*|jxstSr%r&rs r r(z,LLMContextSummarizationConfig.summary_promptr)r"c t|j|jt|j|j |j |j|j|jS)zConvert to the new :class:`LLMAutoContextSummarizationConfig`. Returns: An equivalent ``LLMAutoContextSummarizationConfig`` instance. )rrrrrr)r8r:r<) r6r8r:rrrrrrrrs r to_auto_configz,LLMContextSummarizationConfig.to_auto_configs_ 1#66&*&D&D2&*&@&@+/+J+J%)%>%>)-)F)FHH&*&@&@   r")r*r+r,r-r8rr.r/rr:rrr0rrr1rr2r!r3r(r6rLr4r"r rCrCs2)- ,!%3%/1x}1&''*.(3-.$EcE"&C, &#@5@L<III A r"rCc,eZdZUdZeeed<eed<y)LLMMessagesToSummarizezResult of get_messages_to_summarize operation. Parameters: messages: Messages to include in the summary last_summarized_index: Index of the last message being summarized messageslast_summarized_indexN)r*r+r,r-rdictr/r.r4r"r rNrNs4jr"rNc eZdZdZededefdZededefdZ ede e ded edefd Z eded ede fd Zede e defd Zy)LLMContextSummarizationUtilu?Utility providing context summarization capabilities for LLM processing. This utility enables automatic conversation context compression when token limits are reached. It provides functionality for both aggregators (which decide when to summarize) and LLM services (which generate summaries). Key features: - Token estimation using character-count heuristics (chars // 4) - Smart message selection (preserves system messages and recent context) - Function call awareness (avoids summarizing incomplete tool interactions) - Flexible transcript formatting for summarization - Maximum summary token calculation with safety buffers Usage: Use the static methods directly on the class: tokens = LLMContextSummarizationUtil.estimate_context_tokens(context) result = LLMContextSummarizationUtil.get_messages_to_summarize(context, 4) transcript = LLMContextSummarizationUtil.format_messages_for_summary(messages) Note: Token estimation uses the industry-standard heuristic of 1 token ≈ 4 characters. textr#c,|syt|tzS)u,Estimate token count for text using character count heuristic. Uses the industry-standard approximation of 1 token ≈ 4 characters. This works well across different content types (prose, code, etc.) and languages. Note: For more accurate token counts, use the model's official tokenizer. This is a rough estimate suitable for threshold checks and budgeting. Args: text: Text to estimate tokens for Returns: Estimated token count (characters // 4) r)lenCHARS_PER_TOKEN)rTs r estimate_tokensz+LLMContextSummarizationUtil.estimate_tokens*s$4yO++r"contextc d}|jD]l}t|tr|tz }|j dd}t|t r|t j|z }nvt|trf|D]a}t|ts|j dd}|dk(r)|t j|j ddz }T|dvsY|tz }cd|vr|d}t|trv|D]q}t|ts|j di}t|ts7|t j|j d d|j d dzz }sd |vsd|tz }o|S) aEstimate total token count for a context. Calculates an approximate token count by analyzing all messages, including text content, tool calls, and structural overhead. Args: context: LLM context to estimate. Returns: Estimated total token count including: - Message content (text, images) - Tool calls and their arguments - Tool results - Structural overhead (TOKEN_OVERHEAD_PER_MESSAGE per message) rcontenttyperT) image_urlimage tool_callsfunctionname arguments tool_call_id) rO isinstancer TOKEN_OVERHEAD_PER_MESSAGEgetr0rSrXlistrQIMAGE_TOKEN_ESTIMATE) rYtotalmessager[item item_typer` tool_callfuncs r estimate_context_tokensz3LLMContextSummarizationUtil.estimate_context_tokens@s"'') 4G'#56 / /Ekk)R0G'3'4DDWMMGT*# :D!$-$(HHVR$8 $.!%@%P%P $ 4&E'*@@!%99E :w&$\2 j$/%/" %i6#,==R#@D)$5 %)D)T)T$(HHVR$8488KQS;T$T*"!" "(33S) 4V r"rO start_idx summary_endci}t||D]}||}t|tr|jd}|dk(rWd|vrS|jdg}t|tr1|D],}t|t s|jd} | s(||| <.|dk(s|jd} | s| |vs|j | |rt|jSy)aFind the earliest message index with incomplete function calls. Scans messages from ``start_idx`` up to (but not including) ``summary_end`` to identify tool calls whose responses either don't exist yet or fall in the kept portion of the context (>= summary_end). This prevents summarizing tool call requests when their responses would remain in the kept context as orphans, which the OpenAI API rejects. Args: messages: List of messages to check. start_idx: Index to start checking from. summary_end: Exclusive upper bound for the scan (the first kept message index). Only tool responses within this range count as completing a call; responses beyond it are treated as absent, leaving the call "in progress". Returns: Index of first message with function call in progress, or -1 if all function calls are complete within the scanned range. role assistantr`idtoolrd) rangerer rgrhrQpopminvalues) rOrqrrpending_tool_callsimsgrtr`rnrds r 1_get_earliest_function_call_not_resolved_in_rangezMLLMContextSummarizationUtil._get_earliest_function_call_not_resolved_in_ranges2.0y+. 9A1+C#12776?D{"|s': WW\26 j$/%/E %i6+4==+>L+CD 2< @ Ev~"ww~6 L4F$F&**<81 96 )0023 3r"min_messages_to_keepc|j}t||kr tgdStdt |Dd}|dk\r|dz}nd}t||z }||k\r tgdSt j |||}|dk\rF||krAtjd|d|d||z }|}|dkDrtjd |d ||k\r tgdS|||}|dz } t|| S) aDetermine which messages should be included in summarization. Intelligently selects messages for summarization while preserving: - The first system message (defines assistant behavior) - The last N messages (maintains immediate conversation context) - Incomplete function call sequences (preserves tool interaction integrity) Args: context: The LLM context containing all messages. min_messages_to_keep: Number of recent messages to exclude from summarization. Returns: LLMMessagesToSummarize containing the messages to summarize and the index of the last message included. rx)rOrPc3nK|]-\}}t|ts|jddk(r|/yw)rtsystemN)rer rg).0r~rs r zHLLMContextSummarizationUtil.get_messages_to_summarize..s7 As!#'9:swwvRZ?Z s35rr@z?ContextSummarization: Found function call in progress at index z;, stopping summary before it (was going to summarize up to )zContextSummarization: Skipping zV messages with function calls in progress (will summarize after results are available)) rOrVrNnext enumeraterSrr debuginfo) rYrrOfirst_system_index summary_startrrfunction_call_startskipped_messagesmessages_to_summarizerPs r get_messages_to_summarizez5LLMContextSummarizationUtil.get_messages_to_summarizesb(## x=0 0)2RP P " '1      ".2MM(m&:: K ')2RP P ( Y Y-   ! #(;k(I LLQReQfgLLW=XY[  +-@@ -K!# 56F5GH^_ K ')2RP P ({ C +a%*BW  r"c g}|D]}t|tr|jdd}|jdd}t|tr|}n}t|trbg}|D]I}t|t s|jddk(s)|j |jddKdj|}n t|}|r&|j}|j |d|d |vr|jd g} t| trw| D]r} t| t s| jd i} t| t s7| jd d} | jd d} |j d | d| dt|dk(s|jdd}|j d|d|dj|S)zFormat messages as a transcript for summarization. Args: messages: Messages to format Returns: Formatted transcript string rtunknownr[r\r]rT z: r`rarbrcz TOOL_CALL: (rrwrdz TOOL_RESULT[z]: z ) rer rgr0rhrQappendjoinupper)rOtranscript_partsrrtr[rT text_partsrlformatted_roler`rnrorbargsrds r format_messages_for_summaryz7LLMContextSummarizationUtil.format_messages_for_summary s, PC #127769-Dggi,G'3'GT* #@D!$-$((62Bf2L"))$((62*>?@xx +7|!% ''>*:"TF(CDs" WW\26 j$/%/V %i6#,==R#@D)$5'+xx 'B'+xx R'@ 0 7 7+dV1TFRS8T U Vv~"ww~yA  '',|nCv(NOY, P\{{+,,r"N)r*r+r,r- staticmethodr0r.rXr rprrQrrNrrr4r"r rSrSs0,c,c,,*====~8t*8),8;>8 88tN N 36N N N `9-d4j9-S9-9-r"rS)r-rG dataclassesrrtypingrrrpipecat.services.llm_servicerlogurur *pipecat.processors.aggregators.llm_contextr r r1rWrfriSUMMARY_TOKEN_BUFFERMIN_SUMMARY_TOKENSr'rr6rCrNrSr4r"r rs (007U!& R. 5I5I 5Ip /[/[ /[d [ [  [ |     u-u-r"