qilbdZddlZddlZddlmZmZddlmZmZm Z m Z m Z ddl m Z ddlmZddlmZddlmZmZdd lmZmZmZmZ dd lmZmZmZmZmZm Z Gdde Z%Gddee%Z&y#e!$r7Z"e jFd e"e jFd e$d e"dZ"["wwxYw)zGemini LLM adapter for Pipecat.N) dataclassfield)AnyDictListOptional TypedDict)logger)NotGiven)BaseLLMAdapter) AdapterType ToolsSchema) LLMContextLLMContextMessageLLMSpecificMessageLLMStandardMessage)BlobContentFileData FunctionCallFunctionResponsePartz Exception: zHIn order to use Google AI, you need to `pip install pipecat-ai[google]`.zMissing module: cHeZdZUdZeeed<eeed<ee e zed<y)GeminiLLMInvocationParamsz1Context-based parameters for invoking Gemini LLM.system_instructionmessagestoolsN) __name__ __module__ __qualname____doc__rstr__annotations__rrrr Z/opt/pipecat/venv/lib/python3.12/site-packages/pipecat/adapters/services/gemini_adapter.pyrr"s(;  %7m 9x r%rceZdZdZedefdZdedefdZ de de e ee ffdZdede e ee ffdZeGd d ZeGd d ZeGd dZde edefdZdededefdZde ede ede efdZde ede eddfdZdededefdZdededefdZdededefdZ d e!dedefd!Z"y)"GeminiLLMAdaptera2Gemini-specific adapter for Pipecat. Handles: - Extracting parameters for Gemini's API from a universal LLM context - Converting Pipecat's standardized tools schema to Gemini's function-calling format. - Extracting and sanitizing messages from the LLM context for logging with Gemini. returncy)zCGet the identifier used in LLMSpecificMessage instances for Google.googler$)selfs r&id_for_llm_specific_messagesz-GeminiLLMAdapter.id_for_llm_specific_messages3sr%contextc|j|j|}|j|j|j |j dS)zGet Gemini-specific LLM invocation parameters from a universal LLM context. Args: context: The LLM context containing messages, tools, etc. Returns: Dictionary of parameters for Gemini's API. )rrr) _from_universal_context_messages get_messagesrrfrom_standard_toolsr)r,r.rs r&get_llm_invocation_paramsz*GeminiLLMAdapter.get_llm_invocation_params8sM889J9J79ST"*"="= ))--gmm<   r% tools_schemac~dtttfdtttfffd |j}|rDg}|D]7}|j }|dd|dd<|j |9d|ig}ng}g}|j r*|j jtjg}||zS)ajConvert tool schemas to Gemini's function-calling format. Args: tools_schema: The tools schema containing standard and custom tool definitions. Returns: List of tool definitions formatted for Gemini's function-calling API. Includes both converted standard tools and any custom Gemini-specific tools. schemar)c0t|ts|Si}|jD]h\}}|dk(r t|tr |||<(t|tr,|Dcgc]}t|tr|n|c}||<d|||<j|Scc}w)a Recursively remove "additionalProperties" fields from JSON schema, as they're not supported by Gemini. Args: schema: The JSON schema dict to process. Returns: JSON schema dict with "additionalProperties" stripped out. additionalProperties) isinstancedictitemslist)r6resultkeyvalueitem_strip_additional_propertiess r&rAzOGeminiLLMAdapter.to_provider_tools_format.._strip_additional_propertiesTsfd+ F$lln ( U00t,">u"EF3Kt,%*# ?It>T4T:Z^^#F3K #(F3K (M#s"!B parameters propertiesfunction_declarations) rr"rstandard_toolsto_default_dictappend custom_toolsgetr GEMINI) r,r4functions_schemaformatted_functionsfunc func_dictformatted_standard_toolscustom_gemini_toolsrAs @r&to_provider_tools_formatz)GeminiLLMAdapter.to_provider_tools_formatIs c3h DcN :(66 "$ ( 6 002 8Tl+L99 ,' 5$**95  6 *ABU(V'W $') $   $ $".";";"?"? @R@RTV"W '*===r%cT|j|j|j}g}|D]H}|j} d|vr |dD]}d|vrd|dd<d|vsd|d<|j|J|S#t$r"}t j d|Yd}~;d}~wwxYw)aTGet messages from a universal LLM context in a format ready for logging about Gemini. Removes or truncates sensitive data like image content for safe logging. Args: context: The LLM context containing messages. Returns: List of messages in a format ready for logging about Gemini. parts inline_data...datathought_signaturezError: N)r0r1r to_json_dict Exceptionr debugrG)r,r.rmessages_for_loggingmessageobjpartes r&get_messages_for_loggingz)GeminiLLMAdapter.get_messages_for_loggings889J9J79ST]] " -G&&(C ,c> #G >(D0:?D/7.$68=D!45 > ! ' ' , -$# , wqc]++ ,sA< A<< B'B""B'c6eZdZUdZeeed<dZee ed<y)"GeminiLLMAdapter.ConvertedMessageszIContainer for Google-formatted messages converted from universal context.rNr) rrr r!rrr#rrr"r$r%r&ConvertedMessagesrbsWw-,0HSM0r%rcc`eZdZUdZdZeeed<dZee ed<e e Z e e e fed<y)(GeminiLLMAdapter.MessageConversionResulta/Result of converting a single universal context message to Google format. Either content (a Google Content object) or a system instruction string is guaranteed to be set. Also returns a tool call ID to name mapping for any tool calls discovered in the message. Ncontentr)default_factorytool_call_id_to_name_mapping)rrr r!rfrrr#rr"rr:rhrr$r%r&MessageConversionResultres? &*'"),0HSM07zDGeminiLLMAdapter._from_universal_context_messages..s#   Na  E ! fd3 ECIIaL/4@@ ECIIaL*=tDD E# sA8A:userrurolerS)rr)r9rr\r:rIrG_from_standard_messagernrmrfrrhupdate%_apply_thought_signatures_to_messages'_merge_parallel_tool_calls_for_thinkinganyrrrSrc) r,rorrrhthought_signature_dictsr\r=has_regular_messagesms r&r0z1GeminiLLMAdapter._from_universal_context_messagess*"')$"$2$ YG '#56w5++F37JJ+227??C00033489K4L1M41F~~/**%+%>%>"22,33F4W4WXI$ YN 223JHU??@WYab ##  #   &: OOGBT8U7VW X (3!177A33%%xL^%__4s F7F7r\rrc *|d}|jdg}|dk(r`|jrd}nXd}t|tr|}n(t|trdj d|D}|r|j |S|d k(rd }g}i}|jd r^|d D]T}|d } |d d} | || <|jtt| | tj|d dVnb|dk(rd} tj|d} t| tr| } nd| i} |jd}d}|r||jvr|j|}|jtt||| nt|tr|jt|nt|tr|D]}|ddk(r|jt|d+|ddk(r|ddj!drz|dd}|j#ddj#d d!}|jtt%|t'j(|j#d"d#$|ddk(r!|dd}t+j,d%||dd&k(rE|d&}t'j(|d'}|jtt%d(|#$:|dd)k(sD|d)}|jtt/|jd*|jd+,-|j t1||.|/S#t$r} d|di} Yd} ~ _d} ~ wwxYw)0a(Convert standard universal context message to Google Content object. Handles conversion of text, images, and function calls to Google's format. System instructions are returned as a plain string. Args: message: Message in standard universal context format. already_have_system_instruction: Whether we already have a system instruction params: Parameters for conversion. Returns: MessageConversionResult containing either a Content object or a system instruction string. Examples: Standard text message:: { "role": "user", "content": "Hello there" } Converts to Google Content with:: Content( role="user", parts=[Part(text="Hello there")] ) Standard function call message:: { "role": "assistant", "tool_calls": [ { "function": { "name": "search", "arguments": '{"query": "test"}' } } ] } Converts to Google Content with:: Content( role="user", parts=[Part(function_call=FunctionCall(name="search", args={"query": "test"}))] ) rrfsystemr}N c3PK|]}|jddk(s|d yw)rqruNrIrzr^s r&r|z:GeminiLLMAdapter._from_standard_message..Ys*2)-DHHV::gi&89h-$,M%,h$7M#;;~6L.M 0S0S S & C CL Q LL&6'*!.'  % LL7+ ,  & V9&LL1V9!56vY+-!K.2G2R2RSZ2[K./C # #q 1 7 7 se#DEvY-/"#M"2K"("2"2;v3F"GKLL$S^2_!`avY+- !+ILL&.*3-- *D)2z)B'1 B++U3)E,  q >")')*< =  >s?/M77 NN  Nrrc~|s|Std|D}|s|Sdtdtfd}dtdtfd}g}d}|t|kr||}||r||rt |j } g} |dz} | t|kr\|| } || r*|| rnF| j | j | dz } n| j| | dz } | t|kr\|jtd| |j | | }n|j||dz }|t|kr|S) a,Merge parallel tool calls into single Content objects when thinking is enabled. Gemini expects parallel tool calls (multiple function calls made simultaneously) to be in a single Content with multiple function_call Parts. This method takes a list of Content messages, where parallel tool calls may be split across multiple messages, and merges them into single messages. This only has an effect when thought_signatures are present (i.e., when thinking is enabled). When thinking is disabled, merging doesn't matter. When thinking is enabled, there is a guarantee that the first tool call (and only the first) in any batch of parallel tool calls will have a thought_signature. This allows us to distinguish: - Parallel tool calls: share a single thought_signature (on the first call) - Sequential tool calls: each have their own thought_signature Algorithm: A tool call message with a thought_signature starts a new parallel group. Any tool call messages after it without a thought_signature get merged into that group, regardless of what messages appear in between. Args: thought_signature_dicts: A list of thought signature dicts, used to determine if the work of merging is necessary. messages: List of Content messages to process. Returns: List of Content messages with parallel tool calls merged when thought_signatures are present, otherwise unchanged. c3^K|]%}|jdijd'yw)bookmarkrvNr)rztss r&r|zKGeminiLLMAdapter._merge_parallel_tool_calls_for_thinking..s++ <>BFF:r " & & 7+ s+-r{r)cx|jdk(xr*|jxrtd|jDS)z3Check if message contains only function_call parts.rc36K|]}t|ddyw)rvNryrs r&r|ziGeminiLLMAdapter._merge_parallel_tool_calls_for_thinking..is_tool_call_message..sSot<S)rrSallr{s r&is_tool_call_messagezVGeminiLLMAdapter._merge_parallel_tool_calls_for_thinking..is_tool_call_messages:G#TIITSSS r%c:td|jDS)z9Check if any part in the message has a thought_signature.c36K|]}t|ddyw)rWNrrs r&r|zrGeminiLLMAdapter._merge_parallel_tool_calls_for_thinking..message_has_thought_signature..sVDwt%8$?Vr)rrSrs r&message_has_thought_signaturez_GeminiLLMAdapter._merge_parallel_tool_calls_for_thinking..message_has_thought_signaturesVCIIVV Vr%rrtrr)rrrmrxr<rSextendrG) r,rrhas_function_call_signaturesrrmerged_messagesicurrent merged_partsother_messagesjnext_msgs r&rz8GeminiLLMAdapter._merge_parallel_tool_calls_for_thinkingsoDO (++ BY+ ( $,O g $  Ww W4 W #h-qkG$G,1Nw1W#GMM2 !#E#h-''{H+H58B!)//?FA'--h7Q#h-'  &&wG<'PQ&&~6&&w/Q?#h-Br%Ncn|sytjdt||D]}|jd}|jdrtjd|dA|jdr6|d}t|dkDr|dddn|}tjd ||jd stjd |Dcgc]#}t |t r|jd k(r|%}}d } d } |D]} | jd} | jd}| r|s*t| t|D]H} || }|js|jd}|j||s7| |_ | dz } | dz} tjd| dycc}w)aApply thought signatures to corresponding assistant messages. See GoogleLLMService for more details about thought signatures. Args: thought_signature_dicts: A list of dicts containing: - "signature": a thought signature - "bookmark": a bookmark to identify the message part to apply the signature to. The bookmark may contain one of: - "function_call" (a function call ID string) - "text" (a text string) - "inline_data" (a Blob) The list of thought signature dicts is in order. messages: List of messages to apply the thought signatures to. NzThought signatures to apply: rrvz - To function call: ru2rUz - To text: rTz - To inline datarr signaturertzApplied z thought signatures.) r rZrxrItracer9rrrangerS(_thought_signature_bookmark_matches_partrW)r,rrrrrulog_display_textr\assistant_messagesthought_signatures_appliedmessage_start_indexthought_signature_dictrr last_parts r&rz6GeminiLLMAdapter._apply_thought_signatures_to_messagess$'   4S9P5Q4RST) 3Bvvj)H||O, 4Xo5N4OPQf%'8;D Bd3Bi[#4D  |,<+=>?m, 02 3$ '7+ 0G   &'"&=  ".22;?I-11*=HH.4F0GH ,Q/}}$MM"- @@9U2;I/.!3.+,a%'!  4  x :;;OPQI s(F2rr^c|jdx}r|j||S|jdx}r|j||S|jdx}r|j||St j d|y)NrvrurTz)Unknown thought signature bookmark type: F)rI6_thought_signature_function_call_bookmark_matches_part-_thought_signature_text_bookmark_matches_part4_thought_signature_inline_data_bookmark_matches_partr r)r,rr^function_call_bookmark text_bookmarkrTs r&rz9GeminiLLMAdapter._thought_signature_bookmark_matches_partas%-\\/%B B ! BNN& 'll62 2] 2EEmUYZ Z$LL7 7[ 7LL[Z^_ _ NNFxjQ Rr%bookmark_function_call_idct|dr>|jr2|jj|k(rtjd|yy)Nrvz'Thought signature function call match: TF)hasattrrvrr r)r,rr^s r&rzGGeminiLLMAdapter._thought_signature_function_call_bookmark_matches_partosH D/ *""""%%)BB LLBC\B]^ _r% bookmark_textct|dr|jrdj|j}dj|jj}||k(s"|j |s|j |rOt |jdkDr|jdddn |j}t jd|yy)NrurrrUzThought signature text match: TF)rrurrrrxr r)r,rr^ part_textrs r&rz>GeminiLLMAdapter._thought_signature_text_bookmark_matches_part|s 4 TYYHH]%8%8%:;M!23I ]* ++I6'' 6=@^b=Pdiin%5S#9VZV_V_  =>N=OPQr%bookmark_inline_datact|drW|jrKt|jjt|jk(rt j dyy)NrTz#Thought signature inline data matchTF)rrTrxrVr r)r,rr^s r&rzEGeminiLLMAdapter._thought_signature_inline_data_bookmark_matches_partsQ D- (  D$$))*c2F2K2K.LL LL> @r%)#rrr r!propertyr"r-rrr3rrrrrQr`rrcrirnrr0rrr:rrrrrmrrrrrr$r%r&r(r(*sc  @Y "8>[8>T$sTWx.EY8>t$ $tDcN?S$<111  S S S555 W`*./@*AW` W`r^ )^ 6M^ ^ @_'+Dz_=A']_ g_BGR'+DzGR=A']GR GRR  T VZ  ), 48   3VZ_c($(04 r%r()'r!rr dataclassesrrtypingrrrrr logurur openair !pipecat.adapters.base_llm_adapterr %pipecat.adapters.schemas.tools_schemar r*pipecat.processors.aggregators.llm_contextrrrrgoogle.genai.typesrrrrrrModuleNotFoundErrorr_errorrYrr(r$r%r&rs& (77<J,``  t ~&?@t ,FLL;qc"#FLL[\ &qc* ++,sA22B.72B))B.