qi`ddlmZddlmZddlmZddlZddlmZddl m Z m Z m Z m Z mZmZmZddlmZmZdd lmZdd lmZmZdd lmZmZd d lmZmZmZm Z m!Z!m"Z"ddl#m$Z$m%Z%ddl&m'Z'm(Z(ddl)m*Z*m+Z+ddl,m-Z-ddl.m/Z/ddl0m1Z1ddl2m3Z3ddl4m5Z5ddl6m7Z7ddgZ8GddeZ9GddeZ:GddZ;GddZ<Gdd Z=Gd!d"Z>y)#) annotations)Optional)LiteralN)_legacy_response)BodyOmitQueryHeadersNotGivenomit not_given)maybe_transformasync_maybe_transform)cached_property)SyncAPIResourceAsyncAPIResource)to_streamed_response_wrapper"async_to_streamed_response_wrapper) OutputItemsAsyncOutputItemsOutputItemsWithRawResponseAsyncOutputItemsWithRawResponse OutputItemsWithStreamingResponse%AsyncOutputItemsWithStreamingResponse)SyncCursorPageAsyncCursorPage)run_list_paramsrun_create_params)AsyncPaginatormake_request_options)Metadata)RunListResponse)RunCancelResponse)RunCreateResponse)RunDeleteResponse)RunRetrieveResponseRuns AsyncRunsc LeZdZdZeddZeddZeddZeeddde d ddZ ddde d dd Z eeeeddde d  dd Z ddde d dd Z ddde d dd Zy)r),Manage and run evals in the OpenAI platform.c,t|jSr,)r_clientselfs R/opt/pipecat/venv/lib/python3.12/site-packages/openai/resources/evals/runs/runs.py output_itemszRuns.output_items(s4<<((ct|Sa This property can be used as a prefix for any HTTP method call to return the raw response object instead of the parsed content. For more information, see https://www.github.com/openai/openai-python#accessing-raw-response-data-eg-headers )RunsWithRawResponser0s r2with_raw_responsezRuns.with_raw_response-s#4((r4ct|Sz An alternative to `.with_raw_response` that doesn't eagerly read the response body. For more information, see https://www.github.com/openai/openai-python#with_streaming_response )RunsWithStreamingResponser0s r2with_streaming_responsezRuns.with_streaming_response7s)..r4Nmetadataname extra_headers extra_query extra_bodytimeoutc |std||jd|dt|||dtjt ||||t S)  Kicks off a new run for a given evaluation, specifying the data source, and what model configuration to use to test. The datasource will be validated against the schema specified in the config of the evaluation. Args: data_source: Details about the run's data source. metadata: Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters. name: The name of the run. extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds 6Expected a non-empty value for `eval_id` but received /evals//runs data_sourcer>r?r@rArBrCbodyoptionscast_to) ValueError_postrr RunCreateParamsr"r& r1eval_idrJr>r?r@rArBrCs r2createz Runs.create@szNUV]U`ab bzzgYe $ #. (  "11 )+Q[el&  r4rKc |std||std||jd|d|t||||tS)P Get an evaluation run by ID. Args: extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds rF5Expected a non-empty value for `run_id` but received rG/runs/rKrNrOrP_getr"r(r1run_idrTr@rArBrCs r2retrievez Runs.retrieveysr0UV]U`ab bTU[T^_` `yygYfVH -(+Q[el(   r4afterlimitorderstatusr@rArBrCc|std||jd|dttt |||| t ||||dt jtSa Get a list of runs for an evaluation. Args: after: Identifier for the last run from the previous pagination request. limit: Number of runs to retrieve. order: Sort order for runs by timestamp. Use `asc` for ascending order or `desc` for descending order. Defaults to `asc`. status: Filter runs by status. One of `queued` | `in_progress` | `failed` | `completed` | `canceled`. extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds rFrGrH)rarbrcrd)r@rArBrCquery)pagerNmodel)rP _get_api_listrr$r"rr RunListParams r1rTrarbrcrdr@rArBrCs r2listz Runs.listsJUV]U`ab b!!gYe $0(+'%%!&!&!&"(  $11 "%"  r4c |std||std||jd|d|t||||tS)G Delete an eval run. Args: extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds rFrXrGrYrKrZrP_deleter"r'r]s r2deletez Runs.deletesr0UV]U`ab bTU[T^_` `||gYfVH -(+Q[el&   r4c |std||std||jd|d|t||||tS)U Cancel an ongoing evaluation run. Args: extra_headers: Send extra headers extra_query: Add additional query parameters to the request extra_body: Add additional JSON properties to the request timeout: Override the client-level default timeout for this request, in seconds rFrXrGrYrKrZrPrQr"r%r]s r2cancelz Runs.cancelsr0UV]U`ab bTU[T^_` `zzgYfVH -(+Q[el&   r4)returnr)rwr7)rwr;rTstrrJzrun_create_params.DataSourcer>zOptional[Metadata] | Omitr? str | Omitr@Headers | NonerA Query | NonerB Body | NonerC'float | httpx.Timeout | None | NotGivenrwr&r^ryrTryr@r{rAr|rBr}rCr~rwr()rTryrarzrb int | OmitrcLiteral['asc', 'desc'] | OmitrdJLiteral['queued', 'in_progress', 'completed', 'canceled', 'failed'] | Omitr@r{rAr|rBr}rCr~rwzSyncCursorPage[RunListResponse]r^ryrTryr@r{rAr|rBr}rCr~rwr'r^ryrTryr@r{rAr|rBr}rCr~rwr%__name__ __module__ __qualname____doc__rr3r8r<r rrUr_rmrrrvr4r2r)r)%s?6))))///3)-$("&;D7 7 2 7 , 7  7 &7 "7  7 97  7 @)-$("&;D" "  " &" ""  " 9"  " P! /3]a)-$("&;D: :  :  : - : [: &: ":  : 9:  ): F)-$("&;D" "  " &" ""  " 9"  " V)-$("&;D" "  " &" ""  " 9"  " r4c LeZdZdZeddZeddZeddZeeddde d ddZ ddde d dd Z eeeeddde d  dd Z ddde d dd Z ddde d dd Zy)r*r,c,t|jSr.)rr/r0s r2r3zAsyncRuns.output_items%s  --r4ct|Sr6)AsyncRunsWithRawResponser0s r2r8zAsyncRuns.with_raw_response*s(--r4ct|Sr:)AsyncRunsWithStreamingResponser0s r2r<z!AsyncRuns.with_streaming_response4s.d33r4Nr=c K|std||jd|dt|||dtjd{t ||||t d{S7$7w)rErFrGrHrINrKrL)rPrQrr rRr"r&rSs r2rUzAsyncRuns.create=sNUV]U`ab bZZgYe $,#. (  "11 )+Q[el&     s$AA-A) A-$A+%A-+A-rKc K|std||std||jd|d|t||||td{S7w)rWrFrXrGrYrKrZNr[r]s r2r_zAsyncRuns.retrievevs0UV]U`ab bTU[T^_` `YYgYfVH -(+Q[el(     AAAAr`c|std||jd|dttt |||| t ||||dt jtSrf)rPrjrr$r"rrrkrls r2rmzAsyncRuns.listsJUV]U`ab b!!gYe $ 1(+'%%!&!&!&"(  $11 "%"  r4c K|std||std||jd|d|t||||td{S7w)rorFrXrGrYrKrZNrpr]s r2rrzAsyncRuns.deletes0UV]U`ab bTU[T^_` `\\gYfVH -(+Q[el& "    rc K|std||std||jd|d|t||||td{S7w)rtrFrXrGrYrKrZNrur]s r2rvzAsyncRuns.cancels0UV]U`ab bTU[T^_` `ZZgYfVH -(+Q[el&      r)rwr)rwr)rwrrxr)rTryrarzrbrrcrrdrr@r{rAr|rBr}rCr~rwzAAsyncPaginator[RunListResponse, AsyncCursorPage[RunListResponse]]rrrrr4r2r*r*"s@6....44/3)-$("&;D7 7 2 7 , 7  7 &7 "7  7 97  7 @)-$("&;D" "  " &" ""  " 9"  " P! /3]a)-$("&;D: :  :  : - : [: &: ":  : 9:  K: F)-$("&;D" "  " &" ""  " 9"  " V)-$("&;D" "  " &" ""  " 9"  " r4c&eZdZddZeddZy)r7cz||_tj|j|_tj|j|_tj|j |_tj|j |_tj|j|_yN)_runsrto_raw_response_wrapperrUr_rmrrrvr1runss r2__init__zRunsWithRawResponse.__init__ s &>> KK  )@@ MM  %<< II  '>> KK  '>> KK  r4c@t|jjSr.)rrr3r0s r2r3z RunsWithRawResponse.output_items3s*$***A*ABBr4Nrr)rwNone)rwrrrrrrr3rr4r2r7r7s &CCr4r7c&eZdZddZeddZy)rcz||_tj|j|_tj|j|_tj|j |_tj|j |_tj|j|_yr)rrasync_to_raw_response_wrapperrUr_rmrrrvrs r2rz!AsyncRunsWithRawResponse.__init__:s &DD KK  )FF MM  %BB II  'DD KK  'DD KK  r4c@t|jjSr.)rrr3r0s r2r3z%AsyncRunsWithRawResponse.output_itemsMs/tzz/F/FGGr4Nrr*rwr)rwrrrr4r2rr9s &HHr4rc&eZdZddZeddZy)r;c||_t|j|_t|j|_t|j|_t|j |_t|j |_yr)rrrUr_rmrrrvrs r2rz"RunsWithStreamingResponse.__init__Tsp 2 KK  5 MM  1 II  3 KK  3 KK  r4c@t|jjSr.)rrr3r0s r2r3z&RunsWithStreamingResponse.output_itemsgs0 0G0GHHr4Nr)rwrrrr4r2r;r;Ss &IIr4r;c&eZdZddZeddZy)rc||_t|j|_t|j|_t|j|_t|j |_t|j |_yr)rrrUr_rmrrrvrs r2rz'AsyncRunsWithStreamingResponse.__init__nsp 8 KK  ; MM  7 II  9 KK  9 KK  r4c@t|jjSr.)rrr3r0s r2r3z+AsyncRunsWithStreamingResponse.output_itemss5TZZ5L5LMMr4Nr)rwrrrr4r2rrms &NNr4r)? __future__rtypingrtyping_extensionsrhttpxr_typesrr r r r r r_utilsrr_compatr _resourcerr _responserrr3rrrrrr paginationrr types.evalsrr _base_clientr!r"types.shared_params.metadatar#types.evals.run_list_responser$types.evals.run_cancel_responser%types.evals.run_create_responser&types.evals.run_delete_responser'!types.evals.run_retrieve_responser(__all__r)r*r7rr;rrr4r2rs#% !LLL=';Z;>A5=AAAE ; z ?z zz  z zCC4HH4II4NNr4