qiG7dZddlZddlZddlmZddlmZddlmZddl m Z m Z m Z m Z mZddlmZddlmZd d lmZmZmZmZmZmZd d lmZd d lmZmZe rd d lm Z ejBe"Z#ejHdZ%dZ&dddZ'eGddZ(e d8de e(de e)de)de)de e)de e)de e*e)e)fde e+e)de,e+e*e+e*e e)ffdZ-GddeZ.Gd d!eZ/ d9d"d#d$e*de e)de e*e)e)fde e)ddf d%Z0d&e*fd'Z1d(e*fd)Z2d*e*fd+Z3d"d#d,e)ddfd-Z4d"d#d.e*d/e5d,e)ddf d0Z6d.e*d1e(d/e5de+e)fd2Z7d3e+e*d4e)de/fd5Z8d"d#d6e+e)d/e5de+e*fd7Z9y):z.Git LFS related type definitions and utilitiesN) dataclass)ceil)getsize) TYPE_CHECKINGBinaryIOIterableOptional TypedDict)unquote) constants)build_hf_headersfix_hf_endpoint_in_urlhf_raise_for_status http_backoffloggingvalidate_hf_hub_args) SliceFileObj)sha256 sha_fileobj)CommitOperationAddz^[0-9a-f]{40}$zlfs-multipart-uploadzapplication/vnd.git-lfs+json)Acceptz Content-TypecreZdZUdZeed<eed<eed<edefdZ edefdZ ed e fd Z y ) UploadInfoad Dataclass holding required information to determine whether a blob should be uploaded to the hub using the LFS protocol or the regular protocol Args: sha256 (`bytes`): SHA256 hash of the blob size (`int`): Size in bytes of the blob sample (`bytes`): First 512 bytes of the blob rsizesamplepathct|}tj|d5}|jddd}t |}ddd||S#1swYxYw)Nrbrrr)rioopenpeekr)clsrrfilershas E/opt/pipecat/venv/lib/python3.12/site-packages/huggingface_hub/lfs.py from_pathzUploadInfo.from_pathIs_t} WWT4  $DYYs^DS)Fd#C $S88 $ $s AAdatacbt|j}|t||dd|S)Nr )rrr)rdigestlen)r%r*r's r( from_byteszUploadInfo.from_bytesQs-Tl!!#D $t*SAAfileobjc|jd}|jdtjt |}|j }|jdtj||||S)Nr rr!)readseekr"SEEK_SETrtell)r%r0rr'rs r( from_fileobjzUploadInfo.from_fileobjVsXc" Q $'"||~ Q $S88r/N) __name__ __module__ __qualname____doc__bytes__annotations__int classmethodstrr)r.rr6r/r(rr6sj  M I M9S99BeBB9899r/r upload_infostoken repo_typerepo_idrevisionendpointheaders transfersreturnc||ntj}d}|tjvrtj|}|d||d} d||nddg|D cgc])} | jj | j d+c} d d } |d t |i| d <itt| |xsi}td| || } t| | j} | jdd}t|ts td| jd}t|t r|nd}|Dcgc]}d|vst#|c}|Dcgc]}d|vst%|c}|fScc} wcc}wcc}w)a Requests the LFS batch endpoint to retrieve upload instructions Learn more: https://github.com/git-lfs/git-lfs/blob/main/docs/api/batch.md Args: upload_infos (`Iterable` of `UploadInfo`): `UploadInfo` for the files that are being uploaded, typically obtained from `CommitOperationAdd.upload_info` repo_type (`str`): Type of the repo to upload to: `"model"`, `"dataset"` or `"space"`. repo_id (`str`): A namespace (user or an organization) and a repo name separated by a `/`. revision (`str`, *optional*): The git revision to upload to. headers (`dict`, *optional*): Additional headers to include in the request transfers (`list`, *optional*): List of transfer methods to use. Defaults to ["basic", "multipart"]. Returns: `LfsBatchInfo`: 3-tuple: - First element is the list of upload instructions from the server - Second element is a list of errors, if any - Third element is the chosen transfer adapter if provided by the server (e.g. "basic", "multipart", "xet") Raises: [`ValueError`](https://docs.python.org/3/library/exceptions.html#ValueError) If an argument is invalid or the server response is malformed. [`HfHubHTTPError`] If the server returned an error. N/z.git/info/lfs/objects/batchuploadbasic multipartoidrr) operationrHobjects hash_algonameref)rBPOSTrGjsonrSzMalformed response from servertransfererror)r ENDPOINTREPO_TYPES_URL_PREFIXESrhexrr LFS_HEADERSrrrrYget isinstancelist ValueErrorr?_validate_batch_actions_validate_batch_error)rArBrCrDrErFrGrH url_prefix batch_urlrMpayloadresp batch_inforSchosen_transferobjs r(post_lfs_batch_inform`sX$/xY5G5GHJI55566yA *Aj\'2MNI"+"7Yg{=S '  }}((*    G '("34   ' =bG  7 IDJnnY-G gt $9:: nnZ0O)3OS)IotO29O#G3(>'??mno%Y/9M'$Y/33H=M ]+  x ,FL)J' f(=QJ ZJ Yv*akliJG ]++M&,A8L "  $5'B"..5599;YEZEZE_E_`  K( LLI**++>?@'I& fgqfrrtu  s - E22F lfs_actionct|jdtr0|jd*t|jdts t d|S).validates response from the LFS batch endpointr{ryz"lfs_action is improperly formatted)rar`r?dictrc)rs r(rr sK :>>&)3/ ^^H % -JNN8> r/lfs_batch_actionscPt|jdtrt|jdts t d|jdijd}|jdijd}| t || t ||S)rrQrz)lfs_batch_actions is improperly formattedrwrMrx)rar`r?r=rcr)rrrs r(rdrds (,,U3S 9jIZI^I^_eIfhk>lDEE%)))R8<t|jdtrt|jdts t d|S)rrQrz'lfs_batch_error is improperly formattedr[messagecode)rar`r?r=rcr)r error_infos r(rere"s **513 7JGZGZ[aGbdgz(_get_sorted_parts_urls..ds !A$r/)keyz0Invalid server response to upload large LFS file)sorteditemsisdigitr-r=rrrc)ryrrzpart_numr|_sorted_part_upload_urls num_partss r(rr[s$-3LLN (Hj##%#h-!*;Xr"J/      Az   +,ID))J677KLL ""  s 6B  B%rrQcg}t|D]H\}}|jd}||dk(rtd|d|dz|j|dz|dJ||dS)NrqrKzInvalid etag (`z`) returned for part r )rprq)rQrt) enumerater`rcappend)rrQrt part_numberryrqs r(rrms "E()9:   Vzz&! <42:tf4I+XY/IZ[\ \ )Ao    ''r/rc6g}|jd5}t|D]S\}}t|||z|5}td||}t ||j |j dddU ddd|S#1swYjxYw#1swY|SxYw)NTr) seek_from read_limitrr)rrrrrrrG) rRrrzrGr0part_idxpart_upload_url fileobj_slicepart_upload_ress r(rr|sG   T  * 8g)23D)E 8 %Ho$x/% 8".uoM"Z#O4667 8 8 8 8 N 8 8 8 Ns"#B5B- BB BB)NNNN)NNN):r:r"re dataclassesrmathros.pathrtypingrrrr r urllib.parser huggingface_hubr utilsrrrrrr utils._lfsr utils.sharr _commit_apir get_loggerr7r}compile OID_REGEXLFS_MULTIPART_UPLOAD_COMMANDr_rr?rrbtuplermrorsrrrdrerr=rrrrr@r/r(rs5 !II %%*/   H % BJJ( ) 5-2  &9&9 &9R #"(,%)S:&S C=SS S sm S sm Sd38n %SS "S 4:tDz8C= 01SSl9  (," DA#DADA C=DAd38n % DA sm DA  DANT t  4 &S&T&(("6((RU(cf(ko(.#4#j#c#VZ[^V_#$ (d4j (s (GY (#8