qi=PdZddlZddlmZddlmZmZddlmZddl m Z ddl m Z m Z mZmZmZmZddlmZmZdd lmZ ddlZdd lmZdZeGddeZGddeZ y#e$r7Zej8d eej8d ed edZ[wwxYw)zMoondream vision service implementation. This module provides integration with the Moondream vision-language model for image analysis and description generation. N) dataclass)AsyncGeneratorOptional)logger)Image) ErrorFrameFrameUserImageRawFrameVisionFullResponseEndFrameVisionFullResponseStartFrameVisionTextFrame)VisionSettings_warn_deprecated_param) VisionService)AutoModelForCausalLMz Exception: zKIn order to use Moondream, you need to `pip install pipecat-ai[moondream]`.zMissing module(s): c ddl}tjjr%tjdtj fS tjjr%tjdtjfStjjjr%tjdtjfStjdtj fS#t $rYwxYw)aMDetect the appropriate device to run on. Detects available hardware acceleration and selects the best device and data type for optimal performance. Returns: tuple: A tuple containing (device, dtype) where device is a torch.device and dtype is the recommended torch data type for that device. rNxpucudampscpu) intel_extension_for_pytorchtorchr is_availabledevicefloat32 ImportErrorrfloat16backendsr)rs S/opt/pipecat/venv/lib/python3.12/site-packages/pipecat/services/moondream/vision.py detect_devicer (s * 99 ! ! #<<& 5 5 $ zz ||F#U]]22    ( ( *||E"EMM11||E"EMM11    sAC?? D  D ceZdZdZy)MoondreamSettingszkSettings for the Moondream vision service. Parameters: model: Moondream model identifier. N)__name__ __module__ __qualname____doc__rr"r"Asr(r"cneZdZUdZeZeed<ddddddeedeeffd Z d e d e e dffd Z xZS) MoondreamServicezMoondream vision-language model service. Provides image analysis and description generation using the Moondream vision-language model. Supports various hardware acceleration options including CUDA, MPS, and Intel XPU. _settingsNz 2025-01-09F)modelrevisionuse_cpusettingsr,r/c td}|tdtd||_||j|t |d d|i||st \}}n%tjd}tj}tjdtj|jjd|d |i| j|_tjd y) aeInitialize the Moondream service. Args: model: Hugging Face model identifier for the Moondream model. .. deprecated:: 0.0.105 Use ``settings=MoondreamSettings(model=...)`` instead. revision: Specific model revision to use. use_cpu: Whether to force CPU usage instead of hardware acceleration. settings: Runtime-updatable settings. When provided alongside deprecated parameters, ``settings`` values take precedence. **kwargs: Additional arguments passed to the parent VisionService. zvikhyatk/moondream2)r,Nr,r/rzLoading Moondream model...T)trust_remote_coder- device_mapdtypezLoaded Moondream modelr')r"rr, apply_updatesuper__init__r rrrrdebugrfrom_pretrainedr+eval_model) selfr,r-r.r/kwargsdefault_settingsrr4 __class__s rr7zMoondreamService.__init__Us0-3HI   "7,=w G%*  "    ) )( 3 ="2=f=)OMFE\\%(FMME 12*:: NN "F|  $&   -.r(framereturncKjstdytjdt j ddt dttdtffd }tj|j jd{}tt| ty7.w) zsAnalyze an image and generate a description. Args: frame: The image frame to process. zMoondream model not availableNzAnalyzing image (bytes length: ) image_bytestextrActjjj|}jj |}jj ||d}|S)Nanswer)r frombytesformatsizer; encode_imagequery)rDrEimage image_embeds descriptionr@r<s rget_image_descriptionz:MoondreamService.run_vision..get_image_descriptionsROOELL%**kJE;;33E:L++++L$?IK r()rE)r;rrr8lenrMbytesrstrasyncio to_threadrEr r r )r<r@rPrOs`` r run_visionzMoondreamService.run_visions {{<= =  6s5;;7G6HJK u HSM c  $--.CU[[RWR\R\]] *,,;//(** ^sBC C /C )r#r$r%r&r"Settings__annotations__rrSr7r rr rV __classcell__)r?s@rr*r*Jse!H  $04 5/}5/ ,- 5/n+&7+N5RV;rfs !+M9/122  X+}X+Q/FLL;qc"#FLL^_ )!- ../s A))B%.2B  B%