o پiFX @sdZddlZddlZddlZddlZddlmZmZddlm Z m Z m Z m Z GdddZ Gdd d ZGd d d ZGd d d ZGdddZGdddZGdddZGdddZeeeeeee edZdddddddddZdS)z ComfyUI nodes for SGLang Diffusion integration. Provides nodes for connecting to SGLang Diffusion server and generating images/videos. N)SGLDiffusionGeneratorSGLDiffusionServerAPI)convert_b64_to_tensor_imageconvert_video_to_comfy_videoget_image_pathis_empty_imagec@sveZdZeddZdZdZdZdZ    dd e d e de de de de de de de de de fddZ dS) SGLDOptionscCsigdddifdddifdddddfdd d ddfdd d ddfdd d ddfdd d ddfdddddfdddddfdddifd dd ifd d S)N) auto-detect qwen_imageqwen_image_editfluxlumina2defaultr BOOLEANFINTr)rminstepSTRING model_typeenable_torch_compilenum_gpustp_size sp_degreeulysses_degree ring_degreedp_size dp_degreeenable_cfg_parallelattention_backendrequiredoptionalclsr&r&i/home/ubuntu/.local/lib/python3.10/site-packages/sglang/multimodal_gen/apps/ComfyUI_SGLDiffusion/nodes.py INPUT_TYPESs@ zSGLDOptions.INPUT_TYPES SGLD_OPTIONS) sgld_optionscreate_options SGLDiffusionr Frrrrrrrrrrrr r!r"c Csd|dkrdn|}|dkrdn|}| dkrdn| } ||||||||| | | d } dd| D} | fS)zI Build a dictionary of SGLang Diffusion runtime options. rNrrcSsi|] \}}|dur||qSNr&).0kvr&r&r) msz.SGLDOptions.create_options..)items) selfrrrrrrrrr r!r"optionsr&r&r)r.Hs"zSGLDOptions.create_optionsN) r FrrrrrrrFr) __name__ __module__ __qualname__ classmethodr* RETURN_TYPES RETURN_NAMESFUNCTIONCATEGORYstrboolintr.r&r&r&r)r sR ,    r c@s0eZdZeddZdZdZdZ d d d Zd S) SGLDLoraLoaderc Cs>ddtdfdddddd fd d d ifgd d difdiS)Nr$MODELlorasFLOAT?r g{Gz?rrmaxrrrrall transformer transformer_2criticrM)model lora_namestrength_modelnicknametarget folder_pathsget_filename_listr'r&r&r)r*rs   zSGLDLoraLoader.INPUT_TYPESrD load_lorar/rHrrMc Cstd|}|dus J|}|dkr|n tdtt}|||f|j|<ggggd}|jD]&\}} |d||d| d|d | d |d | d q4|j j j d i||fS)z-Load LoRA adapter using SGLang Diffusion API.rFNrlora) lora_nickname lora_pathstrengthrUr[r\rr]rrUr&) rW get_full_pathcloner@uuiduuid4patchesr5appendrQdiffusion_modelset_lora) r6rQrRrSrTrUr\bi lora_input lora_infor&r&r)rYs"   zSGLDLoraLoader.load_loraN)rHrrM) r8r9r:r;r*r<r>r?rYr&r&r&r)rCqs rCc@s<eZdZddZeddZdZdZdZd d e fd d Z dS) SGLDUNETLoadercCs t|_dSr0)r generator)r6r&r&r)__init__s zSGLDUNETLoader.__init__cCs"tdfgdfdddidS)Ndiffusion_models)r fp8_e4m3fnfp8_e5m2) unet_name weight_dtyper-r+r#rV)sr&r&r)r*s zSGLDUNETLoader.INPUT_TYPESrD load_unetr/Nr-cCsNi}|dkr tj|d<n |dkrtj|d<td|}|jj|||d}|fS)Nrndtyperorm) model_optionsr-)torch float8_e4m3fn float8_e5m2rWr_rk load_model)r6rprqr-ru unet_pathrQr&r&r)rss   zSGLDUNETLoader.load_unetr0) r8r9r:rlr;r*r<r>r?dictrsr&r&r&r)rjs  rjc@s>eZdZdZeddZdZdZdZdZ de d e fd d Z d S) SGLDiffusionServerModelz;Node to load and manage SGLang Diffusion server connection.cCs"dddddfddddfdiS)Nr$rzhttp://localhost:3000/v1F)r multilinezsk-proj-1234567890base_urlapi_keyr&r'r&r&r)r*sz#SGLDiffusionServerModel.INPUT_TYPES) SGLD_CLIENTr) sgld_client model_info load_serverr/rrc Cst||d}z$|}dg}|D]\}}||d|qd|}W||fStyF} zdt| }WYd} ~ ||fSd} ~ ww)z5Initialize OpenAI client for SGLang Diffusion server.r~z=== SGLDiffusion Model Info ===z:  zFailed to get model info: N)rget_model_infor5rdjoin Exceptionr@) r6rrclientr info_lineskeyvaluemodel_info_strer&r&r)rs  z#SGLDiffusionServerModel.load_serverN) r8r9r:__doc__r;r*r<r=r>r?r@rr&r&r&r)r|s r|c@sveZdZdZeddZdZdZdZdZ dZ dde de de de jdedededededefddZd S)SGLDiffusionGenerateImagez/Node to generate images using SGLang Diffusion.c Csdddddfdddddfddd dfd d d d dfd dddddfddddddfd d ddddfd d ddddfdddifddS) Nrrrz Text prompt for image generationrtooltiprpositive_prompt)Negative prompt to avoid certain elementsIMAGEzinput image to use for editingrrrrrKrdrJrG@rH4@皙?@rrF)negative_promptimageseedstepscfgwidthheightenable_teacacher#r&r'r&r&r)r*sr      z%SGLDiffusionGenerateImage.INPUT_TYPES)r)rgenerate_imager/FrNrrrrrrrrrrrrrc  Cs>|std|d| } || dd} |r|| d<|dur!|| d<|dur)|| d<|dur5|d kr5|| d <| r;d | d <|dur`t|rZ|jd |jd}} |d| } | | d<nt|| d<z |jdi| } Wnty}z tdt|d}~ww| dr| dd dstd| dd d}t|}|fS)z*Generate image using SGLang Diffusion API.Prompt cannot be emptyxb64_json)promptsizeresponse_formatrNguidance_scalenum_inference_stepsrrTrr^rr image_pathzFailed to generate image: datazNo image data in responser&) ValueErrorrshaperrr RuntimeErrorr@r)r6rrrrrrrrrrrrequest_paramsresponser image_datar&r&r)rPsD  z(SGLDiffusionGenerateImage.generate_image)rNrrrrrF)r8r9r:rr;r*r<r=r>r? OUTPUT_NODErr@rvTensorrBfloatrArr&r&r&r)rsL P   rc@seZdZdZeddZdZdZdZdZ dZ      d"de de de de jdededededededededefd d!Zd S)#SGLDiffusionGenerateVideoz/Node to generate videos using SGLang Diffusion.cCsdddddfdddddfddd dfd d d d dfd dddddfddddddfd dddddfd dddddfd dddddfd dddddfd d ddddfd!d"d#ifd$ d%S)&Nrrrz Text prompt for video generationrrrrz%input image to use for image-to-videorrrrrrrrrJrGrrHrrrrxi<rrF) rrrrrrr num_framesfpssecondsrr#r&r'r&r&r)r*s         z%SGLDiffusionGenerateVideo.INPUT_TYPES)VIDEOr)video video_pathgenerate_videor/FrNrrrrrrrrrrrrrrrrrrrrrc Cs:|std|d| }t}||| | |d}|r||d<|dur'||d<|dur/||d<|dur;|dkr;||d <| rAd |d <| durI| |d <|durnt|rh|jd |jd}} |d| }||d<nt||d<z|jdi|}|dd}t|| |}W||fSt y}z t dt |d}~ww)z*Generate video using SGLang Diffusion API.rr)rrrr output_pathrNrrrrTrrr^rrinput_reference file_pathrzFailed to generate video: r&) rrWget_temp_directoryrrrrgetrrrr@)r6rrrrrrrrrrrrrr output_dirrrrrrr&r&r)rsJ    z(SGLDiffusionGenerateVideo.generate_video) rNrrrrrrrrF)r8r9r:rr;r*r<r=r>r?rrr@rvrrBrrArr&r&r&r)rs^ k     rc @sReZdZdZeddZdZdZdZdZ dZ dd e d e d e de fddZ dS)SGLDiffusionServerSetLoraz5Node to set LoRA adapter for SGLang Diffusion server.cCs8dddddfdddddfgdd d dfd d S) NrrrzThe name of the LoRA adapterr)rrRz The nickname of the LoRA adapterrLrMz)Which transformer(s) to apply the LoRA to)r[rUr#r&r'r&r&r)r*Fs& z%SGLDiffusionServerSetLora.INPUT_TYPESrrrfr/FrrMrrRr[rUc Csh|dkr tj|d}|||d}z |jdi|}|fWSty3}z tdt|d}~ww)z,Set LoRA adapter using SGLang Diffusion API.rr)r[r\rUzFailed to set LoRA adapter: Nr&)ospathsplitextrfrrr@)r6rrRr[rUrrrr&r&r)rfpsz"SGLDiffusionServerSetLora.set_loraN)rrrM)r8r9r:rr;r*r<r=r>r?rrr@rfr&r&r&r)rCs* #rc@sFeZdZdZeddZdZdZdZdZ dZ dd e d e fd d Z dS)SGLDiffusionServerUnsetLoraz7Node to unset LoRA adapter for SGLang Diffusion server.cCs ddidgddddfidS) NrrrUrLrMz+Which transformer(s) to unset the LoRA fromrr#r&r'r&r&r)r*sz'SGLDiffusionServerUnsetLora.INPUT_TYPESrr unset_lorar/FrMrrUc Cs@z |j|d}|fWSty}z tdt|d}~ww)z.Unset LoRA adapter using SGLang Diffusion API.)rUzFailed to unset LoRA adapter: N)rrrr@)r6rrUrrr&r&r)rs z&SGLDiffusionServerUnsetLora.unset_loraN)rM)r8r9r:rr;r*r<r=r>r?rrr@rr&r&r&r)rs r)r|rrrrrjr rCzSGLDiffusion Server ModelzSGLDiffusion Generate ImagezSGLDiffusion Generate VideozSGLDiffusion Server Set LoRAzSGLDiffusion Server Unset LoRAzSGLDiffusion UNET LoaderzSGLDiffusion OptionszSGLDiffusion LoRA Loader)rrrarWrvcorerrutilsrrrrr rCrjr|rrrrNODE_CLASS_MAPPINGSNODE_DISPLAY_NAME_MAPPINGSr&r&r&r)sF\5$,7G.