o S۷i- @sNddlZddlmZddlZddlmZddlmZddlmZm Z m Z m Z m Z ddl mZddlmZmZddlmZdd lmZerTdd lmZmZmZdd lmZzddlZdd lmZmZmZWn eyoed wd%ddZd&ddZ d'ddZ!          d(ddZ"ddd e#ddddddf d!d"Z$Gd#d$d$eZ%dS))N) TYPE_CHECKING)consts)record_token_usage)get_start_span_functionset_data_normalizedtruncate_and_annotate_messagestransform_openai_content_part&truncate_and_annotate_embedding_inputs)SPANDATA) DidNotEnable Integration)should_send_default_pii)event_from_exception)AnyDictList)datetime)input_callbacksuccess_callbackfailure_callbackzLiteLLM not installedkwargsDict[str, Any]returncCs.|di}|d}|duri}||d<|S)z,Get the metadata dictionary from the kwargs.litellm_paramsmetadataN) setdefaultget)rrrrU/home/ubuntu/vllm_env/lib/python3.10/site-packages/sentry_sdk/integrations/litellm.py_get_metadata_dicts  rmessagesList[Dict[str, Any]]cCst|}|D]9}t|tsq|d}t|ttfr@g}|D]}t|tr6t|}||dur2|n|q||q||d<q|S)z Convert the message parts from OpenAI format to the `gen_ai.request.messages` format using the OpenAI-specific transformer (LiteLLM uses OpenAI's message format). Deep copies messages to avoid mutating original kwargs. contentN) copydeepcopy isinstancedictrlisttuplerappend)r messager" transformeditemresultrrr_convert_message_parts*s     r.cCs$tt}|dur dS|dd}z t|\}}}}Wn ty+|}d}Ynw|dd}|dkr9d}nd}t|dkrEt j j nt j j |d |tj d }||t|d <t|tj|t|tj|tr|jr|dkr|d } | rt} t| tr| n| g} t| || } | durt|tj| d dn#|dg} | rt} t| } t| || } | durt|tj| d dtjtjtj tj!tj"tj#tj$d}|%D]\}}||}|durt|||q|d|d|dd}|%D]\}}|durt|d||qdS)zHandle the start of a request.Nmodelunknown call_type embedding embeddingschat )opnameorigin _sentry_spaninputF)unpackr )r/stream max_tokenspresence_penaltyfrequency_penalty temperaturetop_papi_base api_versioncustom_llm_provider)rCrDrEzgen_ai.litellm.)& sentry_sdk get_clientget_integrationLiteLLMIntegrationrlitellmget_llm_provider ExceptionrrOP GEN_AI_CHATGEN_AI_EMBEDDINGSr9 __enter__rrr GEN_AI_SYSTEMGEN_AI_OPERATION_NAMEr include_promptsget_current_scoper%r'r GEN_AI_EMBEDDINGS_INPUTr.rGEN_AI_REQUEST_MESSAGESGEN_AI_REQUEST_MODELGEN_AI_RESPONSE_STREAMINGGEN_AI_REQUEST_MAX_TOKENSGEN_AI_REQUEST_PRESENCE_PENALTY GEN_AI_REQUEST_FREQUENCY_PENALTYGEN_AI_REQUEST_TEMPERATUREGEN_AI_REQUEST_TOP_Pitems)r integration full_modelr/provider_r2 operationspanembedding_inputscope input_list messages_datar paramskey attributevaluerrrr_input_callbackEs             rmcompletion_responser start_timerend_timec Cst|d}|dur dStt}|durdSzt|dr(t|tj |j t r|j rt|drg}|j D]P}t|drt|jdrN||jq8t|jdr]||jq8i}t|jdrk|jj|d<t|jd rw|jj|d <t|jd r|jj|d <||q8|rt|tj|t|d r|j} t|t| d dt| d dt| dddW|ddddSW|ddddS|dddw)zHandle successful completion.r:Nr/choicesr* model_dumpr&roler" tool_callsusage prompt_tokenscompletion_tokens total_tokens) input_tokens output_tokensrx)rrrFrGrHrIhasattrrr GEN_AI_RESPONSE_MODELr/r rSrqr*r)rrr&rsr"rtGEN_AI_RESPONSE_TEXTrurgetattr__exit__) rrnrorprdr_response_messageschoicemsgrurrr_success_callbacksX                    " r exceptionc Cstt|d}|dur dSz"t|tjdddd\}}tj||dW|t||ddS|t||dw)zHandle request failure.r:NrJF)typehandled)client_options mechanism)hint) rrrrFrGoptions capture_eventrr)rrrorprdeventrrrr_failure_callbacks *rc@sDeZdZdZdZdeZd dddeddfd d Zedd d Z dS)rIa" LiteLLM integration for Sentry. This integration automatically captures LiteLLM API calls and sends them to Sentry for monitoring and error tracking. It supports all 100+ LLM providers that LiteLLM supports, including OpenAI, Anthropic, Google, Cohere, and many others. Features: - Automatic exception capture for all LiteLLM calls - Token usage tracking across all providers - Provider detection and attribution - Input/output message capture (configurable) - Streaming response support - Cost tracking integration Usage: ```python import litellm import sentry_sdk # Initialize Sentry with the LiteLLM integration sentry_sdk.init( dsn="your-dsn", send_default_pii=True integrations=[ sentry_sdk.integrations.LiteLLMIntegration( include_prompts=True # Set to False to exclude message content ) ] ) # All LiteLLM calls will now be monitored response = litellm.completion( model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hello!"}] ) ``` Configuration: - include_prompts (bool): Whether to include prompts and responses in spans. Defaults to True. Set to False to exclude potentially sensitive data. rJzauto.ai.TselfrSrNcCs ||_dS)N)rS)rrSrrr__init__5s zLiteLLMIntegration.__init__cCshtpgt_ttjvrtjttpgt_ttjvr tjttp#gt_ttjvr2tjtdSdS)z(Set up LiteLLM callbacks for monitoring.N)rrJrmr)rrrrrrrr setup_once8s        zLiteLLMIntegration.setup_once)T)rN) __name__ __module__ __qualname____doc__ identifierr9boolr staticmethodrrrrrrIs, rI)rrrr)r r!rr!)rrrN) rrrnrrorrprrN)&r#typingrrFrsentry_sdk.ai.monitoringrsentry_sdk.ai.utilsrrrrr sentry_sdk.constsr sentry_sdk.integrationsr r sentry_sdk.scoper sentry_sdk.utilsrrrrrrJrrr ImportErrorrr.rmrrLrrIrrrrsZ          h @