o xi<@sdZddlmZddlZddlZddlmZmZddlm Z m Z ddl Z ddl Z ddl mZddlmZe jjdd d d ZedurMejd ksMJd ejeeZdddZGdddejjZdS)u&DSPy ↔ Weights & Biases integration.) annotationsN)MappingSequence)AnyLiteral) telemetry)Rundspyz}To use the W&B DSPy integration you need to have the `dspy` python package installed. Install it with `uv pip install dspy`.F)namerequiredlazyz3.0.0z+DSPy 3.0.0 or higher is required. You have rowslist[dict[str, Any]]returncs$ ddfd d fd d |DS)zFlatten a list of nested row dicts into flat key/value dicts. Args: rows (list[dict[str, Any]]): List of nested dictionaries to flatten. Returns: list[dict[str, Any]]: List of flattened dictionaries. .ddict[str, Any] parent_keystrseprcshg}|D])\}}|r|||n|}t|tr(||||dq|||fqt|S)N)r)items isinstancedictextendappend)rrrrkvnew_key_flattenO/home/ubuntu/.local/lib/python3.10/site-packages/wandb/integration/dspy/dspy.pyr +s z_flatten_rows.._flattencsg|]}|qSr!r!.0rowrr!r" 7sz!_flatten_rows..Nrr)rrrrrrrrr!)r r!rr" _flatten_rows s  r(c@seZdZdZd@dAd d Z dBdCddZdDddZdEddZdFd!d"Z dGdHd'd(Z dId+d,Z dJd.d/Z ddd0d1d2d3dKd>d?Z dS)LWandbDSPyCallbackaW&B callback for tracking DSPy evaluation and optimization. This callback logs evaluation scores, per-step predictions (optional), and a table capturing the DSPy program signature over time. It can also save the best program as a W&B Artifact for reproducibility. Examples: Basic usage within DSPy settings: ```python import dspy import wandb from wandb.integration.dspy import WandbDSPyCallback with wandb.init(project="dspy-optimization") as run: dspy.settings.callbacks.append(WandbDSPyCallback(run=run)) # Run your DSPy optimization/evaluation ``` TN log_resultsboolrun Run | NonerNonecCs~|durtjdurtdtj}||_tj|d }d|j_Wdn1s)wY||_d|_ i|_ d|_ d|_ dS)aJInitialize the callback. Args: log_results (bool): Whether to log per-evaluation prediction tables. run (Run | None): Optional W&B run to use. Defaults to the current global run if available. Raises: wandb.Error: If no active run is provided or found. NzFYou must call `wandb.init()` before instantiating WandbDSPyCallback().)r,TFr) wandbr,Errorr*rcontextfeature dspy_callback_run_did_log_config _program_info_program_table_row_idx)selfr*r,telr!r!r"__init__Os   zWandbDSPyCallback.__init__rrnestedrrrrrcs$id fdd ||S) a{Recursively flatten arbitrarily nested mappings and sequences. Args: nested (Any): Nested structure of mappings/lists to flatten. parent_key (str): Prefix to prepend to keys in the flattened output. sep (str): Key separator for nested fields. Returns: dict[str, Any]: Flattened dictionary representation. objrbaserrr.cst|tr#|D]\}}|r||nt|}||q dSt|trNt|tttfsNt|D]\}}|rB||nt|}||q4dS|rR|nd}|r\||<dSdS)Nr)rrrrrbytes bytearray enumerate)r=r>rrridxkey_walkflatrr!r"rE|s       z.WandbDSPyCallback._flatten_dict.._walkN)r=rr>rrr.r!)r9r<rrr!rDr" _flatten_dictms  zWandbDSPyCallback._flatten_dictfieldsrdict[str, str]cCsdd|DS)aConvert signature fields to a flat mapping of strings. Note: The input is expected to be a dict-like mapping from field names to field metadata. Values are stringified for logging. Args: fields (list[dict[str, Any]]): Mapping of field name to metadata. Returns: dict[str, str]: Mapping of field name to string value. cSsi|] \}}|t|qSr!)rr$rrr!r!r" sz5WandbDSPyCallback._extract_fields..)r)r9rHr!r!r"_extract_fieldss z!WandbDSPyCallback._extract_fields program_objc Csi}|dur|SzGtdd|D}t|ddr|j|d<t|ddr*|j|d<t|ddr:|j}|||d<t|ddrJ|j}|||d<||WSt yh}z t d|WYd}~|Sd}~ww) aoExtract signature-related info from a DSPy program. Attempts to read the program signature, instructions, input and output fields from a DSPy `Predict` parameter if available. Args: program_obj (Any): DSPy program/module instance. Returns: dict[str, Any]: Flattened dictionary of signature metadata. Ncss&|]\}}t|tjr|jVqdSN)rr Predict signature)r$_paramr!r!r" s  z:WandbDSPyCallback._extract_program_info..rP instructions input_fields output_fieldsz9Failed to extract program info from Evaluate instance: %s) nextnamed_parametersgetattrrPrTrUrLrVrG Exceptionloggerwarning)r9rM info_dictsigrUrVer!r!r"_extract_program_infos2        z'WandbDSPyCallback._extract_program_infocall_idinstanceinputscCsr|js(t|dr t|ni}dd|D}d|vr|d=|jj|d|_|d}r7|||_ dSdS)aHandle start of a DSPy evaluation call. Logs non-private fields from the evaluator instance to W&B config and captures program signature info for later logging. Args: call_id (str): Unique identifier for the evaluation call. instance (Any): The evaluation instance (e.g., `dspy.Evaluate`). inputs (dict[str, Any]): Inputs passed to the evaluation (may include a `program` key with the DSPy program). __dict__cSs i|] \}}|ds||qS)rQ) startswithrJr!r!r"rKs z7WandbDSPyCallback.on_evaluate_start..devsetTprogramN) r5hasattrvarsrr4configupdategetr`r6)r9rarbrc instance_vars serializablerMr!r!r"on_evaluate_startsz#WandbDSPyCallback.on_evaluate_startoutputs Any | None exceptionException | Nonec Cs4d}|dur. IMMUTABLE)rxdatary predictions_ruN)r(rrr/rr4r~r8)r9r rxr preds_tabler!r!r"rJs "z(WandbDSPyCallback._log_predictions_tablejson)bestlatestz dspy-program) save_programsave_dirfiletypealiases artifact_namemodel dspy.Modulerr str | NonerLiteral['json', 'pkl']r Sequence[str]rc Cs ||}ttdd|jjd|}tj|d|jjd|d} |dur.t j |jj d}t j |}z t j|d d Wnty\} ztd |d | d WYd} ~ dSd} ~ ww|rl|j|d d| |nd|} t j || } |j| dd| | |jj| t|ddS)a~Save and log the best DSPy program as a W&B Artifact. You can choose to save the full program (architecture + state) or only the state to a single file (JSON or pickle). Args: model (dspy.Module): DSPy module to save. save_program (bool): Save full program directory if True; otherwise save only the state file. Defaults to `True`. save_dir (str): Directory to store program files before logging. Defaults to a subdirectory `dspy_program` within the active run's files directory (i.e., `wandb.run.dir`). filetype (Literal["json", "pkl"]): State file format when `save_program` is False. Defaults to `json`. aliases (Sequence[str]): Aliases for the logged Artifact version. Defaults to `("best", "latest")`. artifact_name (str): Base name for the Artifact. Defaults to `dspy-program`. Examples: Save the complete program and add aliases: ```python callback.log_best_model( optimized_program, save_program=True, aliases=("best", "production") ) ``` Save only the state as JSON: ```python callback.log_best_model( optimized_program, save_program=False, filetype="json" ) ``` __version__unknown) dspy_version module_class-r)r rmetadataN dspy_programT)exist_okz&Could not create or access directory 'z': z. Skipping artifact logging.)rzprogram.F)r)r`rYr __class____name__r/Artifactr4idospathjoindirnormpathmakedirsrZrsaveadd_diradd_file log_artifactr) r9rrrrrrr]rartifactexcfilename file_pathr!r!r"log_best_modelXs> -     z WandbDSPyCallback.log_best_model)TN)r*r+r,r-rr.r')r<rrrrrrr)rHrrrI)rMrrr)rarrbrrcrrr.rN)rarrprqrrrsrr.)rrrr)r rrr.)rrrr+rrrrrrrrrr.) r __module__ __qualname____doc__r;rGrLr`rorrrrr!r!r!r"r):s$  #  *% C r))r rrr)r __future__rloggingrcollections.abcrrtypingrrr/ wandb.util wandb.sdk.librwandb.sdk.wandb_runrutil get_moduler r getLoggerrr[r(utils BaseCallbackr)r!r!r!r"s,