o Ti'@s^ddlZddlZddlmZmZddlmZmZddlm Z ddl m Z eGdddZ dS) N) dataclassfield)AnyOptional)is_bitsandbytes_available) flatten_dictc@seZdZUdZeejej dde d ddidZ e e d<ed dd idZe e d <eddd idZee d <eddddgddZee e d<edddidZee d<eeddidZee efe d<eeddidZee efe d<eeddidZee efe d<edddidZe e d<ed dd!idZe e d"<ed#dd$idZee d%<eddd&idZee d'<ed(dd)idZee d*<ed+d,gd-ddZe e d.<ed/dd0idZe e d1<ed dd2idZ!e e d3<ed4dd5idZ"ee d6<ed7dd8idZ#e$e d9<ed:dd;idZ%e$e d<<eddd=idZ&ee d><ed?dd@idZ'e e dA<edBddCidZ(e$e dD<edEddFidZ)e$e dG<edHddIidZ*e$e dJ<edKddLidZ+e$e dM<edNddOidZ,e$e dP<edddQidZ-ee dR<ed7ddSidZ.e$e dT<edddUidZ/ee e dV<ed/ddWidZ0e e dX<edYddZidZ1ee d[<ed\dd]idZ2e3eefe d^<ed?dd_idZ4e e d`<dadbZ5dcddZ6dS)eAlignPropConfiga Configuration class for the [`AlignPropTrainer`]. Using [`~transformers.HfArgumentParser`] we can turn this class into [argparse](https://docs.python.org/3/library/argparse#module-argparse) arguments that can be specified on the command line. Parameters: exp_name (`str`, *optional*, defaults to `os.path.basename(sys.argv[0])[: -len(".py")]`): Name of this experiment (defaults to the file name without the extension). run_name (`str`, *optional*, defaults to `""`): Name of this run. seed (`int`, *optional*, defaults to `0`): Random seed for reproducibility. log_with (`str` or `None`, *optional*, defaults to `None`): Log with either `"wandb"` or `"tensorboard"`. Check [tracking](https://huggingface.co/docs/accelerate/usage_guides/tracking) for more details. log_image_freq (`int`, *optional*, defaults to `1`): Frequency for logging images. tracker_kwargs (`dict[str, Any]`, *optional*, defaults to `{}`): Keyword arguments for the tracker (e.g., `wandb_project`). accelerator_kwargs (`dict[str, Any]`, *optional*, defaults to `{}`): Keyword arguments for the accelerator. project_kwargs (`dict[str, Any]`, *optional*, defaults to `{}`): Keyword arguments for the accelerator project config (e.g., `logging_dir`). tracker_project_name (`str`, *optional*, defaults to `"trl"`): Name of project to use for tracking. logdir (`str`, *optional*, defaults to `"logs"`): Top-level logging directory for checkpoint saving. num_epochs (`int`, *optional*, defaults to `100`): Number of epochs to train. save_freq (`int`, *optional*, defaults to `1`): Number of epochs between saving model checkpoints. num_checkpoint_limit (`int`, *optional*, defaults to `5`): Number of checkpoints to keep before overwriting old ones. mixed_precision (`str`, *optional*, defaults to `"fp16"`): Mixed precision training. allow_tf32 (`bool`, *optional*, defaults to `True`): Allow `tf32` on Ampere GPUs. resume_from (`str`, *optional*, defaults to `""`): Path to resume training from a checkpoint. sample_num_steps (`int`, *optional*, defaults to `50`): Number of sampler inference steps. sample_eta (`float`, *optional*, defaults to `1.0`): Eta parameter for the DDIM sampler. sample_guidance_scale (`float`, *optional*, defaults to `5.0`): Classifier-free guidance weight. train_batch_size (`int`, *optional*, defaults to `1`): Batch size for training. train_use_8bit_adam (`bool`, *optional*, defaults to `False`): Whether to use the 8bit Adam optimizer from `bitsandbytes`. train_learning_rate (`float`, *optional*, defaults to `1e-3`): Learning rate. train_adam_beta1 (`float`, *optional*, defaults to `0.9`): Beta1 for Adam optimizer. train_adam_beta2 (`float`, *optional*, defaults to `0.999`): Beta2 for Adam optimizer. train_adam_weight_decay (`float`, *optional*, defaults to `1e-4`): Weight decay for Adam optimizer. train_adam_epsilon (`float`, *optional*, defaults to `1e-8`): Epsilon value for Adam optimizer. train_gradient_accumulation_steps (`int`, *optional*, defaults to `1`): Number of gradient accumulation steps. train_max_grad_norm (`float`, *optional*, defaults to `1.0`): Maximum gradient norm for gradient clipping. negative_prompts (`str` or `None`, *optional*, defaults to `None`): Comma-separated list of prompts to use as negative examples. truncated_backprop_rand (`bool`, *optional*, defaults to `True`): If `True`, randomized truncation to different diffusion timesteps is used. truncated_backprop_timestep (`int`, *optional*, defaults to `49`): Absolute timestep to which the gradients are backpropagated. Used only if `truncated_backprop_rand=False`. truncated_rand_backprop_minmax (`tuple[int, int]`, *optional*, defaults to `(0, 50)`): Range of diffusion timesteps for randomized truncated backpropagation. push_to_hub (`bool`, *optional*, defaults to `False`): Whether to push the final model to the Hub. rNz.pyhelpzJName of this experiment (defaults to the file name without the extension).)defaultmetadataexp_namezName of this run.run_namez Random seed for reproducibility.seedz)Log with either 'wandb' or 'tensorboard'.wandb tensorboard)r choiceslog_withzFrequency for logging images.log_image_freqz:Keyword arguments for the tracker (e.g., `wandb_project`).)default_factoryr tracker_kwargsz&Keyword arguments for the accelerator.accelerator_kwargszKKeyword arguments for the accelerator project config (e.g., `logging_dir`).project_kwargstrlz$Name of project to use for tracking.tracker_project_namelogsz2Top-level logging directory for checkpoint saving.logdirdzNumber of epochs to train. num_epochsz2Number of epochs between saving model checkpoints. save_freqz:Number of checkpoints to keep before overwriting old ones.num_checkpoint_limitfp16zEMixed precision training. Possible values are 'fp16', 'bf16', 'none'.)r$bf16nonemixed_precisionTzAllow `tf32` on Ampere GPUs. allow_tf32z*Path to resume training from a checkpoint. resume_from2z"Number of sampler inference steps.sample_num_stepsg?z#Eta parameter for the DDIM sampler. sample_etag@z Classifier-free guidance weight.sample_guidance_scalezBatch size for training.train_batch_sizeFz;Whether to use the 8bit Adam optimizer from `bitsandbytes`.train_use_8bit_adamgMbP?zLearning rate.train_learning_rateg?zBeta1 for Adam optimizer.train_adam_beta1g+?zBeta2 for Adam optimizer.train_adam_beta2g-C6?z Weight decay for Adam optimizer.train_adam_weight_decayg:0yE>z!Epsilon value for Adam optimizer.train_adam_epsilonz&Number of gradient accumulation steps.!train_gradient_accumulation_stepsz,Maximum gradient norm for gradient clipping.train_max_grad_normzs