o ߗi3 @sddlZddlmZddlmZmZmZddlZddlmZe e Z ddZ ddZ Gd d d eZd d Zd eeejeejdffdeeejeejdfffddZeGdddZdS)N) dataclass)ListTupleUnion)fxcs&gfdd}tj||}|fS)z\ Flatten the args into a list form and detach the tensors from computational graph. cs8t|tjr||j}||S||SN) isinstancetorchTensordetachrequires_grad_ requires_gradappend)avalflat_detached_argsa/home/ubuntu/transcripts/venv/lib/python3.10/site-packages/torch/distributed/pipelining/_utils.pyextract_tensor_argss   z0flatten_args_detach..extract_tensor_argsrnode map_aggregate)argsrnew_argsrrrflatten_args_detachs  rcs"gfdd}tj||S)z, Flatten the args into a list form. cs||Sr)r)r flat_argsrrr,s z)flatten_args..extract_tensor_argsr)rrrrr flatten_args&s rc@seZdZdZdS)PipeliningShapeErrorz5Shape mismatch between configured and runtime values.N)__name__ __module__ __qualname____doc__rrrrr9srcCs|j|jkst|d|jd|j|j|jks(t|d|jd|j||ks@t|d|d|dS)Nz has a shape mismatch: expected z actual z has a dtype mismatch: expected z! has a stride mismatch: expected )shaperdtypestride)descexpectedgivenrrrvalidate_tensor_metadata=s  r*expected_tensors.actual_tensorscCsft|t|krt|dt|dt|dtt|D]}t|d|||||qdS)Nz: Number of values (z") does not match expected number ()z: value )lenrranger*)r'r+r,irrrvalidate_tensors_metadataLsr1c@s,eZdZUdZejed<eed<eed<dS)PipeInfoz> Captures information for a pipeline (`Pipe` object). graph num_stageshas_loss_and_backwardN) r r!r"r#rGraph__annotations__intboolrrrrr2[s   r2)logging dataclassesrtypingrrrr r getLoggerr loggerrr RuntimeErrorrr*r r1r2rrrrs"