o cip@sddlZddlZddlmZddlmZddlmZddl m Z ddl m Z GdddeZ d d Zd d Zdd dZddZddZdS)N)Param)Kern)minimize)pairwise_distances)euclidean_distancescs<eZdZdZ d fdd ZddZd d Zd d ZZS) TV_SquaredExpzTime varying squared exponential kernel. For more info see the TV-GP-UCB paper: http://proceedings.mlr.press/v51/bogunovic16.pdf ?NcsLt||dtd||_td||_td||_||j|j|jdS)Ntime_sevariance lengthscaleepsilon)super__init__rr r r link_parameters)self input_dimr r r active_dims __class__Q/home/ubuntu/.local/lib/python3.10/site-packages/ray/tune/schedulers/pb2_utils.pyrs    zTV_SquaredExp.__init__cCs|jdkrd|_|durt|}|dddfdd}|dddfdd}t||d}d|jd|}|ddddf}|ddddf}|jttt|| |j }||S)Ng?r cityblock) r npcopyreshaperr expsquarerr )rXX2T1T2dists timekernelRBFrrrKs   zTV_SquaredExp.KcCs|jt|jdS)Nr)r ronesshape)rr rrrKdiag-szTV_SquaredExp.Kdiagc Cs0|dur t|}|dddfdd}|dddfdd}|ddddf}|ddddf}tt|||j}ttt|||j }dt||d|jt|  |jd}t||dd} | d|j | d} t |||j_ t |||j_ t | ||j _ dS)Nrrrr) rrrrrr rr rr sumgradient) rdL_dKr r!r"r#dist2dvardlndepsrrrupdate_gradients_full0s" "z#TV_SquaredExp.update_gradients_full)rrr N) __name__ __module__ __qualname____doc__rr'r*r5 __classcell__rrrrr s rcCs2|tj|ddtj|ddtj|dddS)zgNormalize data to be in range (0,1), with respect to (wrt) boundaries, which can be specified. raxis:0yE>)rminmax)datawrtrrr normalizeFsrBcCs2|tj|ddtj|ddd}t|ddS)z5Standardize to be Gaussian N(0,1). Clip final values.rr;r=r,r+)rmeanstdclip)r@rrr standardizeOs$rFc Csd}d}|tdt||jjd}|durt|n|}t|ddt|ddfj }z| |} | |} | ddd} Wn t yQd} Ynwz| |} | ddd} Wn t yld} Ynw| || S)aPUCB acquisition function. Interesting points to note: 1) We concat with the fixed points, because we are not optimizing wrt these. This is the Reward and Time, which we can't change. We want to find the best hyperparameters *given* the reward and time. 2) We use m to get the mean and m1 to get the variance. If we already have trials running, then m1 contains this information. This reduces the variance at points currently running, even if we don't have their label. Ref: https://jmlr.org/papers/volume15/desautels14a/desautels14a.pdf g?g?rNrri) r?rlogr r)sqrt concatenaterarrayTpredict ValueError) mm1xfixedkappac1c2beta_txtestpredsrCvarrrrUCBUs&  &      rYcsdddd}d}d}jdddf}dd tjjd |D} t|D]1} tjdd jjd |} tfd d | | d |d} | j} | |krW| }| j}q&t|dd S)zOptimize acquisition function.F)maxitermaxfundisp irNcSsg|]}dqS))rrr).0_rrr z optimize_acq..rcs| SNr)rPrQfuncrNrOrrrbzoptimize_acq..zL-BFGS-B)boundsmethodoptions) r ranger)rrandomuniformrrPrE)rerNrOrQnum_foptsrK best_value best_thetargiix0resvalrrdr optimize_acqxs(  rucCs`d}|jd|kr|jdS|d}g}|d|jdkr|d7}tt|j}|| dddf}|| d} |ddd|f} ttj| ddtj| ddf d| jd} tj| |fdd} t || } t |  | j d}t | jdddd d }tj| ||}|jd d |||d|jdkst|}|t|dd}|S) z?Select the number of datapoints to keep, using cross validationrZrr^Nr;r+rrg?)rr r r T)messages)r)rrJlistvaluesrKrIr?r>rrBrFsizerGPymodels GPRegressionoptimizeappendlog_likelihoodargmaxint)Xrawyrawrgrmmin_lenlengthscores base_valsX_leny_len oldpointsold_limslimitsr ykernelrNidxrrr select_lengths:    rrc)rznumpyrGPy.corerGPy.kernrscipy.optimizersklearn.metricsrsklearn.metrics.pairwiserrrBrFrYrurrrrrs     < #