From 397a7d5a41972a9f6d45e2468c9a32059750a2ba Mon Sep 17 00:00:00 2001 From: pollfly <75068813+pollfly@users.noreply.github.com> Date: Wed, 1 Sep 2021 09:39:38 +0300 Subject: [PATCH] Add screenshots (#58) --- docs/fundamentals/hpo.md | 13 ++++++++++++- docs/fundamentals/hyperparameters.md | 2 ++ docs/fundamentals/task.md | 1 + .../fundamentals_hpo_parallel_coordinates.png | Bin 0 -> 98492 bytes docs/img/fundamentals_hpo_scalars.png | Bin 0 -> 167841 bytes ...amentals_hyperparameters_config_objects.png | Bin 0 -> 50236 bytes docs/img/fundamentals_task.png | Bin 0 -> 48790 bytes 7 files changed, 15 insertions(+), 1 deletion(-) create mode 100644 docs/img/fundamentals_hpo_parallel_coordinates.png create mode 100644 docs/img/fundamentals_hpo_scalars.png create mode 100644 docs/img/fundamentals_hyperparameters_config_objects.png create mode 100644 docs/img/fundamentals_task.png diff --git a/docs/fundamentals/hpo.md b/docs/fundamentals/hpo.md index 402403fd..2668f4d1 100644 --- a/docs/fundamentals/hpo.md +++ b/docs/fundamentals/hpo.md @@ -28,11 +28,22 @@ The diagram above demonstrates the typical flow of hyperparameter optimization w 1. Clone the base task. Each clone's parameter is overridden with a value from the optimization task 1. Enqueue each clone for execution by a ClearML Agent 1. The Optimization Task records and monitors the cloned tasks' configuration and execution details, and returns a - summary of the optimization results + summary of the optimization results in tabular and parallel coordinate formats, and in a scalar plot. ![Optimization results summary chart](../img/fundamentals_hpo_summary.png) +
+Parallel coordinate and scalar plots +
+ +![Parallel Coordinates](../img/fundamentals_hpo_parallel_coordinates.png) + +![Scalars](../img/fundamentals_hpo_scalars.png) + +
+
+ ### Supported Optimizers The `HyperParameterOptimizer` class contains **ClearML**’s hyperparameter optimization modules. Its modular design enables diff --git a/docs/fundamentals/hyperparameters.md b/docs/fundamentals/hyperparameters.md index 687aabdf..bba2dc11 100644 --- a/docs/fundamentals/hyperparameters.md +++ b/docs/fundamentals/hyperparameters.md @@ -160,6 +160,8 @@ The "name" argument, is the name of the section that the object will go into. If See [here](https://github.com/allegroai/clearml/blob/master/examples/reporting/model_config.py) for a detailed example. +![Configuration objects](../img/fundamentals_hyperparameters_config_objects.png) + ## Manual Parameter Access ### Manual Parameter Input diff --git a/docs/fundamentals/task.md b/docs/fundamentals/task.md index db795fba..d0f2ca32 100644 --- a/docs/fundamentals/task.md +++ b/docs/fundamentals/task.md @@ -25,6 +25,7 @@ project name & task name combination or by a unique ID. It's possible to copy ([clone](../webapp/webapp_exp_reproducing.md)) a task multiple times and to modify it for re-execution. +![Task](../img/fundamentals_task.png) ## Task sections diff --git a/docs/img/fundamentals_hpo_parallel_coordinates.png b/docs/img/fundamentals_hpo_parallel_coordinates.png new file mode 100644 index 0000000000000000000000000000000000000000..41190c8bed001b968328eaba51cc235be72ca7bf GIT binary patch literal 98492 zcmce-WmH_t7Bxx?g40NF2#tGihXfjTcL?t8o)Dyg0KuIO?(Ux8PH<@4-JOQ!ao#=W z-248#AKw`7?a^cGvHC~tRaL82t(tRIn394dItmdA0s;cMv=m4M0Rc$_0pVrx+n3M( z7=OujeSW=i5tUYZ`}Xa^n&RrSN$4uB<*Mpn?&@LWY=&TA?_g)f>|)|EL?u zszVq7;XQ&h=#!df`r%T5p6brL$mtMO=;uJH*V18Dmf0ZX?2mxjETGDGw%dB~OoDrz zaGm}Ew8`!0Lf?+3L8GNWL}OvpeY&j9X1){}Ha*C|tinY18Br@R<|k<4Eh4fAl@zb^ z1N^tc+rTf>ZjhS@k^TX{V;6|PPCAw5vCGm%xj~!Ev!{PQt7g0=OaBiXFW>!Z{UWCDM*Y`Xjl^nu^RIO(0bc%VjQ;a@Qi2>_{9_=r0AbT`WNaFco_NmUiLbXUpHq1aS9!QhrEGLsBF`_s|eIRqiKDI-YiR?QoymkF?7rZ8vGgcJhG)+vgPfQ5tz1 zBY56QV#gK6x)hbwjccjJ+>O)HNbYY3$5z=V)||F^G~nG8*m_$g7&p4{7T>hjvp%+} z3i5Hhtk14mntt9WX{w@18sJtDf4z;Dn5Xo7LuYp^m7$0NaSP_54lk??i6yjretps)ISbl!F0mwH{4w6 zwT9;*q)Vz({ZXafTQZN~1hhWdTBm25nB30Y5PxEUHU5%Sel;h2zx?{hQi{Lh`Scs- z{NERC4lr_$GuVm%P9at%V?8oyl-oB%Ztd?6CHrMdp!S-fX$Fot0-6)!vO--QF~d1%3R?e zGEu6GhHJIZ=xC>191ihEq=97`;KNuKa~s5u&fKjAqE7S+)qB$FX07yotEG!dVe?Wgry*v?71t$%|js#tLDx3F`Vqe{zodo$%0e#5A}vmZ^8R+X#sI2sio4`2K|gR^MvDS zW$p>k)bkI`P$cgATXQpz{d)^40`gR$QgO%BPRqB6`ER-RwK z`fQ+N7c<}-%&2aoAw)oqqcra^SwgzNO~!=weYu*?z@Odk(U5VgYW&l5h;c*)Xrht4 zr;ii&M97+*OZCs@>C}_-cVsI5xM{Z!yCTr4tK>Eda*nDYcOh&U=IIp3w)s1CT^%VV zRm*_d_gpf`%$(g8GoAmC1DW@Bi-!Ww!ul-@SqdoIgjZmeW0s<$s1{PSW}b1V(V zl%AP$UYbf)E2EYB{+^Wo{_zqtcoav3}=#yR473N;Sv>rUBZ#&@E?9&l5F`x?HG@w1eTbX(wZuH^7E27or zW36oAlV16bMp=}LR*>>3$KHrPUVpRj)w$CU4fPTFH>D-iq!r9|GgMZU>?j=Pt zuhUrKo+0kJv{B1WlPV)HW0MF1liBPG4YsR>771eL_enYYy>tjql+x`2t(`8_W( zU|@&FzLiH#CUXAdXdy*izW6jR%-YJYXW2t!NqvUTFs((|VY7!NOfdt0%DjB8&TeH# zQ;og^JBe<}c!&d>%$)=Wad1~fcdkOps+=|)E zi}QIiuPYO6KVLLiGm*Q=hs^F{)SFd?R9s;Kd6VTtB+QkDI>sATnAL6R9Oqo$Ou1*R zS{I{UrW4nwCVy{|QgO^y13GLl z_$Q}m+`BzS>OC%p>D~Hj351A<##*ByN--9SikaJS;pUNC+M*Cw0N-Qk%`9ofB1vhM zAl{oTd}24b2;Kg9dt1jS`f7hc-dBB%j+`L#~Du-VLNpaqw5n%6Uc_o`S+M~tblrQ&f~lGqw#wn&W2>Z8<`BJW*V z8@EtZc4QCDnvYI4wF{r}VeAR|SSwljEkLP#SP})|?oi7~pW?iTOsyUK-X`ZFIed z@4id9U`lR_ZY46THR;*h;=80soEjZ+Jhl!%771WE|B<)Wj(>I2PkZQPpV}TtbRF8! zup)W)16~EGooD@S8U1%8ufjOJO&oL?Jlc~p^=I!Yi;Q6>jLnd(Q_6tY`{4&-4g-g| z#pxz~Vq{ShF2Rs|Nfd^2SHunHBJWktNM25+>f&VZYlmdJZ5MEJe`3Z#BRPZ1|%VpLv8P?*}68qHWGAKTU|0iHnr#zAw@!i$jUe zT2mowqMnKMOX+2ZaIoC86+bJu^G>qtJyhZfEfgrWWxbtHlATZ&QItleF~%F7`8IEs zkiS*!zhOYkVdzCU*`;s_G*MfBDdwciWkuCfV67d+$)Yn+h2HKwO$j9kv8)g1%D=SL zFY>Re-HO1Dkd3YO*OvK^M5ksczo4t6)2f3f4FY{a9;z=@{s5XXt~f%Qt^9 z^^0TYawNbRdqJ~50-opH{f;4r`}4$P8w#oAv8dpws95H^tq|+f6}AB#&5Tm@-K>Dc zto>`}eZGT}NGzNYN40myMH(zp1}#dWx%rbJ8HfjWG$;=A(yUHzvtuD%AJn93>2j~| zPt-TbJeah}baa0S46hAw8O5LgYerSs7GgbtKov{K0%w6C?$~@MHu7y-Ecrzv8h|g| zRx(z@XmNj&dJ-(fT)I^Cs4G@4okle6Gx>Ntthrogc_fOgoUAC`bB*oanYtNo1#&Y7}PV zH&2>1LS!38)Q+e`ikkhf$aAC~mvD+QBhKTi05SR=_YFEO9o)0^6`Yhz6DvRTt*3b8 z83YeZX+nFX!tR5*ZdK~OS4T$T6WLz9qiwR=(_NN6@lG_UusgohP#=EkSZ!)9-xNmWxa z`sqG&!3nhyvx^eg+tAk1B#Z`7CYEJZ$1k-vj$r2&oNu$336TIkWy1I!*?_>*RpM2O zpJO;5K&3geepd?kvLIiwN#*tfgw>V&c%`tTz-j3?nC2E?7w!6K^7VV(q=PQ3Durk( zt$7zd2Y(Iu-+%+vg-PDi1d7h;NA0vHqZ)1jEqs9TLLE-yjJ7;;_pn-UL&4WoMj-7) zb%eD%*A!VBwE`^jlpsXb-dHjj<2uwND2eGxQT8~(#P2rhC%yYw9Y?<_2?NF%h{q>p zMeZ@X0jF5?>BHJL5k=Thwd3#j2oZmU4p)`2=_R!2esNbT-)|8n_jc=w{tNIRUF|W> zDVJp3aw#9Xw||F+KY|3Sw~y}Sqf^2{)V!diTW%KSO3r2ZA7;J?MYSpHnB!HbWXn5# z6vbrKf}-|@{Q6z?`gX%{ZoAR0j<;=gtz3_h!>!6Cx`hg0Lp&g7wKWDgRl_y*kT2R)fGF?G4E* zlkI+`5{Bu?m{z5*D4S~ft!Ojb8|PxvD~mzSM~M>!bMu^M6*7V;_3MO%TmAkNFQmeE zF59@^P@gzRGL5k!+5E%Z=O!+w1>ymIF1)>9@I1D)fu!BJohF^4dQKwQ{9|D;Dr5>%)-_Z0W5qG7>oxDgJ-#K@j6$LRA!MsF0_UcK^I)TF@HD{NTe}B?Vz%sIdH~6o zfAn+K_vwIb^Mku9&}{$W-{X#8V2NwNj3XhTh)q^=g{}ht#C)!nAQs%`$c_Bo6|ZYd z`&CUqah!0x5}%wSSw<#;w3r~I538z-rQYvUf6S?9@tV{LJLV$=`g$_6#V|9f_)B1Q z@(@v&vZ9J}hOv($AN^e;C!zCO*kb+gO$`d) ze&~#Ne}&18*P2HVA}Qm*s6CZLR>soBae!u@{u8r#Td>soRI0?crh>&b)%>~E=n!yW zIaEr8ZSp2RK%yADM@Z7#+jeBHt1V8+R-DJkwdM=VJo)$rX%F)I?PlzI_F0|O?zjb< zWPW)ITNV0_$E^eF%`ZLWZ0YG#Y&Cg%<~G+^;o=9PZ$O*&!|;i@j}OW7`r`q#3;R^- z0Io!JPJkq@bnijU9-wB&FF8fO57W>^F+BSM*M>7mJDpQIF0~LFhbw(gxwUV6AwrE` z`Xf!8_Q~6bO&q@BD;nB6M0s>zF;RlML3H{1b9DLWRRB)?8LLqS=3#KCjHHmb*dHn+ z7wPj6Wi}^Tf0qzTjXvk=-17~;_M&?v9DTGT@Q%4) zR5p4W0grrBz=>UtvFsnbf$;zr{0?`1!S=6S+A(nLuE382JmMs&n^xj~Px+~^!8_$g zvby+m7JHiCdPg}o?tRdy$b^c_4_k@6#^$mKK=w>V`1^0fqjTr%Bl?UekGE(sd!7oZ zvNf(w-4m4)`qYv<9Tt-^q-L}-%v3U3Bs65s<#I*CkSy!R*=Es(!cR#MeZ|ck(=ae4 zE-JW~5%Yngf=yUsp)KTtyV0&e4#nfx-ZcTe&+-p_D4jlm$CB87d}?DD^`3^Yc~H+a ziTFX`!)hejkgJiHm6Z|48*1xLLJDRK5j~CA1{)q5HAcS39$GXSul4!@%KPA?knfbS zvUjFUKm6w2Cm&8(;OZ`RSCUB-V9be#aq{tto6Kpac3dU&2JHc1USjSk9BWAjGYT8X z54W53TV#XA+Y=0gYM~M#r$nRb?{rCw2oPCPPc))AmLkr~qgX!K=QRom+0ojQs6s3Sg z{1*|{KwwhbJz1NOd$clho2-uKPgfZ6sQ6^?%w(RIo1oeGq130R?p7Xb$?DBn8x=7D z{>m-A(QQpv9`2qpi_t_Pm+P-NKCailhYV=XCeHD^IBrCEU};JVQ)CriKAYD(8m^tF z$37fSMnB%7TtzQ4$cqH{KllWs+%0s(c|v`4RR;{gD%*8Z6k!$AU=~)^@$H<$ zF42DclMY?#2}WX%cx75;O{n_EDUO}PW^^A6eO%En&5WmE(x4V80AT~jfKhNau(`8Mg`k=ZiWbI)*NzmdZB$5l?xpCxB}7u;j*vna_7 zmV&k_G#)5fnM+XLNmC}}m+I5B9IIiHSjp8=3FXiWYAi7l(^5vnFStY#5lQ7{;&;Fe z-iU!Fu^~P($?j5yzDym-l<{N9HBN3pb6r8YVtupuG`fp?lI_Eu34kIC`uJ^>$vm2f zwEOSx$tnXvWa3Zfh#=1}c?Jri+V{J>Mp0l)Dqrj# zlNl<15bWxC{n|>iJC9~48PGOJc`ej!x0J8|@ih?u0Px)59Q^S+1}wZgMt4CWcn+kx z<395f`jUy)%pQB=6;W7#@vYGSsdlxe(7byxvqM>teZ3f;vz9G$!%$MODjNscNGAi>OjPGg_XrE@Vp<3KH^o`T>_RIA5ReVgq> z$dl}NeW=6IOk3LI#DP$LQSd?jUpQA&yYyJOE-}C7+LL>!(Lx+ya96~%vl{WT+&IP~ zejITPvk-FmM1qm}K3?zmt)n&JKQQwz`9XD$)ts6Hs&)Z4Yl2CKwQs6INc=+OS8Yzy zr1!UnA95P>&XzFIDphAsIy?@K0%}6o_3zol>|?|PS%DHCJiWdUM=;{T09@!c`XX7o z&E=~+=DIkR$~dG18hHjrFPVuBX#UQ-o;c+Jx5FQa?6+U{&;{FDx#l!5%U^#rFSJNY zB?5QnG_n!A_$xid$P1c`h8(ANAee;aNJ&S2$lHsQpG}HQI7^~=pyGaLKF(}pTX+WV zSteu1cTX3bIEBFm1pjmi!4pry4TkH`PxZk&1&3r?pY?dC)?LteGfQ@-s9SS*J_S(~ z6>w%UBUI~jY?*iP2H})NQ%X~o6<%rFxMBJ5S_c=z`n_Pc%KmTGVzJd9oPyKZYZ=+* zmWpBK7Hls~bT<4Xi?)LkL-UDp@v?e!-5vLMU|6Go>UGov;tPb<0nc!Ztjj98Zl*{ghkvxH3dw^gql1KGp{AD3xLzqJ&+$uBsfL(aY4QP@$xTqa2l->2VbY z;fGHOMzVd*=FX-vxGQ3K|F(8&IAGwE^|O>=(lNN-u7-2IuUQBL>R$66?iIjYbjnF6 zO_ZQ;5v`#(`e=rF)}>iG*QlfdNvh&1tNxmvZ<3N0ZFbE-uk?&jr~^I!%LW}SGE?Wt zD*#=Lwv+`b)^%HePBTjsT^Jcr*!I5yd-ng(?5tA5)=wWE#Uv~4D8DcBhl`91*Z_@b zsF~uC39yGFz=B#SM@Bz66f@GNEC$)HTkT;h=MibaF*j5WQt{sD0`_!f|Mp@PX>UIP zI`r%~cTzkSfY3E@VT)pd%?=lUocF@WWEl)FSNP26{N%K#0P5C3RhZMt4gr2^pI24x z_>+g6y&Q2k4?9P(A+fC~JDns^e6d22_%}WA7sVvrb~Q&RmoaVh+QdQI=3YdWiFDrQ zD45)@(H}F~jW-N@JCQpLCFpJSXc(On1yTJ862Bfv`RrhMOyYA`y}*idceB!~H2AN# zUm9Glyr=kR33wdupugb316;kY_>N}0FDf}`zZ)QaU6XWR*4Bok`7rOjUBYJD2Tyr1 zgs9nS@0HM!fLeKZwokP3W7a_TJ`=rcO5LuLs2< z_rXNl@-6}z zx{Td^kZ`0fVNK6$Q}Rj3%3n;Em@LHuhgJeBgKgT^k=oNY2umKGoy6O=~WG!$1DYWAMT5xGWrC=1RYR?vNKsKH(GaWoKF1s>~dF0&i!l0r!=t!P<%pyIOy%_ z-;CrJ{BzBsW7VSGX1rMw5<~W{s`H+HYuoOv$nDuqIWs?uBxSu~Zn}e$%)BylO3CXz ze$B_GxXrDbZl+3-heZRp-W`w4ISspeDt_uj)wy#deoaO?!1he4a@wayV|GNk(a|bi z7~ST(Sm@~LA{x%a8kbmfGZMg2o3}!>?8Q5(Pa|0i?*F=j@Es#+rPaq`jQBtUYy7PQ z8Iq*nh3h$XBsgl@wU^bd!x4Xc0&C3f#p%SXl?4u6%S)LY&Fpsc#~;}^F@I=6(q zfTgxGO@|7<^Nh%VVJTLtRydJNJ>0r)H1>c1Ye~h?v8!U@qCIJNC^6L>cy8oNQ(_^P z!96sq`D@KNA+F%=gYZhiYJ}39W{IDO`y@FgxCd^UA4$P8G=# z=9LcbuKFa7OB_lK5l%yTTdu906R#VbNhpka6i?rU%V+yQ;2NBGi{m%9B?0CxV+eaq3|w7*$yxrI5vl7G@BeYEtM0;bn%$NUtR2K zL1%XcEP72&;1nN@;APsg3T$gGJ2m7Fhl4f;%Pd4X3@8KXFcqri8x97rD4FMhqOze7 z{PHVw+|nD9;Ojr>$Z?V&POwJwiW6CV&RIl=h-E?|{ICbf**OioNx|{5eaxIU;${bz zJ$1)Fg-c*?_fTuHB-zqWC#XO*p+LlquL_CNijM#9d=FV-@N0myE;(*~M|!5d zB#0`x<#2>+;}z>O5%3=^z+&yrWX*!)Z^W_rYHe9Ut27KPfqfIxq7L1fTw$B818=wJ z)cO2f`O)&EOtBdl(J;I0XxkHb#doxs$U zY3R^`R)J%fwDFdB_UxYX0;535f>9N}l<;NY{%0*0`yzcWO;=Bkgx1rb4_j61q9!W2 zy)kWeeFh4iXf}gF78ZB#wTq2eR?V=n9YNUhR!Woot%x?SNLMB4$1{!Qdp8Ers;IJP zZ#Tiz#F4~D8G|0`jZYCAJ-Qr1P8jLAd8#+4V?1abwCK#hb2xhk9^*L<`IbZMGaWX& z318eLCPpKPUMLUU#nvjizS$w3Ka1^u6K0dd&<#o)o8;blyD@dW4othx>f`Cf*xQO@ zq`^_vKk7kUQD?A6-Ew?}raGp#*iOnOoog+46Z(_!8=j#~yG4JQYM{8?eN&H*mM*CeFl(L|1xykO=yz}?MTo;>RH-a~le7*P=AyZ$JS9wgf zI26lLaM;_Mia(F(E?mcR-BUfC)u_j>kdP2DtuM5Dd0f;#6z+HkxrcT~bNexQJy~=x zD=y#4ts>oY%-q)8Kp@`jflcz%7dge>Ygr$Ub;zhOFKw9aSM!*W80ej+IYexhtB-{` z3Sh|=^l<}gyiXo;Evz}{88*wXFrJ4ieDegXXQ@5;d;__VwbgGHwT zk)LsouEnz^*N)@Q@i49LNxznmFz}$!^I10ZD99jei%FKRhrTmVstl+IR*}YN2lIa-9@mK4QVN0UIW%_x`NJq-#(muE->&Y88qW1Fce9m<&1BiDU2>;qr} zc9`^~eeB$ic`8?NQyB}8#cDaz#iX;QJP#}R^|T71=Epk8hK}o9M$C|KKG7q=gfHMy z>V@V03iUpR`=w@P%wl5p$(4b9ss5pvH2DomZg|I3UFtzZD-z<;#LTv1Z9Ud_*`QLM zb?SQkABxLZilZNPz2UdHKHD~=%>kzK{3WQ9%Q|g495ue~T1q>gy1CP?Y%M!$uS3pE z)^TU5t?L~k6=Xl$<_y zDK6!uly3`Bc5Nre)E+Bsrdtl|Oe1E5g!kN#LP`I&bf48(#uNWlH^1F35!rX7K7L1& zK8$Io%k57^(ciyukJ0u^b{th99s8Vru^D*fvBLeGR1!n)?*iltc{5x1GIk84L+#Hr zO1mD^L#d3K6=p`luTBhK0=l4>JpU#a8)@j??}p9HFk27?5hf@o;IbSt|gF80!{57Zf1+sBPB z3fvE$zJZHYbtoOOp9p=%9+may)sfnq3WzZjroRuv%aSRQBFmD`U}O@p-zD(fBX@sN z=^6o9dS=prWRBZaiaPxLEHBN=F2iM4+Q_eZ`ws8mb+Y z#~;xN2qFC~>w5>$nDJbyHBKih9kx;W1Ol1hLK#D~X$dtK+cBP`>6$L2^zO2GExj4J zDGj4S>vH-*kx4nzk`FVt(x#6ZXA$0;MJ=f}tyf`Hk7Z^t#Nl=Qfx`Ti5r7y9;;;+W@{W zH?Lin2Ih;$Qe)-I?P-)3fSAYDp3ziiZu{L~=2hovpwG9o*C4bcfzRnqxSk4n?=RP| z&w}&?Mq{iU@OLqjwG+=3F33|E7hqOg069$2b{O{NWd!HpzwPWv}YBL62>)EkjgQ)D< zHOEQGH}AvwI6Hru+Z-f>efkaRuIGc!aVz2^dA(PZk|U~E@zP3d=I6eN_L4}HB7Da& zpqYF>X$R`Q4c9FQ4bUiaU|llp=kBHH2}T^*#($bzq>0#tj`$nRpX|)pl-StDNqzGx zY>os6Qh&8`a~qj5$!&<1Bl@%G>y^83mUb~0hnnUgk}^Z)wKbS1@+hY^f8zfL0}^{I zyFf}?%C{pEySqWT?f)DqZ`X)ghLOKv6&FZ@51SxlBOD95sij^mF3$yvDFugw*(x)D zEu1&(rFsa zKrAi;4DP~*odM&32^G8U{!}^vik3fe_C|5063xWHNi?Np|IP@b%x%S~n!oqYX?u@V-FPywtRM z{>j)FONpJMV^18elSf}3NR|Kyl)SF@aIdq9^<_1MR8{#(-#&Ds4=r3wsK|;h*Jtqi z$tvxm+9W!RKt+9uP} zoD3oq)9#NibZ;&rC;gZ9>N>T3i1jdKHkOiaN+}U?ozgzHIM)d%&dHuF57R*DKlg&> zlg#HQ;Y^E>akOn6nbgZ_e1$+N96adi>gsx2kB6~1Xgv!EIV>E!oejpQQE_lWTk93i z$wg#^ujW*rctt$t$>0Ca;9!^#wAER(!{Ox|3P7Ml)pcHiG?@wzEpHfoy?^<7l%-=- z=@4^fMuhypJ6N9uKWWJBsbHee!DOMg+Klqonqz@Yb|#E3yMgSz>%Xl^jTFLp*B9G_3VR2Xk|Hrc8|DRRN|DS*Ethf1}HTyg|BB0W}6-cYcu4_}?nF`0j25mBdpq06;t7xZQ znal?U;{OPxi~b?65fD~8WRJqiI-pK}CM@W4(w{cu6})jQ;EB0*4wnq~OmL}f!i@pbBR?|O*N z-2+k_4ywLTem8h`G(14x1XMI|kxc#qPIZO&C%9Mzx=(lRHajejlsa9PzNfkm^HK9{ zXj|Vb^8GeFv%ZbeQ+&F2)_{Z>#w_Ieev&A*np>lX0$80yvJ|sGRAp?mI`B$+yIp=w zzG>quIQ|TNhT?S?CoAd++AL%on8k;b*f=}hepMMcs;P|&25oZsU)YU2@t!*)Z_# zL=omq!i71jOvcf%7uxA-c=f>kt$^A@In0e{$m8ik%}wdH*{2`^Y)jkmQ28_K#Nb|D z`|eBcwO6#cQc2UUn!UE$kr|IipcWUUps)%q8! zb2JUzh_u%@@u~%VeS_l5*DxQqaTsh2Wj%Hl^mb%E^g5v`+$%z36c<}rlJ>*efrbJ; z#+34_bgpF!HAP=xzETU9C`ef?_x#d#s@s|VrW&BV5n@T1M#^Q5nYFz~PAW|Q<1Ga} zriUJ+Eo8`CFd?TVnOG+bfEY}-v7XQP8U(^Pm|uLT!Q+y5KTqi29EWNXNzX1fG1Rb6 zpyxKx^3ms-WbG&rG`8}={&1JN%hWOE$?ocO3tY2|VA zXyW1}?}aW^UI;x1GGpK9hy}z8^C{2}vtg&YAe9+>T$j1;Suj0P?<~{`u8}`A)~N`t34c9rVv$83wytpHC;S3os{liX-k>KOUG%g@xb|1eliiov2u7@ zEJ0BBd5QEZX#~H5oZ(3ICYAV3)qeSqjDU`Fim3-xMVPTq+umvLrv`?dZh6hSnCC7F zfBSIu^$oy3^FB*f>JsOs(DLWda{^hUQ-{Ibnp0jwVK{p$p)9T}b}b+KSSKH>9>b#{ z@KZ{cj@zTKIHYz4L#LG~IK(jIubU@d(vl#00tJkN8scHl=BBlN#YZO^wPuI zY9kRJhMjaKuWHSJn@ZxSlhNWe`g}`^Fs~7=9`S-}!|i+)2$1Oa^4ZoTj)mN==PKj% z$rY9E-PQOv#3WAJ{g+J@8?o9qW%lF;7k~86KM_Iw(S8gW_;Gird4PjNB9rPnZWcz9 zizOfjW6@-xZ1 zxMR=qI%;7q7RNtr7un`3tmlq}Nz6HTR`inP6*li57G7zOaqE zRqaxpAGo{kopsPs%-qG*a?OEB{gV5VGl7ejPkJ$<$?YFUd9CH{p7`KJjPL>{l@mDH z4mB#9G)8bJ0$?R7Cx#3yRYpXVe`l>%LAzDYGz5}IoMr>MtbC0&n5))VOa zfdDMIJh!TDtm(wbj;M)uJZ<%hF47w>?mH@^y#<7Rd_>>b{qu>z;kl&STMI255x+)( zIt6x~WQ(%{Sq8*iO_BV3zplTV*hi?|wR{pO{HiDdxqCcb{v%}}At}2QiC-cEcX^1D z9lv2#c6+S}^PK|3eQP#!3Y|`WIx2dk^1V!TNFpt$_YHSXOg^bhc$YvfxT#Y^BQx{( zW}$qjf24^DEh1_PWl3YhYDtq1!`x{{ddd336n_{|B#n#h&+OURbB&xV^RJ@I{ypyg z7R8N=J`1Er9(?;Y;SraiEUNp1d*3d_#-gpu)RBY?Bp%9@kq{8R%k>N#G61D775(ow zv?D|>%3Xbv8a-nT#(oa>TwX@g{k)-N>kv5BA@N-&=W5QuyOBR4UnYHd0^h_$K%iXP zs7QSWJZhpc)p&^C|F%_dE~Iy`N{uc+m)k*}Q^O;eozqGW17(*tKB+7jwXu+a+nm( zdcbYOFnb#XgRpI7+Xc&`dk3t@UucHt!;;Hb<2}d$mhowmJ;Bf7CV}ChbeluHK4p@c z^Yr&5yLibvoS1K1W{LWM0d*saql8Lo{4_9mb_S1n54DPHFpGXsmp$!12h;Gvsg})X zmCQ7@dWWIUy?gw{;TMZYkDhUf$L@-f5_@VZ(|8R^s_Z`qqgjE?&lgxO)<4&IO`N=3 z*H2}cS&$|W;$c{S2SiEf&S6q5DeIZmQZqQhwn9tqFDdHOfCEsb znwPKbNN2KF+#=eI(=#@oASZkO0$?S5{8MCSoCmq~eIU3nFVa2z{hn#K9VRBf)zcee zn#txCsgIz^q9c>yI#uMW3aFKmcY zrG!tF3PNJni9#XP^RaZ#G86{4FAg&)^}Xh?Zu%Db==dy6bXpmnE#Zo|`M47?=e@l& z*&l(~_k$DGNoo{!wZ59(!YXV{KB}F_?e401S?c9y)W8|ndw~{+(%i(xQBKPv1$-oQ zV#tOetpL-P=Ef(dD&D8{`;V1M3HKKWzpywp)7w^o5W-1|yB~`Er9qivO?zsHc7l?M zBBC|k{`MOo!|4=1OJ(!$B024NA?t029$wF<5OGa?me8Sr=Je8`d4PzS=tifyuzYh@ zA3C$>h+t%PpB78f(2{bp8z5#&J8Oz2qHQMJp>a4|@__hr+P8DKGz%`Z1;h30IG z&F`e$1a#Zlw6e#*wSWE{cJNEJ&_9;(hH_h zq1@rl6w>9Zt(8|Ose|@paNf;~E?uZIz5`C=24T7h?IS?fHv*h%9jkH7`20;l)q4r@o1OWQHVpZ+l z8Mxy0Ug7Uiq6)Ts+IF>f3_u&FPv&7g(Dn6!`~s#9)sj_=8nMOFoQs#o!L&cAKs2Nl ztb95@97=6WrP(>$x2g-3zP3T3GC-@@7b9sTkGoGyiYep5=QH-a`a8G-8^&AMNp+WU zE91wHdL_P$L;NJR8{(6D*7#u$d5d3DIqzY@D2r3h!8+wXwm9*)uqkQA@B=v1`K|Wl z_Mu^gjzA>B4&t9WWIZ7`J|S?|FPGDxjlHi|mqT|vrkD~jrr~;nR$RLLI8e{q!pRW> zyP{?}QL9xh|GW2+ZFOHgt3PY^vAVF71DUMWN-HyGwe5fMcPBC3e`~T}v|~E^wvp7s z8nJ)7HD>)mm;S@yXpyyHeW%jaZdK+fcT?c=jw=9AKJds>x1Wz+HHfqdm~^VVmQ4(g zV7hxNKKpQ2GbUO0m4$M)_?cOi*j6cPi?lR${4s{`W`FF@g)aT0@0i(e?(=Ho($)=W zQ2zdWqqfm&I_U(i>jitRy9FRpXf#0V!fdgtguF5;jqFY@vd_Jdyw+RxetA5Lw>?n!1fHam=W5*Q5U1w*nT>C$r8(CPfW*ku6hwmi2Q&48S}}l&@!X7#K_` zuR~nX=&pJ&kLD;ocxvW!ROR{C>J;FiliRDv{+W}%KjBg;BNWs!wVwJe6y;z1Z`_pGfzPt&R>GK?q3G<=L12jOOg5yM&NAY)ajsxO+j974*>0JRDVu=RB z7P{{#8iO|_&w~~4 zoi?Oj_!Mon%4Rpf?6QKyd(Q@2%IY_;`NaK%|448Zb{cl5sRd*iM<91wbkh!4nT3ai zJZuhB&9=t2%xM6HyS4x^;td-g^Pb^IV)vWq*cQq0`w(WmU##Oh8T<(bD&3fGXNEKL z)L|Fkelun+?s4WC(N}?~@$GpQ!y)GCn(aEz5%Zl=T&gDm<2Wm%M`*4rOn~komn1yjDshSI)7Oj(~FAE>yya227(bCPFi5O)|TOVP!1A~x>8;Ypf zS9>JMVwCJwl(;<%tKDBnW4{`)uS(7KwWB0Ts-}M2Kpcea+Aa#t#4s77W1QgQ$VR<> z_oaG95GDVUsOL62*fHnyOE&Vl_OaTXXR5(b6pNi7 zSy7j;0RY2dnO4gFwt;`N0FnuFP&F&gZM<}u3bPa4vQ?)rz0%2}wUA%?lj_{q5{y9@ ziMSM<4-O?VtRa0X?4KWJ8OF|{8*LI(b(3JzR%ro0tc@+8W|;CQH}&zqK~!%G3`Z!} ze<;ZfhP`@cati`oGcL*Mlw-+Kkyu({0nX^oS;AW*h(;&JS*@P>)MPJ*79`8Ic!F}K zKXo76+s$uCl1A}*4;+3lbgom|+1cgjpi5d7IHh9K>-8X+=GRjIW^yuZ8RTs|6#3D@ z5EYduq-vsT=MsnW$U_xy{3!D?C>>N?A~PCk9NI!SRk zC~?S7cHCNKi<548TcppL0pXXWQIJZL8buPrrbj?VLekfFulbOYs8AQY9+)R-dF6Q}1BNz$~+f)`G?{>@%HkNe)ReH3dK zb$yu)Z**H;0NU8 zrUO=Ng!jWa>&S`>2~PU>a?=@-@-}%znaK#^W0rwQ9q!+b+Y88uiON5GD16*f?eFQ7 zrRk#1PAvY`g3)>bPx8VK(_zkgrm;u_Gn=fPuSul^G&ZQ#BXUz4`@vyTp6$)RhPG+Egj?2I8J_fs`<1d2^ZQW}E-MPe)fU4jHTB7VQ6ErYD&FIG6j$q{YU^WS{y+*^H^mAV->~l{MtI#mivD zywg86!S-QbT+Rf^treA7p%n3gu-n{rw=d`B-h1lz7t%oBnO*I=Kgy1h^89WcthX}b z@#%2M%U5k{>FMeJV(vYoqFTCkQQW4j5(ES!Z%~pX$r-i;K@bqhp+R!aG)=}tkSIBW zg2blDISL3!j!kOlMsjX)rn$xI-rxED+;PS^RzC`b$be12%p-1U+-Y4IHQFqpykP739(1D|bq3zxKYgJq}1UIA?F zk|;Dh{cTN6R!<*|W!@u3n=H0bJHl|4hbeS$sDQwjsOq^({=G;-Qf-KGfQ+o|Q=)rV z^#Qf;+mbk^i{JlUAAEEUY>q@&&zZK!QdYRyr!(x_N7 zof)e7lDA^na+A}FKtbixFx!^tlBqSplCY~mp@`_*=+Nf(uT!769$9+0T*LctKQ{JZ zR(yKv^|Mz6iFr zQ`sI`XbFc~Cn$#t)aQ1_7qeqxX(?a`Iy_69P*}g6M1=OAR5zicY~8X53I^Ff`j3;3wxUfC%32ga?