From 2c7349ce5efa895699dadc33c9fe18a44752d1ef Mon Sep 17 00:00:00 2001 From: Kristian Aune Date: Fri, 13 Dec 2024 11:13:05 +0100 Subject: [PATCH] Make the Cloud deployment work --- retrieval-augmented-generation/README.md | 57 ++++++++++++------ .../ext/edit-app-access.png | Bin 0 -> 15358 bytes 2 files changed, 40 insertions(+), 17 deletions(-) create mode 100644 retrieval-augmented-generation/ext/edit-app-access.png diff --git a/retrieval-augmented-generation/README.md b/retrieval-augmented-generation/README.md index 45cf0c876..89f6c31d5 100644 --- a/retrieval-augmented-generation/README.md +++ b/retrieval-augmented-generation/README.md @@ -127,11 +127,12 @@ as a secret. The `services.xml` file must refer to the newly added secret in the secret store. Replace `` and `` below with your own values: -
-    <secrets>
-      <openai-api-key vault="<my-vault-name>" name="<my-secret-name>"/>
-    </secrets>
-
+ +```xml + + + +``` Configure the vespa client. Replace `tenant-name` below with your tenant name. We use the application name `rag-app` here, but you are free to choose your own @@ -141,19 +142,38 @@ $ vespa config set target cloud $ vespa config set application tenant-name.rag-app -Authorize Vespa Cloud access and add your public certificates to the application: +Log in and add your public certificates to the application for Dataplane access:
 $ vespa auth login
 $ vespa auth cert
 
-Deploy the application. This can take some time for all nodes to be provisioned: +Assign application access to the secret. +Applications must be created first, so one can use the Vespa Cloud Console to grant access. +The easiest way to do this is to do a deployment, which will auto-create the application. +The first deployment will fail: +
 $ vespa deploy --wait 900
 
-Now the application should be deployed! You can continue to the -[querying](#querying) section below for testing this application. +``` +[09:47:43] warning Deployment failed: Invalid application: Vault 'my_vault' does not exist, +or application does not have access to it +``` + +At this point, open the console +(the link is like https://console.vespa-cloud.com/tenant/mytenant/account/secrets) +and assign access: + +![edit application access dialog](/ext/edit-app-access.png) + +Deploy the application again. This can take some time for all nodes to be provisioned: +
+$ vespa deploy --wait 900
+
+ +Now the application should be deployed! ## Querying @@ -184,14 +204,17 @@ $ vespa query \ traceLevel=1 -On Vespa cloud, just skip the `--header` parameter, as the API key is already -set up in the services.xml file, and will be retrieved from the Vespa secret -store. - -Here, we specifically set the search chain to `openai`. This calls the -`RAGSearcher` which is set up to use the `OpenAI` client. Note that this -requires an OpenAI API key, which is sent in the header. We also add a timeout -as token generation can take some time. +On Vespa cloud, just skip the `--header` parameter, +as the API key is already set up in [services.xml](services.xml), +and will be retrieved from the Vespa secret store. + +Here, we specifically set the search chain to `openai`. +This calls the +[RAGSearcher](https://github.com/vespa-engine/vespa/blob/master/container-search/src/main/java/ai/vespa/search/llm/RAGSearcher.java) +which is set up to use the +[OpenAI](https://github.com/vespa-engine/vespa/blob/master/model-integration/src/main/java/ai/vespa/llm/clients/OpenAI.java) client. +Note that this requires an OpenAI API key. +We also add a timeout as token generation can take some time. #### Local diff --git a/retrieval-augmented-generation/ext/edit-app-access.png b/retrieval-augmented-generation/ext/edit-app-access.png new file mode 100644 index 0000000000000000000000000000000000000000..bc41193b81102769be690d6b9e39e4b41d650609 GIT binary patch literal 15358 zcmeIZc{r4R_&oyCfm|zK3kXP?)jrNhOp-$i8obEMrCpNu{w5W;B-U z#y-s0oyVuor*zJB{yXP7zwh-su4`r<_w&Br%YEPPdwIQ|=b5gy8XXNQ4H+33o%-F| z`ebCsG=b|W>f^w7IMNM5Mn?X?MMXtdT}6dc*VDt%#m#|?jO!KnrH&?r9%EaaBele_ zvw9b~B?TyGFM`qE_%wvhXl}z<)eMxK*hSCvz%q5t^yI5Xb7s@OOg-OI3{}=WL&Hf(n=cx7b9(sMJxYh*ln;JclKdZ6Zyl7uOWRnbS+M)Q-fb;dZ|M(Efut*SU&DJYcZ zsmpgxEy>51?4H_RI%k^wVxRQMBK?|}kejj^)%671cv)o{6(-f6jD}nnR34k&O5o|N z+E2?Waq;>%d2Wo&yC(30lugctj}^Zq`6t&g&$zG1-Pfh9!+7 zD4i9Chyl3T)SF6!SN+O#7K!%ww8&$jfY^;4Ow}E=w8(gXYics`C>JtH;EEjhumT@I zL-9|@P5{3Qz~^=*#ea8?X=Wb#@0x7tu%WVnin==RYhdr`;Nb4%?BOlALpKd{HQ-`s z>TRlZPsZK@EMWV*{6g<2mC@df(_`7d_Rql`~qwC`9;AVE)1q_G> z7(+osL|X2j_WzeVe?~mQY5Ip#N=)=9=aD=AXHFw82Tv6bFfgRI!XM52%Y5|azl?H% zhbuoKiC<#=hYHA9fksa7x6Tx3BJe=&5uBYK*uku6JM*^HkA?Z~= z9zowG<03H*&>62lep#=bpivAyXi1uM&%BoRjxrfJ@Rw!KNgK{pV<6!JtCLzExa><___m1W)aWKSB!2WxQ)@YF-O%pu4Ep#*^5wqS!-} z;N|V$(>Cpu!(cE7N47Qe?cqjOxK?V@r)`MKayxjWv2`hE9K6%PG-DR??Tx2a2A{`! z)Lv3tkoqLCajVUD7~vFW1@X+VjNbDn%B4=VEOg=ms5KD}d?S7<_MsG2@K$*<)UYET z+8WyUNK&QGD#Y9dk1cljF~=Y;Vr$DiRH12!YK!q(a1#Ot2OsPqqrppSM$p}};9IY| zY$`wB*q0iGBJS(H)2KciIWBoM2MyET3RO)XE-6_YsTz!R8U7ji=+qzkQW|@Bl8Z~8 zw8|X3QAzPSymt~oYh}u_U3SmcRAXIW{UP(__DR8xfin6K3pZ=E;iTK{yYFi*Wqhb`1y*sal;u9A|_tbWF>&=3Z^fv>YjCc~@G*jQOX6=_CA3v~W zvc4fTr7t8_emC|Qa^uGjR`O~ufoqC)cX#`uCJM0m;Niu!;D&|h<&cVUOFC3BU23U) z)g9=RdPA8dMkE-6oo*lHur)o3s+emSr8WB-y%p2psfgU?Rx88=bM9<(N~8YH@8$gd%; zHdgE6w!>`lHEB>PbE&yKD7GN`VZKw-fZGo@9lC=bv0gP^L9%ed#Cq3A(9S|cdBM;r z$>nW+i!#*)EVNgTtxVj=FD&{+^`rS(O`yO1l$yhDZ_FJ=GX{*LajJIdFeNEXIC^Uw^_n{~H6X!N#d{C#8kd9d~AasSz zy4*LcL~)X9RijPCrzglgYlLkMx?B&X{jg&Won_>l@}_e&rf2l6WeIYkCr{O0c4r7? zEVJPI_{KEVZ=dt=^2tv$_o8IETnaQ4$G%-&#O)`Ccrqu_2&>hAQki{N3=MCQ4@?-$c#vQd$M*2 zTrcpGiMZsyL>I8yS7}@nDD7m_yxes{fb1JGeYpIUfm2<#C}zM7YgbQNsxlI|g!0m2 zW;&JH51YHyd?R2MWvg)i5>BR|>e+@xEk^6Wzh~0AO5=J{*XwD4ha(e#QVTymRE|3C z5mqc(gEISMf=%DI6gwDu??`khK-yKi3L&0ZBIZ48;fjrGjf(knuEua~*!FtJXt2}V z5~Aj+g~Ulzd{S8YUax05>wa=vAOAdh)!IB52Qn(vYW3gWs@Sn%x6`S2fl7-S%?X$y zu>CGicP+#V@q-0pM0|nuDtWy@)#s(R9#u06TP^02g)^m~&6kk5R-Ivx(-61`9k7Hz z(77Y+u{Ec_qJn>-3&YKBNJyTFZfou+$ls?9ln^ejNpF2^o+QN_>TSznZFr7hWbWhZ z);q~5FI;9Kq8;T!yxT>~F`uMqTbxiE$Uw|{VoTcKu;P=zpUb<7(AGQ!zR47>g_!Uc z{-LtvT|HSVd6~J$|H4 zn*=Ow;E=WMj*=M|LGR{r)?CqpO`QPSPZv071ms{*bW+HwjfS1_fdYHior$TLO}~jf zQRkt%JCEjQ0@en2RmP3QwR{dpS{EmYDqWIPRo;1fOYv21N~F%<%6adJ?>h&! z5mry@7e3kNz%*JiL_avN@pbh&_JPp4y*;f=e#!3u}HWTS9e#BAlSn9V}=8Z*GL72|hk3 zOuWv86k?kL`NV=pc^DJKv}@$pQty;!QBU*FdZ7I+eXu99GIO^AAkK?|@ge4o+o5f9 zw`ApJXMb8cl)5G|Tlc@<`%si3Ftp<%+cK!ZXIdi>*jrZ~^&)lBwZmN+pR23ed#YHT z=2qy4%~H!<--ksLAOrLI6b3Xq+dVxY!#dr*s+IhR;#KK}_E(LF98f?@bV>u2vU)4p zTOYOCAE>9bnaw1=Wnj>dS;&$aTkk*BH*@>C;&BhebPA1VGly=)htIGZT+UaDu+86K z>iH=*pCoz3E;@n~q=&_)48MN+MjS5r$dbUxHT9zFJ)gg=3C54=nulwcV6Lp z8YzbZzS?Ab6F1l9=^#Z_g~9GpaX7*xWDo3h*H)uhkaVY={Tn z^jUN+Ks>a}vx~Isn4Cv>qd8R!?veS)R5NJU zSECMlmYu#=WF_9G=z7j0-FKoE;poW2vLg$l&M@v$b*M8>;QanLj(66Ts=dIfonij% z<*HlKgUPpy4){zrJ=a8Z3)uaSa~tc@B<87LD_xG&Py8^5P(}oooYzoAi7+{$PZ0>q zjytF2#g^tQ%GYZA{6FKEr%XIu)&k^>NU7OgE&vqJJZlxug)PJzUbsnvG6W?JRIgTtcjzs>kn==L_w@;RjCT zb+P}WF6Wi9hp=n>ji2Lx1E1?D~;B9GzCePFxuOorQPMdT%W#vCECB) zU@H*IcVOAK2zwp2lJEWsMIGT(wkFae;)?iK{4q0qoRPYzn9L9>PAg1&wTt! z%5?~RnR`tRQ;;wSv+m7dUES&((JzNcb3htu$2lTlckh(rrLSKd{ERTU^LBdQzl~2 z$L1jtx4z80hzw~QYmzK>oVu5~0MhGM-8u{7T_J~c5LCE9o(0+YZu91rVGeJHqkOCg z@RfLuV|!s-nqd^<1pOzQd2UACDV5xpV zC(zT2vWuclSs%B;@HlO&^ijFl>jv=I_lE6s%ZOM$!`JzXx_VXh8%qRVm(lb_<9mw^ zH}~s43dmPFZ8{QIf7e}{I8k! z1nKS0T(%_&$Wwg2*{EtR_ARf@(uB%4)Dne4c{Y~X>PPHDEB}y5ftNdo!Fi(08ioTR z+Tgr#aewD=^}P))p90tTg5~XJ3nP{j=mrE5Wn5JUlrh0RQ0yD?VFtBF&ZJ)5C3=`C-x@HuH4bvJqdc4`fOt}9TK}R=0n*_E0y804CF&Ej;Y5`rS<~<-LpiAGmbd5Y}$jh55ee@}J8Qgv1>Tas1Lx<@E zRvbL`HKu5E8f7ZmcRzjtd<4TI)_QzPN^lOH76x5D54O;+wAp7x2km zZUiEmaS6ylc4M{^YMrrUj5Orlx&Px@fX4!y*p+(R9L19y%pO@xI9RWf_uYsbz%1UR zBBx}yxiTx;y4c}TwMQY!n*fK#;D;f{w5$EmB}lLCzS`gpV_qX36sXQiFXJi}%d`}j z(`1CxWg(&-kVM;TLLVdwI^l+Es$F&PuKs&{08+VQAQ&ZK7=mX@5vwlQ=2%WN`WoKk zT83wChtg^J%&A0_j=o~s^<;E=5-k7qNt_RUs15^Quoc!o#oMtBStzo-szRU?EH*B7 zF8>iwrJ6pN|J~}oV35q`^JV6)d74S@Yh&hzW1Q9rGdideLQbbv2x9)0rq6`Y4FIwW z@!yiaxII(9yT22*3jnVtnYs@M83(@AHu;^RbRa```l1C!H7LLwv%Y?;^J+?k3(g$s zoqC?Q3LnLUEPrUuU=EM*pWUE$!(i!%ljY^7k4FM1EXo#oD~V`aZ_cF1H-gZadF&Cg z$t_lHy-Vz^+Jk65q%*#pQz~9!G7OWM@^!^`roNhyL*;n(grYA07!zhSUjVoqMJ&ww)6^k|q?+;9)7GRXvJ)S%=4%lz9A1k9*rcCTiX$Od)SzBzPd;ke^^L0Lk zO{d8&Zp1fPyu@O-Jmhoubz;lSoW&=7J8eJM0gtXUPQ6Q*=G~2XJ|!^wfrydX8L=Ym zZeXKTcnLGF9AkEOMDn~~v?FYyBaPl6qsNdwtF43XSi1?Qt|~l^?(LUs);wuVlOYd5 zS=qrt-SuzXj1h?g|NkRpJseR5}wB8-uN zYrSHs=Go4^ht!9k{50X&Yn{Ay|$jqP9lseVg(+9V5rE@jvN*>_1!TW!= z`u=XXzMjKazACSy|a8nmmA-Lp$K-72mqjI`7y%H(nHBdZn%3hUZ4d?`z zROP13N?H~o=f1z*-84OzoklqsI1TYPNwp0soakdi^AsdnNLm>7TGj-Jo8;s|)Ofo> zb|W{SHG0Q%=FhQ~E!O#AXo=XuN5lXh`@4B>XakW3xMF2_=0YGd3!_|5gWHnMA^*lAX5=RGBpIr z8->dv4ZjfZIvx1E9gJG~>NL0Mn3!LH8N1kc>P9QF>%8d=@yl~h(8$vZgo zXo-Humqb8)N7SOlJ7@)w5!u+cB&&0hMpP%lRQo<%WNv)%We_Rlf-AJ<;!=(6>Vm3v zp4$)ZDHd-;%28`HOW}l#dwk!V8suAX_mQ0O^2y4j75OFk^eMB66a)&2@=yfijztob zy4QV@Y3~3UoC7&M*)eCKI+@06*uQ@5L!kuXj-#V<;HFfAu~wz~M?TLP_y>)O*wb1U zr^Q{qugZ87v)E2TixE2bCy+f<;eAK(0z|KP0gIUTr)DGp$(4nIyuZESPsPh+ zPw)5~%s>uuZdCCUK$jO%8&Niu*wG)WVTJa?mMEasF|XpfJ9pgs!84yFi=XV8)C1CD zrEgv&W~rU*Ked%6*gmYjlwT8Wq$_4w^K;wmy}y;v+fnM=cOSb#m?A6iNQ3NpZqT>< zde*!j%RMl&k>vJEOy%MopOx>Bjy3Cvtk=KT6TXjbAZRJD8*A`Z~?x80cDbH}pk)s){2(wj#Oe=5>-K?8S~ntlq9K2@Yeu%w+ZDF>}M9 zR9c)NltyWG zF8;6zvDtx=JyFFet1|ru&m;vru`48$6#?Plh!TeU%sf%usu>LD4NU(sSY}1m>!mrm zSKS@!h7l9<&klI1&nyOO{0^ki~@+WlXeJdyd*e#*=%wS;OMZS>{h#J zrjRe0(Qh(mley8yi=IUbXeSihsB=2gpVivTh+9N^uK(!UYTuydw~MM5Et?Szs5{Ml zrDJi1P6+qV|IsJrS2@0MWE85L0G>+0k%+BhUCQ+KX`Y>xKt;g!;qF`+3!v!!Kxk4u zv{!5Ti}?esLNyjKt7?YjA0{<+RDFcxo^KiD$d8ib{V}-OJxcOqdu`Rp#Db)e8qJ-J_s<1{V!VCa=8;%l`4?rfEtR#Ji+cnf8BC4q zkB^EO-gyCBmf!?PbM_2#|Fo3v@ymhI&5@d$LI9=&pr{&2Jd2%@Z|npN#RnzpMke;B z{a>?664dbMteCWkbm2=s{bBu79ePXCQR%6MabEL6%BU!`U-Y;6nAE_kpKmToU@vBb zEV{l3AIqNe&dOoW@mA2)c{`tGaQP0sL5A-GwE&;*l+2|X7R>P2H|}g4$n@3V8UMWE zp8^lo47_1mIkE-vYs826(X!WyaSeniN*jB~qfvW$)5q?NIIo`MMzw}xcSL{`n?_Zs z%c-E=>zRK-pWkiyQK%OiVz%KMkIz1JCpYqB_DqW5y5wH0tXg4Cl&g+xGd=Brnxu1n zr^N-r){O6r0vdDz-!I~7f|^qmG~JhOh$G^I?7p-&+LzpyYC5aLsg~K@(pvyKu72|= zCzlIEt&aVE{|{}{SUkL~`A3_C2=jq`+?n#Pt!Y&_g zVv;_DH+3IV%wMxDEphPoAL7FImsT!k!8F>|8u1S|!V7?m^b7*3G-}EAL2kl-(BLHU zbW_NfUseh}>?Vm=gOcGWlxWCu{Yl#C=v-A+C;adn;BMS{kdNqScSGb3yRWXwT{8S(9^pCRNWdlm<`Ae=u#-mj#4d+YiP#1e`q{HCS z+ldQdN2f;37^#G#EWgG-{DxBSb=&15H8~kYn{}w8Z`=Nv_s5P_;JO!pg}3NBB>zQR z01DjyC7aGgR_Z1`1J=x#_c5L>6+G|_0+hUs6lJw;{w}UI@F(KM#t0>XurQ#e;!Q<^|emr9nHyk=Z@cD7-UG)*_OIsnn(^xE6%wQ4IVVe;vB^^`=Fcc}=N9MkGh`EF&AfAJG-+~v=n)GO<>id2m2)Ar*k@Q2P$0$ z1)Ats3;fu9$H-+#KfJU$bSqe;9%ec?Ik}1f?(G+VBjrqvzCabm6?l0ZIHBdpAi8fl zAI3ulT1InI8lqYARB2V`dv_ettVlo4tvge>(Xq%8l>%+1Vjk)*GT6(dkuG@W`8=ZV%Om?+ zd4tvgD9LBEpC#Rfy3K1q4*0=xIQg;TLjlVTGipY~hF_N&NF&{G>3Xap51xkP&t*po zydQ6k+QDYumwwfBQzw1FHSIK43Rs`i@E-dU<&wo}l|J$i;7|QPUhxGMi*0=x!l*gqH68z0i_;}1?kKEUn8EUbQHDA$=9z>CE09; z=CI~DPrdIDsA9GW>=B%3p%q;3Qi;?lh`ru_E+gHd7}%XKR=$ng!73*m*B^D2+#edY z=S#+`)_xx=NnH!$6e?n2SDt&xmCO>>3S^C4d_@S$2ZCPn{kFsfOk+`6x|+VOK4-6} z>U_oJT}*b8wf|R1c=V0_0v(RgR9qszMc~cTulg*5x27728d?h?A|jqK#*))r@}zg* z(MFt)O}gtfkKbyhq~;Quivmtn+AM&4NT2UK)wQ5@Jzu@2`cHd&t`rTZrOCZum&dMK z8kA;R6Z$QzOS98SS0eN`tGCl{{yAAV3`b`)GrzRzSJUZLu!w^a4mxwU2s5cIeOKe; ze%8~+Nd$ecjW|J1)LPEg%2@{Mi^>7q34CGi=kmZ4v1!INfe4Dlx6?fzfc!yeOqcR~ zFVSp*hR$u3=s;!u;urm{b(q+oVXVgkt&xe=#LJ;d3Y3X%LvB^Zuqs>tEwm>qv%k@u z&egEYQ(>Ux8W7r)yvfrd^7$-m^WM`!398C-g~vCfPN+ILa8alOURUFJ>&L$kbfx!z zqCI@S))L(4kER5`+5i+=^U3(paQU(qh@lAzy9;thn|@*d!1j1ssiU;qY9OP?qgRv# zDvUopR06@j^1+4Jqcm=QU{-UjU7de{?R-E+z{m3BFPa7|0Jd+hR1yAkc&LK^f0{=m z{69vTFWC_!`x*Y2OP>Ocrh-bDw`ta0FhbM8c@FgqUVK;o+qVCBsW3*SOcv|s?7q<7 zUhD9D?k8XvKX-)A|L4gLY5Imh;*otK9e3ww;;{+X@20{F)uS{H+5azw0#b>U;`b1|DRFF8pu+4r-p6 zUg|hok=iNNeSQGu;58%YjoR`29OXFn zuYI532CARCQjUkZ(pSD8OUtqS%y{iwLiiTa+j-(xUwT6TxcO%uaZ2(WxUocCzsRsM zJZ-2x%A=)#HSY!;ei(+U2X`cv88>P3o7g8f@*0<~_PLK%wn`wz{fX*ns&3d1Xmc@_ z2W;5t6~!9h{GhGFc++L;*DDlZDu53i$)XWa1lFzFLN&cx{bkwK7p12h9`7hH)` zb46GdttJRrkR5x~xExjFhh0k9=6?Bac769T;7-pz=R0<2Fj*8dckvQ#?^iw=c4snb z8zA&mO^)^V(rFOH@6o9FLtW-yU}`-#%nwe*gW_AF!(V;o7UI5OR2C0BVY^1JfH>)j zE}V8P2=~*dpAJ0Cod7v99OgEUul(a_G%A$3U};+9>S5fBDV9#B$Hle$7WYL=XbrpM z4HzjeL)J6(1+KPE3z$x}z8o;f@i#IdRQKkiwuP(jqWRu%zjoJ5D)&S!)ta`~(pbQ@ z((%-{RM}+3plq@LAoI@#JIro+viax_mlH;zd|#h5HopPmR95dgUu05`U}D~KdRdE@ zi8Zccy#ndhm9EO>w@e`fX<`x=Q^)zuDn*hqDUA1IYH(F|JAwp#{LAzUjOj)Lb=VXV zpkYNCQ}+zGcqjZ(v=l}#s@3DbD8Lq{0G>6Vq5eDx>|CW{8X$*_cDmZ6#w{3!6)&j| zMfWp;roY=v9@9b1qXS7}E-?Dle3(iU8V0Xlecc3q@#6hJZ_tUpd@T!4eq*gwv8F_$ zS~nI&ZE+1X4!_4*8S?FdlCrYHlI^a8#eOh9zh7^?d~Bx4MI0a=a4dXp4!mgPar<9+ zuQBKnWYN1^*CLtsGMwW+FCLuEUwO3Um-gn3!}iI>Xl_M<&_B#U05E);$)p-^sH?9fa9t8xvyO|ue`cpLPd@0VVG$(u_Uh}#Ta zT$r;*sdsY$ExbzMqy0xd|9)w_0G$7)OBzthqf`c9%W34^==kdi8wLQN;t`IDXZs5^ zyA6n$q3G-TUnBpDT$!J!&m?Mm&w8MCl*;)EnAQIaQ+@feA_0I3zJTq_jz+vtr7%jG ztFNzJ<9j2RWE)xeh^p7t?k{<$lRKPsh`*gw?PkKzpqm(VG|i@F^a*?1&&T9LS<7)$ zGH0GLa`LG@Ut#}?_>ki9b%o=9j}F8MeMZLPe`9Sxv%gTbP^JHMnjLU3@Qw{Aea?M% Q_>!l(s`hQzt^48s2U?9elmGw# literal 0 HcmV?d00001