Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Issues building mosh #56

Open
wknapik opened this issue Mar 8, 2021 · 1 comment
Open

Issues building mosh #56

wknapik opened this issue Mar 8, 2021 · 1 comment

Comments

@wknapik
Copy link

wknapik commented Mar 8, 2021 •
edited
Loading

Hi,

I'm following the README and using docker-compose to set up gg and build mosh to test. I've faced two issues so far. At the end of this post is also the cloudformation template used to provision the S3 bucket and role. Any hints would be appreciated.

Issue number one (invocation failed; this is the one I care about most, it happened after issue number two below and some files had to be uploaded on the second call and a third call is pasted here):

# gg force --jobs 1 --engine lambda src/frontend/mosh-server
→ Loading the thunks...  done (20 ms).
No files to upload.
[warning] invocation failed: TqVbABmm1eYSvjvnhYcF6pJ9.R5ihZhH6MziZd1.Jtm400006cb7
[warning] invocation failed: T8hFsUsSKG9c_Ccu2XSrK.3SqKQjrgiJlS5qPnUV2t8A0000702f
^Cgg-force: killed by signal
# gg describe TqVbABmm1eYSvjvnhYcF6pJ9.R5ihZhH6MziZd1.Jtm400006cb7
{
 "function": {
  "hash": "VYA7BN_Oi7TEF.SFqo2yJu2fVpJOGPyeu5ThcID2g86400123508",
  "args": [
   "/__gg__/g++",
   "-specs=/__gg__/gcc-specs",
   "-DHAVE_CONFIG_H",
   "-I.",
   "-I../../src/include",
   "-pthread",
   "-Wall",
   "-fno-strict-overflow",
   "-D_FORTIFY_SOURCE=2",
   "-fstack-protector-all",
   "-Wstack-protector",
   "--param",
   "ssp-buffer-size=1",
   "-fPIE",
   "-fno-default-inline",
   "-pipe",
   "-Wno-error",
   "-g",
   "-O2",
   "-x",
   "c++",
   "userinput.pb.cc",
   "-E",
   "-frandom-seed=__GG_BEGIN_REPLACE__userinput.pb.cc__GG_END_REPLACE__",
   "-Wno-builtin-macro-redefined",
   "-D__TIMESTAMP__=\"REDACTED\"",
   "-D__DATE__=\"REDACTED\"",
   "-D__TIME__=\"REDACTED\"",
   "-fno-canonical-system-headers",
   "-nostdinc",
   "-isystem/usr/include/c++/7",
   "-isystem/usr/include/x86_64-linux-gnu/c++/7",
   "-isystem/usr/include/c++/7/backward",
   "-isystem/usr/lib/gcc/x86_64-linux-gnu/7/include",
   "-isystem/usr/local/include",
   "-isystem/usr/lib/gcc/x86_64-linux-gnu/7/include-fixed",
   "-isystem/usr/include/x86_64-linux-gnu",
   "-isystem/usr/include",
   "-o",
   "output_0_1"
  ],
  "envars": [
   "PATH=/__gg__",
   "GG_MANIFEST=@{GGHASH:VMpFPyhr5uGl3L5PZFp8iQWiR6e3IsH2RV0MYtueFtWU00000125}"
  ]
 },
 "values": [
  "V.3taUEAV86kHTGQec3GyUNhn11HZDkoiqKwX_O09UK800001610=/usr/include/asm-generic/errno.h",
  "V.49svpdoo.7xZhVkWq3GPF3VwqP7V6sfXf5LX6uzxF40000090c=/usr/include/x86_64-linux-gnu/c++/7/bits/ctype_base.h",
  "V.C2qH8TqgKsSwV.ff60pY4wty.ilMuVFnWtFyfaptJc000010d9=/usr/include/x86_64-linux-gnu/bits/string_fortified.h",
  "V.jrEiDvDcZFIbuMu8QUNWsFdSt0TEiJmdHIXFcm7a3s000044d6=/usr/include/x86_64-linux-gnu/sys/cdefs.h",
  "V08i.J6dUCnwu8IO.2xpL.kv_S0P5RHeNX4tRJYkR2_Y00004489=/usr/include/string.h",
  "V0NxArI.SAw.GLY9pqa_3LohagozFE0X1t_hTtycSiVA00012444=/usr/include/c++/7/bits/stl_tree.h",
  "V0OlsgyqpHFQ8tSiJyQfkvQpKN8lPUE_.dDivVkPWn2U000004b3=/usr/include/alloca.h",
  "V0QLkf2JgR2GCMrEInyNl8yfGxUGIECIDjvthCcT61kQ00002e2b=/usr/include/google/protobuf/arenastring.h",
  "V0eVlIf8SwmML5kNIS3G1pStqcXA0B9EqJZvLRmnxBBo00000557=/usr/include/x86_64-linux-gnu/bits/locale.h",
  "V0gqP4frfw4nbCm28.nIAnVNippoKlU3OPM3ng5nv53M00000425=/usr/include/x86_64-linux-gnu/bits/types/stack_t.h",
  "V12y5zWm_CWOkLTJfP6vww0naSn6RTJMP1oRHsQ3qSOU00001437=/usr/include/x86_64-linux-gnu/c++/7/bits/error_constants.h",
  "V1TljjXHLRvACDDDoTEZN37hw1b7Nc5Wv4LCna7.hbJY000010a8=/usr/include/x86_64-linux-gnu/bits/sigcontext.h",
  "V1b4jVPUU_Yv4bH4L8Hja.2r0v9XV0vSz6jpthZhtbTw0000d955=/usr/include/c++/7/bits/stl_vector.h",
  "V1bGU2FZrf7AU1dTUsy.DtwstqmXr.KERTuWjGntrgRE000017c9=/usr/lib/gcc/x86_64-linux-gnu/7/include-fixed/limits.h",
  "V1fOilAW1nnnYvxr2kk8nrBgzxPjw4w0kcp2hpf0tIfw00000418=/usr/include/x86_64-linux-gnu/bits/stdint-uintn.h",
  "V1imfvEb8V.fRRjXNXPaRJXEO0CGrcwVYbIRkSgOjn4400000120=/usr/include/x86_64-linux-gnu/bits/types/struct_itimerspec.h",
  "V1lpuUrulhZVH.o7tfdU4ctMNreMFGaBW9FoY14zFrWs00000641=/usr/include/c++/7/ios",
  "V1ogjObsaf97z3mctg3Q77QmdQ0OA7MOJi4vPf6VvD.000003eca=/usr/include/c++/7/bits/basic_ios.h",
  "V2WokUtoNkzlBJz0R0QSOpSXwMzBokoNbCnoJmbE4U6o0000009f=/usr/include/x86_64-linux-gnu/bits/types/timer_t.h",
  "V2dw84TzZ9RJ0aVZMXnK92DnxC3xgcRWnLNvz_qKeVZI00006141=/usr/include/c++/7/bits/locale_classes.h",
  "V31ebhE0MaJQJzOl5anA9G8w2195E.yuIY0DfkL1247Y000008ac=/usr/include/c++/7/bits/cxxabi_init_exception.h",
  "V33BPR6Ab5wvxEaE1BDB4eplQP3.P5lZnxxlRHVpN2Uo00001465=/usr/include/c++/7/debug/debug.h",
  "V3Dl78tTA0VDpJFCLGqSyK7arA7cCDUCOd.9JgYrrLKU0000c815=/usr/include/c++/7/bits/stl_map.h",
  "V3FI4RLHzOemJWJP7gwRw8c1qEBu__YR0wxIFw7U8mM0000008cd=/usr/include/c++/7/bits/atomic_lockfree_defines.h",
  "V3NZNGEhVskXlx6zyGRCgqspZDx.vCU5ADaXTBVHRHrA000000ae=/usr/include/x86_64-linux-gnu/bits/types/clockid_t.h",
  "V41.RGOqjT0IaFMXZZA0Yy6QrFZU4V0FNCjFtCXOJZ_s00008bc8=/usr/include/stdlib.h",
  "V42CByvqMzIFgPuVfXro3fE0p8nw2rFb4YgpmV60hje400000598=/usr/include/x86_64-linux-gnu/bits/param.h",
  "V482npK7j9eb2lfw3.0Q.dJNxmB0cQb2vWs0hfrXJcmU0000133a=/usr/include/x86_64-linux-gnu/bits/posix1_lim.h",
  "V4AZ73pd4997izJN9FZvfzdZ83uTq.4ShUyxfCWUb2go00000561=/usr/include/x86_64-linux-gnu/bits/stdlib-bsearch.h",
  "V4Dn5FOJ95cGSkROwm.tIAF00Jftll85Yu22RzIHGpu40000167e=/usr/include/x86_64-linux-gnu/sys/ucontext.h",
  "V4GWvR7cQJiojY2eHnKwRkJQ2KO2wgLReUnexK4G4ySA00010958=/usr/include/google/protobuf/extension_set.h",
  "V4MjzmFuf9kKGbk9a4DHTlrI83zctS97gKy9XVJ2r87o0000171a=/usr/include/c++/7/ext/type_traits.h",
  "V4eBPVRHjhse_DIv5l3llMziBi75LKMj6J7Nmpi1M9l4000008ee=/usr/include/x86_64-linux-gnu/bits/waitstatus.h",
  "V573aP0LrBjOFDtFQ_vCgz2GTy7qPp0DnUFklQkpZVWY000002f8=/usr/include/x86_64-linux-gnu/bits/types/struct_tm.h",
  "V5FNOo5ITSF0SD2x9gKI8aNVobLmmAExVO3wUoNuHZIw000011f3=/usr/include/x86_64-linux-gnu/bits/timex.h",
  "V5HtR_NrsMZgjSde1ld_NDR1VSHGF_01WOuY1ahooBUE00000b37=/usr/include/x86_64-linux-gnu/bits/sigaction.h",
  "V5e1WwMi1uj_uzmH6bbCWX4yv2IeDBZGVZ_K00BQxsqo000008ec=/usr/include/x86_64-linux-gnu/c++/7/bits/ctype_inline.h",
  "V5fsnzgBnhSeTfBOtztbDhUofCzZ73NTW5AvB7DSC36I000006b9=/usr/include/x86_64-linux-gnu/bits/types/__locale_t.h",
  "V6E5q9KT_7TJkmtLY67MCsulhXx0Qj78hRRE7X2rlgJ800000cda=/usr/include/c++/7/bits/std_abs.h",
  "V6PitvWUqF5imPzH_sGWAXUNG7ozg6S2fIjNMtmkhaog00000ef6=/usr/include/c++/7/bits/stl_raw_storage_iter.h",
  "V6W9NqNXGeB6MOWXEDMAJCQYdLK_qWvHaRVU9MFeBImM00002d02=/usr/include/c++/7/system_error",
  "V6aMQpQuKFK7qS94.i2I1BmMd43p6N9xCJ1fzJ1_JN1U00000178=/usr/include/x86_64-linux-gnu/bits/types/struct_timespec.h",
  "V6eRNCERRn7WwI2bLgdeCRtS8lRXThC3RpKuwPArMDes00000739=/usr/include/x86_64-linux-gnu/c++/7/bits/c++allocator.h",
  "V6ezgtSAXBTfDU2tNjioACVd2TDaDFNMptN_QVvLAh3E00000ceb=/usr/include/x86_64-linux-gnu/c++/7/bits/c++locale.h",
  "V70RClaCp6PXECRg5mgDMkboL15d0EQzfbmUrqmz62y400000180=/usr/include/x86_64-linux-gnu/gnu/stubs.h",
  "V7aqTaler2VL1ziKoO2PVC8wezAnsEq8aehLLq_wosms00008807=userinput.pb.cc",
  "V7aqTaler2VL1ziKoO2PVC8wezAnsEq8aehLLq_wosms00008807=userinput.pb.cc",
  "V7jXJX44J0HEnjgXnVaG9Qfu_RRc3TZRkvquxsWwbWOk0000093b=/usr/include/stdc-predef.h",
  "V7vIAKHRX2tau.PTXvLpkYHvO585NJXG0NzGxxxtwpVo00001574=/usr/include/x86_64-linux-gnu/bits/stdio.h",
  "V8CGINbJu8wrKZWJnPBQxSB5PsKpmI49O87Ulq5XwoWU000035ef=/usr/include/c++/7/bits/refwrap.h",
  "V8N0sWFCWi3a60l3aoJ7EMU5PvJ5.guxjGsvfRt.Swzg000003d6=/usr/include/x86_64-linux-gnu/bits/types/locale_t.h",
  "V8hc0mZBCSmNRWXWFW7Ue4sIfo92gLm0D8WYl_Ja28_w000021b2=/usr/include/stdint.h",
  "V8zScDrdnWfUfNrtfDym.Lm3dliAN4zxhCH546r0_bwI0000059d=/usr/include/x86_64-linux-gnu/bits/select2.h",
  "V97Kn8kdigec1MtupLbDJ0n_wQer2S_G50zRxNceX80g00000878=/usr/include/google/protobuf/stubs/atomic_sequence_num.h",
  "V9F0eLX7Lr.dBauXGhkn2HB2DoLU2ZFuSvId36IG.fvY0000a135=/usr/include/pthread.h",
  "V9j8xD_EjJZPPoOsCXMIprfmygTmZjZ4rBfC9x4lWpN800002629=/usr/include/c++/7/bits/shared_ptr_atomic.h",
  "VA6X9CMxshJsK8orm3klBQAtV4eGLMrDW5rDeogHnq4w0000e8e1=/usr/include/c++/7/tuple",
  "VA8sNdoL_Yn7D4b9roFaIMzgQt1Sd4U8McXzMLpWhil400004345=/usr/include/x86_64-linux-gnu/bits/libio.h",
  "VAD4U3.h7zONX3EMqnF6myeAWm.bpd4.g3Ln_GDakgiA00009c04=/usr/include/c++/7/bits/stl_multimap.h",
  "VAZEyTozqXbHk8nNNHyATNLBP734obcs3_GtLim09b5w000004b8=/usr/include/x86_64-linux-gnu/bits/strings_fortified.h",
  "VAeMlPbodSO4yhGdowKiYQLWSpG7wKNbHsLFPsu47I0A000018ef=/usr/include/c++/7/bits/uses_allocator.h",
  "VAjoQVctdj7z7SYVgjV4npEIvdCiZ.soSir6O9o5kO_s000091b8=/usr/include/google/protobuf/wire_format_lite_inl.h",
  "VAx8J0WQcsQcNpjIsLwv83VR8FmcYlhHsgIXJwYBOWSM0000045a=/usr/include/x86_64-linux-gnu/bits/stdlib-float.h",
  "VBELAvt4lXHeVbCE3j_cAa1a306EYwPe2JvaGMTMZEWc00001928=/usr/include/c++/7/cstddef",
  "VBKClVtOSXDzhUntRmbiluefjlLYOcWbj3xzYZldeXqU00000862=/usr/include/c++/7/bits/hash_bytes.h",
  "VBSkafTi7HQ2h7QHAn.hforH01bxwjga7kw7.xnLbCNU000000ce=/usr/include/x86_64-linux-gnu/bits/types/__sigset_t.h",
  "VB_8QA4X2yR2dneetI1_enEFSL5KMobhhsMEgF6iHNTs00000605=/usr/include/x86_64-linux-gnu/bits/uintn-identity.h",
  "VC2mTczaERld8KF6NRuYzbM8vBp7bcjjybPgR01K1IsQ00001a40=/usr/include/x86_64-linux-gnu/bits/thread-shared-types.h",
  "VC8bKYstUDhxHpMSuPmmmLAen5KDbFrGpzWl2.StGfmc00001641=/usr/include/google/protobuf/stubs/casts.h",
  "VCDm9x68pEYL_mCKntkGWWfsmpJr15CZIV6ca9MyxIhM000002d9=/usr/include/x86_64-linux-gnu/bits/siginfo-arch.h",
  "VCKAbkXUtyd2pPrD4xXHixX.7Z7l2j1wjrRgP0IlOl6M0000008a=/usr/include/x86_64-linux-gnu/bits/types/time_t.h",
  "VCQmlk9f2364cpc72zx5itqZ6qlhuUPYhXzYipHvoVNQ0000011f=/usr/include/x86_64-linux-gnu/bits/types/struct_timeval.h",
  "VCT6EEmD0LINgqntKZm85RbxHSRwGY5Uj58fcVuAPxd40000804b=/usr/include/c++/7/istream",
  "VCaWESV_nq1MXJ4kDMpk7QmZnIYobHANF3yrwtVVyvBM0000047b=/usr/include/x86_64-linux-gnu/bits/types/__sigval_t.h",
  "VCwsjSc1J.pak7BVxeXwDZ99tQAIXkUTa7n4g8rS4M8M00000a2f=/usr/include/x86_64-linux-gnu/bits/libc-header-start.h",
  "VCyk1bCF0KYle8wry5rE0B8zinF1A_ZI2SiOIuKWwsg000002010=/usr/include/c++/7/bits/postypes.h",
  "VDKFkWW2eF5K2kAgbTMVRLiuAkBgXsZ3bnHOoav_GC.M00002704=/usr/include/c++/7/bits/range_access.h",
  "VDRkvujxo5U3prLjuqgkDxsOAJZXvSclFubLfYthTcD800000a2f=/usr/include/c++/7/bits/stringfwd.h",
  "VDWPFLA8bjd83lm8fxeGZeSqCjnnWow9YTibuT2v2Kzs00001755=/usr/include/x86_64-linux-gnu/bits/siginfo-consts.h",
  "VDiW7b.6iLq9cwqcQCyGt55uL4AOrwLYfbyAMYOOK2_o00000430=/usr/include/x86_64-linux-gnu/bits/types/struct_sigstack.h",
  "VDkYmInafRdwWwiiW8.YrmLwPJ6vRtZ6iPPt_2mninHo0000161a=/usr/include/c++/7/ext/alloc_traits.h",
  "VDzh_G8xmXH3ynvccHiWtoSbhnkFyUrHwOj3MqllVy_U000011f7=/usr/include/c++/7/bits/stl_relops.h",
  "VERa6lKvFXWPPjpgMXZLL3wwPZVVcFtQBEw0.rg4DlhQ00000973=/usr/include/c++/7/bits/memoryfwd.h",
  "VEofYezJ6lj7zvcbzxORwSSwfS5imBT8QkurUK_DibEI000009ff=/usr/include/c++/7/map",
  "VF60otc8iqUcUS4CzxuFdNkJY.c1VhRdRre12rjhFNHo00001e0c=/usr/include/c++/7/typeinfo",
  "VFIg_T_i_ocQbON0ZWv9gCwlaaZ2v7LJR8zmMHbE41fg0000d1fb=/usr/include/c++/7/bits/basic_string.tcc",
  "VFJuA6ZRGQeHjDR3TVccSU3bAemVrElRmh5Bys8fJayg00000fe8=/usr/lib/gcc/x86_64-linux-gnu/7/include/stdarg.h",
  "VFNlxTqYxtXgTGlXEP4mYM8prLFEj9kwmWrb00Nk9JWI000012a9=/usr/include/c++/7/exception",
  "VFtapfhXrP2WrIgMe.FL17ILy7v9VKOr0UYCvlSBdn0E0000048f=/usr/include/x86_64-linux-gnu/bits/sigstack.h",
  "VGA9LqrN7hT7v8S9dokR0GF5Xbp8MLAzHngEkf8bvTGo00007376=/usr/include/c++/7/streambuf",
  "VHbNLLDb0sSt1n0G6.MugQeHghb40.2rgjMwTlUPlMCY00002878=/usr/include/time.h",
  "VHoCQHLHO3wk8VSx5ZZIH0Ua7wDXaqwWF1yscw4NHtzQ00001ce5=/usr/include/c++/7/bits/stl_construct.h",
  "VHsaN6sLFwPhGlJ3mfYJmwClkEy.iFhZeouxOcMj8uTA0000373c=/usr/lib/gcc/x86_64-linux-gnu/7/include/stddef.h",
  "VI49SYDRhKXQ2oMNaRx0jw_HC2HOd.x0BgbDNX_RIhF400000a54=/usr/include/c++/7/iterator",
  "VIWRGJT4oEF3M2YunbkRcp29jnCireynu0nsH5fRWQsE000004bc=/usr/include/x86_64-linux-gnu/bits/stdio_lim.h",
  "VIWUEDRq7i4xcbN3aYSRCWX443xYPTy5VnNznWbdpR3Q00001f36=/usr/include/x86_64-linux-gnu/bits/types.h",
  "VIeZJyl_4x5kUvQVcRY.ZuBntObM2xKCxzug9VQ4YIRM00000535=/usr/include/x86_64-linux-gnu/c++/7/bits/cpu_defines.h",
  "VIt0KuBuq3VqNRVtEWFXzQdQZx7Ame.yiNd4At.x45Sk0000014a=/usr/lib/gcc/x86_64-linux-gnu/7/include-fixed/syslimits.h",
  "VJ6HcSICgBGtiOu586EvZEbSm3jZ4kqZJWOY1TY3ykNY00001157=/usr/include/c++/7/cstdio",
  "VJQy2aFp..3UP3lCFGn2qlSKko5zex4vV54UtAc8.Zso00003103=/usr/include/x86_64-linux-gnu/bits/stdio2.h",
  "VJnZ35jJP_eAXG.DSSMHw.6jJnTaxgBRO7wu4lXDxu7k0000237b=/usr/include/c++/7/bits/predefined_ops.h",
  "VJskJlC6Ka8gnJenEAvOTrtnjLZQZAlfd9x4bX4GbTl40000006e=/usr/include/x86_64-linux-gnu/bits/types/__FILE.h",
  "VJvZNBlokkstmXxis44cwLjuwRlTdV0ckihbW.jYlePQ000012fc=/usr/include/c++/7/bits/ptr_traits.h",
  "VK_2brouvQpljdni2DplyKQ5kr7xZIxYvBAA7VNTHPmw00001393=/usr/include/google/protobuf/generated_message_util.h",
  "VL7RxjWMk2Cx782pVCMiQUVpSRytNUaozfPxEXPx07yI000005ee=/usr/include/x86_64-linux-gnu/c++/7/bits/atomic_word.h",
  "VLAKKCN66CYtdOTQGF_.T8VoApSVqELyOb8mCkaK0lEo000000b0=/usr/include/x86_64-linux-gnu/bits/endian.h",
  "VLBSOkrgxje63Z_q9YIIhE.V17mZP8JoYc67Y96SVeTw0000064c=/usr/include/asm-generic/errno-base.h",
  "VLJQKWlq_kIxo9TQEZkA433RquP0ihOGS6jwC2QOQq60000027c9=/usr/include/google/protobuf/io/zero_copy_stream.h",
  "VLXPxif9A.O2e9C23jeQ1nCGlV9qx2iaACzFBXPyjUSI000006ea=/usr/include/c++/7/cerrno",
  "VLlZZHnnIYqnQDCbqXPQ_ox5hN6JqHLsHFaJBX5N96sI00005109=/usr/include/c++/7/bits/char_traits.h",
  "VLxD6dPsy99Sfqx0v1aPdaKB80dsFU2huaavJa30SeKU00000c70=/usr/include/x86_64-linux-gnu/bits/local_lim.h",
  "VLz7tl.PaAtf99h04bdQmXmdgTG7ZoCK_jo4gr_tZb0o00001ae8=/usr/include/c++/7/bits/allocator.h",
  "VM6vELLu250Jacw8Q1iE.ce4pcKwJ2bbGG_Wv9RpCO8U00000abb=/usr/include/c++/7/vector",
  "VMLYvlsrHTkOg67iDbo5gXn8044b3VgY7SgghGfubV3Q00002c79=/usr/include/c++/7/array",
  "VMO_CnrrosjOBrSh9stFfqwKXBn2Tho7Ch4kgCfFdGTU000010f4=/usr/include/x86_64-linux-gnu/bits/signum-generic.h",
  "VMg_vJMmgsGGYeSG8dFQmGrhdN7vaqow_RWzzU4toqa800001267=/usr/include/c++/7/memory",
  "VMgipxzdgQE9.tCq6agTxQRYYACLCzloBX7PgGtnFS_w000019fd=/usr/include/c++/7/bits/move.h",
  "VMpFPyhr5uGl3L5PZFp8iQWiR6e3IsH2RV0MYtueFtWU00000125",
  "VN1FfeL44by7a1HRvKUfJKPSAcoxW2alOJTAuu9rvPJY000000b4=/usr/include/x86_64-linux-gnu/bits/types/FILE.h",
  "VN2zB9gmhpqBUeznT1LHb4l4vKmzBtPVO9CAZNP7FuWA00007975=/usr/include/c++/7/bits/istream.tcc",
  "VN859uolaCGrfdmAuTtalO9_WodsFJ4905WQbREF.SIc0000e2af=/usr/include/x86_64-linux-gnu/c++/7/bits/c++config.h",
  "VNBzmqYpm8KToDu3yvH4Kt4GpZfuxXR1BURrEPlHH1kQ00000771=/usr/include/x86_64-linux-gnu/bits/wchar.h",
  "VNDR7gtbI0NgXmZeLUdauPW8c9go9AVk6DYnoaSDOMMU00004eee=/usr/include/c++/7/bits/stl_heap.h",
  "VNcIMEhumTTTEvM53XiGwjdkRk8St8wCZvcTkgR_4N2M000015ac=/usr/include/wctype.h",
  "VNfoPrOxB5D_RD35B4zERPolrh7mQ9DlKnkXrLfQJ87Y00007987=/usr/include/wchar.h",
  "VOJfmc4Ca8cGTFRjzzvmbu7KbJfq_QR9DbcTtNuKJ4zI0000069b=/usr/include/x86_64-linux-gnu/bits/sigthread.h",
  "VOKefuOnB.ueKjtKJAX73JqnomM3tCW8cafZYnYTpjO80000102c=/usr/include/x86_64-linux-gnu/sys/select.h",
  "VOUX3.4NFNpIFOYHMKZW_Ib6m8pvnQxzVOASYQ1RYQVM0000039b=/usr/include/x86_64-linux-gnu/bits/long-double.h",
  "VOiBtG5Sgu5sZLKP4TJ5hllV4Z9DdhxIcNBODLPabdak000029d3=/usr/include/google/protobuf/stubs/atomicops_internals_x86_gcc.h",
  "VP6jgy2bh3K564MZZJ60nUT8w4WUgspU_WAObZ_2aZ6g00003cb5=/usr/include/features.h",
  "VPCEB.2SyZAIO9JqEIKmPoWVKpX32UOFfryrbqITgI0c0000310f=/usr/include/google/protobuf/stubs/port.h",
  "VPHkNqt9LuhUWtCjj31dKWBiDob4J4GNBHsmKuqNLF9A000038af=/usr/include/google/protobuf/message_lite.h",
  "VPOK_qI_jCOAvkw9vgSdP98cash9d2fi77s04FDGqRaw00001f27=/usr/include/c++/7/stdexcept",
  "VPbXe2dmKaU6McZagZpCy6Bbsa7gmiZQNKB9lBFM_N.E000000c3=/usr/include/x86_64-linux-gnu/bits/types/sigset_t.h",
  "VPcMYX_FNQmuNN24levGn49mUmikDbJFTIo0zFNgqUC4000005be=/usr/include/x86_64-linux-gnu/bits/sigevent-consts.h",
  "VQ2TWSQNeGl41nysASb4MJi1Csx3V095Z689WE1Pg9oI00002002=/usr/include/c++/7/bits/functional_hash.h",
  "VQAapb2ZpSByyQ5iFRQRSquVWDHwHsXO2sbcJzzpZ0uQ0000057c=/usr/include/byteswap.h",
  "VROKeJukbPnww4sme1wk2M6FDRlMAfJtqXJymCtIQg0E000048d2=/usr/include/c++/7/bits/stl_pair.h",
  "VS0LdLHCmzQDOg9ElqUl.Fpu3v.lWBPZVEDjkNmYgcLM00001dfa=/usr/include/locale.h",
  "VSAs5ENS.p7Z.TgI3q6oIK7QOI8wBKzbF9PsgVE.wmQg00000ae9=/usr/include/c++/7/cwctype",
  "VSj6lenPpfZmfm.21lHKvTwTOYtsGvjwt8q_jFCZ3AQM00000c75=/usr/include/endian.h",
  "VSsPP8abWdjflLRJtfMmAJIZehyhu3rkKoi8W1_CERJY00001f87=/usr/include/c++/7/bits/stl_iterator_base_funcs.h",
  "VSt8zy.lNV05pVtLwqEj247.h8P3guXEu.N4b3U57Fyw000004a3=/usr/include/x86_64-linux-gnu/bits/ss_flags.h",
  "VTJUYgL20P68aLS0IUHLmjcOTwPy5eP8DWES9ok7w94U00001dbb=/usr/include/c++/7/ext/concurrence.h",
  "VTLFz_LjjhGXy.0jV99W24vqu30py6SWvGu2uXvW6UQY000004bf=/usr/include/x86_64-linux-gnu/bits/sys_errlist.h",
  "VTQpe82mk2732vQSWa8fdDOuQhobNEqKRx9rrUuvy7mU00004cc7=userinput.pb.h",
  "VTRXl3HnHNA78Kpf.vOhVBQ.jWxFwYtDbQC2uW.X8y3s0000127c=/usr/include/sched.h",
  "VTaAwn1IjR6j6ZEC3NDRiuY7IrYUd_vhLdovc4QbNvlg00003ce3=/usr/include/google/protobuf/stubs/type_traits.h",
  "VThsO8JJQHY3FyMVEpHw_8d05qZPqbv1VHO5BDvQyBYQ000022ce=/usr/include/google/protobuf/stubs/logging.h",
  "VTuOEUVpDJTFZkuYd2Vfw.pHzhY_nwHneWfVu3e1R3oY0000001f=/usr/include/x86_64-linux-gnu/asm/errno.h",
  "VUA.eDtAtsJ5x4heh4WcqHIG.54f5NYtJv_damd2J54800005ead=/usr/include/x86_64-linux-gnu/c++/7/bits/gthr-default.h",
  "VUts8GyYKfeIJuSHs29EK5wzcPEXZsUS8kQxuu6yiOkA00002932=/usr/include/google/protobuf/stubs/atomicops.h",
  "VUzZPjy9KenJbDbeOcSQ43psOuQP.PlgRcTIGkV8tetc00000e0d=/usr/include/c++/7/ext/string_conversions.h",
  "VVf5JW1b7RxSsEA6woaWgzyryaxh1gUlbALWVcQjE20I00000506=/usr/include/x86_64-linux-gnu/bits/setjmp.h",
  "VVgwCc8wo3auIUIWogGwgrW4GomhNcG9fgaI.mhed9zc0003ac99=/usr/include/c++/7/bits/basic_string.h",
  "VVn27SJPSC7mGmW8Ki493SwLKbLffjktkKJXq3n5__Nk000003a9=/usr/include/linux/limits.h",
  "VVuAZMwuFnMTiivMIVN9NKb4SDsS_90hJMBuzxAyfGco000000cc=/usr/include/x86_64-linux-gnu/bits/siginfo-consts-arch.h",
  "VWLL13sKRcsV_Xg2S6kaiq4HfkZxcrzH.MfllX2FWG7U00006bce=/usr/include/c++/7/bits/vector.tcc",
  "VWSlBtiNOy9IlwCeS8rMTCS_QrdMxAca0IvB6U6BBt9s00005d22=/usr/include/c++/7/bits/atomic_base.h",
  "VX.V9u.ZS0FwpyPKvme2CuhmWrnFWYZe1EZH7arrtR8w00000839=/usr/include/x86_64-linux-gnu/bits/select.h",
  "VXkROiLWfIx5xE1eR4UcRw9xBOoVCo0k9Jpe2W1eYHbI00000c4c=/usr/include/x86_64-linux-gnu/sys/param.h",
  "VYaE5axErBud67KKeZr_0RQLcTBzkdcoBMtWXWZuRfEU00000568=/usr/include/x86_64-linux-gnu/bits/uio_lim.h",
  "VZEcT4vE7yRnZU1JtEReFo68Mvhb20YFW2XcsBI32HdU00006c02=/usr/include/c++/7/bits/stl_uninitialized.h",
  "VZMaS5BycS6X6MssILsI2S09cMRVikEUDxL6P7UK4Flc00002098=/usr/include/c++/7/bits/stl_tempbuf.h",
  "VZwslfsxlx67hWwGHk1Oz9a_TquBWpW3sgyDYpgioioY00001710=/usr/include/google/protobuf/stubs/fastmem.h",
  "V_.3m6r2XbzYRSK6cfkx8MbulHbAkk_BQFdIUXbGk6kQ000005a0=/usr/include/x86_64-linux-gnu/bits/_G_config.h",
  "V_24AsTrAovxvmm0iYpdl2GLe21u.O3bufOguChsTFtA000021e8=/usr/include/c++/7/bits/stl_iterator_base_types.h",
  "VaEwmepgrS8zZ5wMwRKl0dSv9r9EjZWXjYI6iQRd0RMo000081fc=/usr/include/c++/7/bits/stl_function.h",
  "VaWp6_._xdgxwJHVFoayWOujRuHSjHnB_lt1gb3YiBRI0000189d=/usr/include/x86_64-linux-gnu/bits/wctype-wchar.h",
  "VaYFdjgJVDX9i8TvdFaKEciT8vsMMZI82v_yRzs0OgUk000009d6=/usr/include/c++/7/algorithm",
  "Va_GGXw3bpcgId0Y61mUNf.5mkxSGlWoHiE9T3qoT4rg0000001f=/usr/include/x86_64-linux-gnu/asm/param.h",
  "VahDW.xGf59IQV_4_oxp0zWctWw5pIULbqZj3AsjfcxY00001147=/usr/include/x86_64-linux-gnu/bits/floatn.h",
  "VauklKHI4dJWyvvgh.bTcklJsT5svNvCezoAm_gT_s.M00000b32=/usr/include/x86_64-linux-gnu/bits/posix2_lim.h",
  "Vbb6AZD5XIZb2bqEzIWLvOIVTmu0ebdmEXrbL8E200PM00001d53=/usr/include/c++/7/new",
  "VbhPLCyDgwrFiRNrTZqXFiiOtb40DwEOJQyKoCG4jQXg00002fd3=/usr/include/signal.h",
  "VcHb3gHftQsdb2RK4kK1EgdkODmGr1jS7uqsvedhAr_Q00000db5=/usr/include/c++/7/ext/atomicity.h",
  "VcSUIO4CW3RRuZJimHzi_O8chXBySN.kkaVULs1VUW6s000004b4=/usr/include/x86_64-linux-gnu/bits/types/sigevent_t.h",
  "VcW2EWQdOKggtZaKBkrMu3b_XWHNINyOVscKDpyOuTBg000001ba=/usr/include/x86_64-linux-gnu/bits/wordsize.h",
  "VcczKjF5YpR.yc5dHkGgHiM6KT5dKk_rl5MYItFXIJLg000028bc=/__gg__/gcc-specs",
  "Vce29P3NNEywdAdAxLUpAtO.uKQSha6cOWWdg1BzPrCs00001b06=/usr/include/c++/7/iosfwd",
  "VcobYd14_hEPid9VooZACzZ4NP6TGGSzgOHX.6WujScU00000b9e=/usr/include/c++/7/initializer_list",
  "VcyGjnkuqBOXXML_k5S0AzoLyrmLb2CkqSmowc7auNZM0000031c=/usr/include/x86_64-linux-gnu/bits/types/wint_t.h",
  "VdKCOmehref92hrjWA0SKHusCuvIVjufxId2EjIMkHro0000040c=/usr/include/x86_64-linux-gnu/bits/stdint-intn.h",
  "VdLX.3fjgBpk9.3SIhMMXwgWK_NyDFfm_AtHB49v32WQ00010e3d=/usr/include/c++/7/limits",
  "VdPn9VadaOjiQ6KdFSccF0wAVQ.p9mOE7y3_P8PNOAUc00000bff=/usr/include/x86_64-linux-gnu/bits/pthreadtypes.h",
  "VdcPIEKtXMq4ryj1ZR4IdteYdIDujsA9lhHIjIpXWaEk0000008d=/usr/include/linux/param.h",
  "VdivtgBAiEF.tz2QC8HtbdmkkXPjaTm_jt73E_p49YJ800000f11=/usr/include/x86_64-linux-gnu/bits/xopen_lim.h",
  "Ve4FbU7ushx9o6zyGclM4XO0h9sN2P4MlOVJ.hBKSUAs00001edf=/usr/include/google/protobuf/stubs/scoped_ptr.h",
  "VeWWbVMKQnOR.6qJCAXcd.w5rfN3TCxDMa.7d.SFTvAg000012e2=/usr/include/google/protobuf/stubs/template_util.h",
  "Vedl07NtlenigWYGjbuQygBiwH9HtBPopsD2YQKthB1c00003e2c=/usr/include/google/protobuf/stubs/callback.h",
  "VemVNEY7gg4ZbP_IjDy760m64bf1nULp6mgVwCZiBqkk000019b8=/usr/include/x86_64-linux-gnu/sys/types.h",
  "VeowUkb7J0O97pLcxIcDde1r9iFwtzFkefW15a2Qoqvo000055d9=/usr/include/c++/7/ostream",
  "Vf.YeZ_bv_nYoGRx.g8BfYTW5FjOBlRcasr2.yjh1WXM000019e1=/usr/include/google/protobuf/stubs/macros.h",
  "VfMrvcTmfVefCyvh..ANo4UoNfzhV.ZSIQlmi.o1jdNg00000fbe=/usr/include/x86_64-linux-gnu/sys/sysmacros.h",
  "VfS8OIC0PfmHoSZBYq_c.T7L5tCUGFw9Xw.BUS9oi0tY0001691d=/usr/include/c++/7/type_traits",
  "VfYjoDktx59BQsSAw99BXKZ.X_4Dap2PfNq_hKfJAHYk0000160e=/usr/include/c++/7/bits/localefwd.h",
  "VfZSolL7QuUkJZ.jbzVNyXA8q.VWeWNSxm9FNNweXTuM00000257=/usr/include/x86_64-linux-gnu/bits/types/sigval_t.h",
  "VfdwxPnYoGBb6ZDL2OxBRonhn.WOGGGiTCmeZsjV4Lvo00000b88=/usr/include/x86_64-linux-gnu/bits/sysmacros.h",
  "VgGj9fKqqbDjnndylZWSbbo2lzV0Y.rkZhPVztF6Sebc00015734=/usr/include/google/protobuf/repeated_field.h",
  "VgRZaVSnJua8GqvGZTURXnSGG7WSQeOr48z7dNWMraPk000019fe=/usr/include/c++/7/bits/stream_iterator.h",
  "VgYyEkXqtTfe4BY1hTSFgboH7yAI63313K1w80yT_oFA000018b5=/usr/include/c++/7/cstdlib",
  "VgeYOio2ZsovHI8FLFW1_f8j_Zj2YqFej9a6pBDH1d.M0000008f=/usr/include/x86_64-linux-gnu/bits/types/clock_t.h",
  "VgiGRVVSpTByY1Ek0lOqfXPn4WzNI2KYrocU6kANVWPM000012c5=/usr/include/c++/7/ext/new_allocator.h",
  "Vgj0AzCvm0EpvteNIqIW.Q0kJJ9YIEX8l0MIxNa_F.Vo0000257a=/usr/include/c++/7/bits/cpp_type_traits.h",
  "VgxOOAhjLLTTyliQNKW.UdmtFXjITcqwc9Blp67VRuQo000008c8=/usr/include/c++/7/stdlib.h",
  "VgyUvF9fhWI.FrBv73O.rfzIC7Uj98xpCCY3eq.QeA9E00006380=/usr/include/c++/7/bits/unique_ptr.h",
  "VgzHbdpTjZP.Q_QjPO77ZS9khyvKbJHEzaKaiAUWUwJI000008e8=/usr/include/c++/7/bits/exception.h",
  "Vh3ONuu.Ge1k71tJW0RSoSrq99TZpIvXY644bScKfc3s0000309f=/usr/include/c++/7/utility",
  "VhLGsv_Iuy6pvqV0RpL4qsJ3mWpEqZzc2ZyNA6x7bF1E00001290=/usr/include/strings.h",
  "VhqeRSrfWoYRlpvuae1T5RcrWBP2_PKDRDk4CVOADeRE00000148=/usr/lib/gcc/x86_64-linux-gnu/7/include/stdint.h",
  "VhtlJxsc8XyULMNjHjquBzyLg9yG8bmvipmbX3qbI36Y00000713=/usr/include/c++/7/bits/cxxabi_forced.h",
  "Vi67u9WJ5owMMc33DqQR1m4oLwGvgVB2s841Uw7ZQNM0000054e9=/usr/include/c++/7/bits/algorithmfwd.h",
  "ViK5h1QCjlQxmUvfiOHC8LTS7w.FTYHhJB7GxN8KZY2Q0000597d=/usr/include/c++/7/bits/shared_ptr.h",
  "ViLE5K4ivOgjZCIZvTpxWUMXN2wyOAhsKP4KTzvZhb.s00000f34=/usr/include/x86_64-linux-gnu/bits/types/siginfo_t.h",
  "ViLXwRaEIQ1l3wUb6HsSdyl8nJLKmXk123uCjEsXEIa000002ad3=/usr/include/ctype.h",
  "ViY4snQ1V6toRXxG.Fje1iqyQT349F.cJEw2yzc.p94Y000008a9=/usr/include/x86_64-linux-gnu/bits/waitflags.h",
  "Vio1AI5o6MKjiAN8pe07HiIW3OX_aguWzDwao7gGVp.A00009a90=/usr/include/c++/7/bits/locale_facets.tcc",
  "VjES5HC625QmQXhvs7GuY.KwAZwm_pdhdhuNKRuuQE8w00000110=/usr/include/x86_64-linux-gnu/bits/types/sig_atomic_t.h",
  "VjZA9UEgiHGai2eQ.Bne_M9NLHjkxQ6Dkbr4AZ0Zj4mI00000d45=/usr/include/x86_64-linux-gnu/bits/typesizes.h",
  "VjkjFnfJUOblnEiZ7u_YKNsuZIa6daeq2gPjZd1e4._Q00000dbc=/usr/include/c++/7/bits/allocated_ptr.h",
  "Vjqhr0kDIUMLdY_wZpdth.sO5ahKHy7BX93j59kUsAQE00001357=/usr/include/google/protobuf/stubs/platform_macros.h",
  "Vk1hvema763Q6wYqZYSM75.5BkJUuwNu9QuU7yrqNa.M000011df=/usr/include/c++/7/ext/numeric_traits.h",
  "Vk6TNp51ANdeD7KmdTbMZLafiTXBOfAr_mPm722KT1Ko000167ee=/usr/include/c++/7/bits/locale_facets.h",
  "VkHXQKugUDUt_foOchof1lYftaM0stsMIsz6uAmrdDVE00000969=/usr/include/c++/7/cctype",
  "VkPzy8jeOY0QThg_HKhgwyaqwlfVDWTzL9flJ57XJTb00000c4e3=/usr/include/c++/7/bits/stl_algobase.h",
  "VklEFYvksveayAcLy5HNLbfHdNaS0Pu8pqtgj4gGd7.M0000066d=/usr/include/c++/7/bits/exception_defines.h",
  "Vl56CvYjXrQ4.FG4moSmQY5qpaNQFs7.WtbfVFWmo8fI00000fa2=/usr/include/c++/7/bits/ostream_insert.h",
  "VlR3o350Du9C1SwLTKw_ajW0RoBy5Cy_T.hU8_p31ThU00000e49=/usr/include/c++/7/bits/invoke.h",
  "VlzrL_gTwiwEbD44qkmTK_uJC6TzhmJwsJNdA9kHDo9Y00004c81=/usr/include/c++/7/bits/alloc_traits.h",
  "VmMX3Yo0zQCwfOPzGwVjedxrMQAmZ4f1Q5MKa9lr3zus000012a0=/usr/include/c++/7/bits/nested_exception.h",
  "VmRqBIE5cMS7eqCMMHsviEC.W.xuxogJWDt112YdGmjE00000794=/usr/include/c++/7/string",
  "VmgtX1C8BzR7.3A0OmFaC9Yp5J0hyPYzQ_nBgXcPi6ZY00001804=/usr/include/google/protobuf/stubs/once.h",
  "VnT1PORLsjx7mfcsAxj1vf5r7EbfOtWeF6Nl.v3lPMH800000161=/usr/include/asm-generic/param.h",
  "Vnk.7rCVvEhPNxRVR2xA8Mk2IAZ53MgXyC6rGgp5rTAM00001bff=/usr/include/c++/7/backward/binders.h",
  "VoNKazcwpj.6uHz35CiqPhuSRPpri5gCmsKAOPuRwY6c00034363=/usr/include/c++/7/bits/stl_algo.h",
  "VoqhI1cyFYAtbnjkuKNCHhtzi.0u.yKXFfUXEY6rgLdk00005019=/usr/include/x86_64-linux-gnu/bits/wchar2.h",
  "Voqueaz6xZ.OUKw5akRz1p3nxjK3aew196YnrcvL2xTo00001940=/usr/include/c++/7/bits/exception_ptr.h",
  "Voy2WBlg95TA0xrLIZE6JzVWS4ReKAeRF0fd5CWPJq_w00001222=/usr/include/assert.h",
  "VpH.y9JvOBUah6IgFzslUM4q7GyrIfeBCFm6OOnSyMmE00000017=/usr/include/linux/errno.h",
  "VpQ_kLg2z9I2xN40ccRJzWqcldv3ms7v841uXvX.nAoA0000de81=/usr/include/google/protobuf/io/coded_stream.h",
  "Vpfchl1t1k.Fe_iz1XOXN1XcGnbQzYe8Oaisr1iYAAQI00000265=/usr/include/x86_64-linux-gnu/gnu/stubs-64.h",
  "VqFSKU_QGz47aQqZZU_2XUalylaAAsJF9jH5GPc4h3ao0000096b=/usr/include/c++/7/debug/assertions.h",
  "VqGadcFzWtPRzv84ySdzz8f8.hVQt1taDMcS0ceNp6Bc00000e9e=/usr/include/c++/7/ext/aligned_buffer.h",
  "VqNpZECEJzFv9tuGZ4erpGpqKjptXCHs1.pyqjOTqNdI00000661=/usr/include/x86_64-linux-gnu/bits/signum.h",
  "VqVwdrLIJomasu0CNmiYxVzDiq1Fc5Lh7btSXbsJr7eE00009e10=/usr/include/google/protobuf/arena.h",
  "Vqx1LQaJOB37VUHiFFQZwYFdzRDn4gwu5lo2BYxv3feQ00001446=/usr/include/google/protobuf/stubs/stl_util.h",
  "Vr3Uqnm4xkF06Kget7mye5bXAxUp2i8kdy18KOqkCK4E000006c3=/usr/include/x86_64-linux-gnu/bits/byteswap-16.h",
  "Vr7GlIh8Obl5RFi4l660qRmz9xU0FgLTvSgVrwCJRgUs00001341=/usr/include/c++/7/bits/streambuf.tcc",
  "Vr9b3ZFgAbkzP_yjCWOh2u1cb6F9qxpafsiKyC6hlS400000a310=/usr/include/c++/7/bits/stl_iterator.h",
  "Vr_ivTHomnnp8PqlWF4fsLUaPEcSyeC89F4znYTPqYKg00004171=/usr/include/google/protobuf/io/zero_copy_stream_impl_lite.h",
  "VrsGdbuTcoXuhxBn_yilEFtwDFLD4vhFg2dVBhCA56pQ00000591=/usr/include/x86_64-linux-gnu/bits/errno.h",
  "VrvamK2K1oiWzTuD16THoy.zP1zIXb1yz_z8ZT5DHpos000024cb=/usr/include/x86_64-linux-gnu/bits/floatn-common.h",
  "VsM9KNwyb7rmXiawqQYWx6KYJv4s..hTWFcWR3Vd101Q000012b9=/usr/include/google/protobuf/stubs/mutex.h",
  "VsURvll.IvIAQpPlMLpGDf_xd8DFefF5pj3xpnli4MzY00000ee3=/usr/include/x86_64-linux-gnu/bits/sched.h",
  "VscWZ0rkcFFIJ4PmhTBXTX8fHKzm22RWX0yGgij2SRso0000301b=/usr/include/c++/7/bits/ostream.tcc",
  "VslOItWs92qTAxbMoCs6indO6iqo7aNgWHs8aEgUuIm80000161a=/usr/include/x86_64-linux-gnu/bits/stdlib.h",
  "Vt4EgFg312qWTKiVYh95YZ91c1r5N.LyT5MQoTHx.tCI000073e1=/usr/include/stdio.h",
  "Vt5DXSkY_HmxAynQHph21NpS_Wkp1bRiJwKzbGBnZm.o000011a3=/usr/include/x86_64-linux-gnu/bits/cpu-set.h",
  "VtUAwPKbewonpMqHRP7pNR8Zlz2Q1rnFT1mgPX.ozWso00001524=/usr/include/limits.h",
  "VtpCMLb3KT_pdLhDFE5qvuWSAtNvLnME75b5AL25ZAIA0000d22b=/usr/include/c++/7/bits/shared_ptr_base.h",
  "Vtt68qWmixqp1YjLi3Wnr_F7o_tvcAim4aZF1GNFWiaU00000728=/usr/include/errno.h",
  "Vu.VAyrh4kqp_dGnVRrmUuvxgw55OVQH3h.95MbbDYjw00000771=/usr/include/c++/7/clocale",
  "Vu6Jf7pf.IXUywzX5QrS1X8.7wLEIXoRQON83At_t6GU0000841d=/usr/include/c++/7/bits/stl_bvector.h",
  "VuMFDPFFlyfX6gU2y7c7vUCQ_Yi99iDTWloPAu9PTsGs000017c3=/usr/include/c++/7/bits/basic_ios.tcc",
  "VudQf8aLuEPsvZ.TkA9DvMI5mmYEKBTLR2Gjl7021Caw00003150=/usr/include/c++/7/bits/streambuf_iterator.h",
  "VvC7rs_sZMYY9_yQ84HVL2pybE4u_sog9VRkQ1MEFUc4000029c1=/usr/include/c++/7/backward/auto_ptr.h",
  "VvCqVG80NxMQGHf7_EI3BOscjd764Vgnw0eW76lvCR3g000078e0=/usr/include/c++/7/bits/ios_base.h",
  "VvikKtPIAZJq9u5.LuNltL6Zork7OfcUwyorh_H9buVI0000791a=/usr/include/google/protobuf/wire_format_lite.h",
  "Vw3k7OeE.vDmFwmxQsMDn_aATCFYuMzKtORQwXNDPyyI0000196d=/usr/include/c++/7/cwchar",
  "VwC.vGVdRSwZZcUsc4k_uvLPtiUakr46KYdRfHJlM_fw00000d5f=/usr/include/c++/7/bits/concept_check.h",
  "VwGCIMJc8ThpxIZ5He0cLftd7IR3IcEEHduwTobe7dZo00000087=/usr/include/x86_64-linux-gnu/bits/types/mbstate_t.h",
  "VwOzfJXnrAvKwYEMqqSC3U.QxNRVSR8fM9JjfbxKnVpU000015e8=/usr/include/x86_64-linux-gnu/c++/7/bits/gthr.h",
  "VxWHfctW5h8y9AiBZ_bG8tomcPpS0A5l6BXbvPhr7D5Q00000234=/usr/include/x86_64-linux-gnu/bits/types/__mbstate_t.h",
  "Vx_uSBKzxwCaTJqhzYjKnyb6C01q143a52um1J25.92I0000211f=/usr/include/google/protobuf/stubs/common.h",
  "VxalusxikRtn9VMQFuZmPJO6c2XaMMMvSoP5fzCIkQ7c000007d7=/usr/include/x86_64-linux-gnu/c++/7/bits/os_defines.h",
  "Vxbb1nRDivDZttu2As5kdTclXOn0gqRmqg.WoK4KZ9t800001620=/usr/include/x86_64-linux-gnu/bits/byteswap.h",
  "Vyie7.9sdS0apI4AOY.aZ0dEMKVAkqmCiJ2KTZMT47Os00000cb6=/usr/include/c++/7/bits/functexcept.h",
  "VykcYMfZCS9u9reNmndKf0CDcgUVLvJLwNqCQJQenWb000000bb6=/usr/include/x86_64-linux-gnu/bits/time.h",
  "Vytbp4YI6lFNdVOXAqOt5ANnWgLUTTQrJtJmftOj8Ves00000cda=/usr/include/x86_64-linux-gnu/bits/pthreadtypes-arch.h",
  "Vz.e6aOWrdSzePtWHK7dspAEsMgU8lM7lFjaOdZIybEA00000877=/usr/include/c++/7/cstdint",
  "Vz8ZVvJEcpskOhCl_flN0aGcWa82Q7vUXNVMHLRU0Gfo000020b7=/usr/include/c++/7/bits/locale_classes.tcc",
  "Vzmzzcz4ZUbKDBeaDTUHm3QQhwiSuofjCVPfAGhBu6BU000027a3=/usr/include/c++/7/bits/uniform_int_dist.h"
 ],
 "thunks": [],
 "executables": [
  "VYA7BN_Oi7TEF.SFqo2yJu2fVpJOGPyeu5ThcID2g86400123508=/__gg__/g++",
  "Vzp_Kkqw4__bBJS.wMVaHKjGQRpgIAg4I3QCjFv6XNew017dc9f0=/__gg__/cc1plus"
 ],
 "outputs": [
  "output"
 ],
 "links": [],
 "timeout": 12500
}
#

Issue number two (gg gets stuck and has to be killed):

# gg force --jobs 1 --engine lambda src/frontend/mosh-server
→ Loading the thunks...  done (12 ms).
↗ Uploading 515 files (50.7 MiB)... ^Cterminate called without an active exception
Aborted (core dumped)
#

gg got stuck there and I had to ^C to kill it. At this point exactly 500 files were present in the S3 bucket (out of 515). That's a suspiciously round number. I ran strace on the gg process before sending SIGINT, it was hanging on the futex call:

# strace -f -s1000 -p 357736
strace: Process 357736 attached with 2 threads
[pid 357736] futex(0x7fa804ff99d0, FUTEX_WAIT, 3385, NULL) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
[pid 357736] --- SIGINT {si_signo=SIGINT, si_code=SI_KERNEL} ---
[pid 357736] write(2, "terminate called without an active exception\n", 45) = 45
[pid 357736] rt_sigprocmask(SIG_UNBLOCK, [ABRT], NULL, 8) = 0
[pid 357736] rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [HUP INT QUIT TERM CHLD CONT], 8) = 0
[pid 357736] getpid()                   = 3363
[pid 357736] gettid()                   = 3363
[pid 357736] tgkill(3363, 3363, SIGABRT) = 0
[pid 357736] rt_sigprocmask(SIG_SETMASK, [HUP INT QUIT TERM CHLD CONT], NULL, 8) = 0
[pid 357736] --- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=3363, si_uid=0} ---
[pid 357758] +++ killed by SIGABRT (core dumped) +++
+++ killed by SIGABRT (core dumped) +++
#

This is a cloudformation template I used to set up the bucket and role:

---
AWSTemplateFormatVersion: '2010-09-09'

Description: Create resources for gg

Resources:
  GgRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: redacted
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Principal:
              Service: lambda.amazonaws.com
            Effect: Allow
            Action: "sts:AssumeRole"
      ManagedPolicyArns:
        - "arn:aws:iam::aws:policy/AmazonS3FullAccess"
        - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
  GgBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: redacted

Outputs:
  GgRole:
    Value: !Ref GgRole
  GgBucket:
    Value: !Ref GgBucket
@wknapik
Copy link
Author

wknapik commented Mar 10, 2021

@sadjad I could use hint. gg doesn't appear to have a verbose mode and invocation failed doesn't really explain the problem.

One issue I did sort out was that this doesn't work when I'm on a vpn (with routes to 0.0.0.0/1 and 128.0.0.0/1 present).

Manually executing the lambda with the output of gg describe for a thunk hash as input, produces:

'storageBackend': KeyError
Traceback (most recent call last):
  File "/var/task/main.py", line 31, in handler
    os.environ['GG_STORAGE_URI'] = event['storageBackend']
KeyError: 'storageBackend'

Adding the missing key to the inputs

"storageBackend": "s3://redacted/?region=us-west-2",

causes the invocation to succeed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@wknapik