Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bug: Double Curly Braces get replaced with undefined #667

Open
ipkpjersi opened this issue Dec 31, 2024 · 0 comments
Open

Bug: Double Curly Braces get replaced with undefined #667

ipkpjersi opened this issue Dec 31, 2024 · 0 comments
Labels
bug medium severity

Comments

@ipkpjersi
Copy link

Contact Details

No response

What happened?

I see double curly braces are replaced with undefined instead of displaying properly like {{ $time }}, here is an example:

I made this as a medium severity issue, since I'm not sure if it quite classifies as high severity.

I'm also not sure if any of the log info is relevant so I've included it all.

Version

llamafile v1.5

What operating system are you seeing the problem on?

Linux

Relevant log output

sh ./llava-v1.5-7b-q4.llamafile
note: if you have an AMD or NVIDIA GPU then you need to pass -ngl 9999 to enable GPU offloading
{"build":1500,"commit":"a30b324","function":"server_cli","level":"INFO","line":2869,"msg":"build info","tid":"10437056","timestamp":1735674619}
{"function":"server_cli","level":"INFO","line":2872,"msg":"system info","n_threads":12,"n_threads_batch":-1,"system_info":"AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | ","tid":"10437056","timestamp":1735674619,"total_threads":24}
{"function":"load_model","level":"INFO","line":435,"msg":"Multi Modal Mode Enabled","tid":"10437056","timestamp":1735674619}
clip_model_load: model name:   openai/clip-vit-large-patch14-336
clip_model_load: description:  image encoder for LLaVA
clip_model_load: GGUF version: 3
clip_model_load: alignment:    32
clip_model_load: n_tensors:    377
clip_model_load: n_kv:         19
clip_model_load: ftype:        q4_0
clip_model_load: loaded meta data with 19 key-value pairs and 377 tensors from llava-v1.5-7b-mmproj-Q4_0.gguf
clip_model_load: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
clip_model_load: - kv   0:                       general.architecture str              = clip
clip_model_load: - kv   1:                      clip.has_text_encoder bool             = false
clip_model_load: - kv   2:                    clip.has_vision_encoder bool             = true
clip_model_load: - kv   3:                   clip.has_llava_projector bool             = true
clip_model_load: - kv   4:                          general.file_type u32              = 2
clip_model_load: - kv   5:                               general.name str              = openai/clip-vit-large-patch14-336
clip_model_load: - kv   6:                        general.description str              = image encoder for LLaVA
clip_model_load: - kv   7:                     clip.vision.image_size u32              = 336
clip_model_load: - kv   8:                     clip.vision.patch_size u32              = 14
clip_model_load: - kv   9:               clip.vision.embedding_length u32              = 1024
clip_model_load: - kv  10:            clip.vision.feed_forward_length u32              = 4096
clip_model_load: - kv  11:                 clip.vision.projection_dim u32              = 768
clip_model_load: - kv  12:           clip.vision.attention.head_count u32              = 16
clip_model_load: - kv  13:   clip.vision.attention.layer_norm_epsilon f32              = 0.000010
clip_model_load: - kv  14:                    clip.vision.block_count u32              = 23
clip_model_load: - kv  15:                     clip.vision.image_mean arr[f32,3]       = [0.481455, 0.457828, 0.408211]
clip_model_load: - kv  16:                      clip.vision.image_std arr[f32,3]       = [0.268630, 0.261303, 0.275777]
clip_model_load: - kv  17:                              clip.use_gelu bool             = false
clip_model_load: - kv  18:               general.quantization_version u32              = 2
clip_model_load: - type  f32:  235 tensors
clip_model_load: - type  f16:    1 tensors
clip_model_load: - type q4_0:  141 tensors
clip_model_load: CLIP using CPU backend
clip_model_load: text_encoder:   0
clip_model_load: vision_encoder: 1
clip_model_load: llava_projector:  1
clip_model_load: model size:     169.18 MB
clip_model_load: metadata size:  0.17 MB
clip_model_load: params backend buffer size =  169.18 MB (377 tensors)
clip_model_load: compute allocated memory: 32.89 MB
llama_model_loader: loaded meta data with 19 key-value pairs and 291 tensors from llava-v1.5-7b-Q4_K.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = llama
llama_model_loader: - kv   1:                               general.name str              = LLaMA v2
llama_model_loader: - kv   2:                       llama.context_length u32              = 4096
llama_model_loader: - kv   3:                     llama.embedding_length u32              = 4096
llama_model_loader: - kv   4:                          llama.block_count u32              = 32
llama_model_loader: - kv   5:                  llama.feed_forward_length u32              = 11008
llama_model_loader: - kv   6:                 llama.rope.dimension_count u32              = 128
llama_model_loader: - kv   7:                 llama.attention.head_count u32              = 32
llama_model_loader: - kv   8:              llama.attention.head_count_kv u32              = 32
llama_model_loader: - kv   9:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010
llama_model_loader: - kv  10:                          general.file_type u32              = 15
llama_model_loader: - kv  11:                       tokenizer.ggml.model str              = llama
llama_model_loader: - kv  12:                      tokenizer.ggml.tokens arr[str,32000]   = ["<unk>", "<s>", "</s>", "<0x00>", "<...
llama_model_loader: - kv  13:                      tokenizer.ggml.scores arr[f32,32000]   = [0.000000, 0.000000, 0.000000, 0.0000...
llama_model_loader: - kv  14:                  tokenizer.ggml.token_type arr[i32,32000]   = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ...
llama_model_loader: - kv  15:                tokenizer.ggml.bos_token_id u32              = 1
llama_model_loader: - kv  16:                tokenizer.ggml.eos_token_id u32              = 2
llama_model_loader: - kv  17:            tokenizer.ggml.padding_token_id u32              = 0
llama_model_loader: - kv  18:               general.quantization_version u32              = 2
llama_model_loader: - type  f32:   65 tensors
llama_model_loader: - type q4_K:  193 tensors
llama_model_loader: - type q6_K:   33 tensors
llm_load_vocab: special tokens definition check successful ( 259/32000 ).
llm_load_print_meta: format           = GGUF V3 (latest)
llm_load_print_meta: arch             = llama
llm_load_print_meta: vocab type       = SPM
llm_load_print_meta: n_vocab          = 32000
llm_load_print_meta: n_merges         = 0
llm_load_print_meta: n_ctx_train      = 4096
llm_load_print_meta: n_embd           = 4096
llm_load_print_meta: n_head           = 32
llm_load_print_meta: n_head_kv        = 32
llm_load_print_meta: n_layer          = 32
llm_load_print_meta: n_rot            = 128
llm_load_print_meta: n_swa            = 0
llm_load_print_meta: n_embd_head_k    = 128
llm_load_print_meta: n_embd_head_v    = 128
llm_load_print_meta: n_gqa            = 1
llm_load_print_meta: n_embd_k_gqa     = 4096
llm_load_print_meta: n_embd_v_gqa     = 4096
llm_load_print_meta: f_norm_eps       = 0.0e+00
llm_load_print_meta: f_norm_rms_eps   = 1.0e-05
llm_load_print_meta: f_clamp_kqv      = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: f_logit_scale    = 0.0e+00
llm_load_print_meta: n_ff             = 11008
llm_load_print_meta: n_expert         = 0
llm_load_print_meta: n_expert_used    = 0
llm_load_print_meta: causal attn      = 1
llm_load_print_meta: pooling type     = 0
llm_load_print_meta: rope type        = 0
llm_load_print_meta: rope scaling     = linear
llm_load_print_meta: freq_base_train  = 10000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_yarn_orig_ctx  = 4096
llm_load_print_meta: rope_finetuned   = unknown
llm_load_print_meta: ssm_d_conv       = 0
llm_load_print_meta: ssm_d_inner      = 0
llm_load_print_meta: ssm_d_state      = 0
llm_load_print_meta: ssm_dt_rank      = 0
llm_load_print_meta: model type       = 7B
llm_load_print_meta: model ftype      = Q4_K - Medium
llm_load_print_meta: model params     = 6.74 B
llm_load_print_meta: model size       = 3.80 GiB (4.84 BPW) 
llm_load_print_meta: general.name     = LLaMA v2
llm_load_print_meta: BOS token        = 1 '<s>'
llm_load_print_meta: EOS token        = 2 '</s>'
llm_load_print_meta: UNK token        = 0 '<unk>'
llm_load_print_meta: PAD token        = 0 '<unk>'
llm_load_print_meta: LF token         = 13 '<0x0A>'
llm_load_tensors: ggml ctx size =    0.17 MiB
llm_load_tensors:        CPU buffer size =  3891.24 MiB
..................................................................................................
llama_new_context_with_model: n_ctx      = 2048
llama_new_context_with_model: n_batch    = 2048
llama_new_context_with_model: n_ubatch   = 512
llama_new_context_with_model: flash_attn = 0
llama_new_context_with_model: freq_base  = 10000.0
llama_new_context_with_model: freq_scale = 1
llama_kv_cache_init:        CPU KV buffer size =  1024.00 MiB
llama_new_context_with_model: KV self size  = 1024.00 MiB, K (f16):  512.00 MiB, V (f16):  512.00 MiB
llama_new_context_with_model:        CPU  output buffer size =     0.14 MiB
llama_new_context_with_model:        CPU compute buffer size =   164.01 MiB
llama_new_context_with_model: graph nodes  = 1030
llama_new_context_with_model: graph splits = 1
{"function":"initialize","level":"INFO","line":489,"msg":"initializing slots","n_slots":1,"tid":"10437056","timestamp":1735674625}
{"function":"initialize","level":"INFO","line":498,"msg":"new slot","n_ctx_slot":2048,"slot_id":0,"tid":"10437056","timestamp":1735674625}
{"function":"server_cli","level":"INFO","line":3090,"msg":"model loaded","tid":"10437056","timestamp":1735674625}

llama server listening at http://127.0.0.1:8080

opening browser tab... (pass --nobrowser to disable)
In the sandboxing block!
{"function":"server_cli","hostname":"127.0.0.1","level":"INFO","line":3213,"msg":"HTTP server listening","port":"8080","tid":"10437056","timestamp":1735674625}
{"function":"update_slots","level":"INFO","line":1659,"msg":"all slots are idle and system prompt is empty, clear the KV cache","tid":"10437056","timestamp":1735674625}
Gtk-Message: 14:50:25.242: Failed to load module "xapp-gtk3-module"
Gtk-Message: 14:50:25.300: Failed to load module "xapp-gtk3-module"
Gtk-Message: 14:50:25.761: Failed to load module "xapp-gtk3-module"
Gtk-Message: 14:50:25.770: Not loading module "atk-bridge": The functionality is provided by GTK natively. Please try to not load it.
{"function":"log_server_request","level":"INFO","line":2794,"method":"GET","msg":"request","params":{},"path":"/","remote_addr":"127.0.0.1","remote_port":40206,"status":200,"tid":"124449073865952","timestamp":1735674626}
{"function":"log_server_request","level":"INFO","line":2794,"method":"GET","msg":"request","params":{},"path":"/index.js","remote_addr":"127.0.0.1","remote_port":40206,"status":200,"tid":"124449073865952","timestamp":1735674626}
{"function":"log_server_request","level":"INFO","line":2794,"method":"GET","msg":"request","params":{},"path":"/completion.js","remote_addr":"127.0.0.1","remote_port":40206,"status":200,"tid":"124449073865952","timestamp":1735674626}
{"function":"log_server_request","level":"INFO","line":2794,"method":"GET","msg":"request","params":{},"path":"/json-schema-to-grammar.mjs","remote_addr":"127.0.0.1","remote_port":40216,"status":200,"tid":"124449073870784","timestamp":1735674626}
{"function":"log_server_request","level":"INFO","line":2794,"method":"GET","msg":"request","params":{},"path":"/index.js","remote_addr":"127.0.0.1","remote_port":40232,"status":200,"tid":"124449073875568","timestamp":1735674626}
{"function":"log_server_request","level":"INFO","line":2794,"method":"GET","msg":"request","params":{},"path":"/json-schema-to-grammar.mjs","remote_addr":"127.0.0.1","remote_port":40234,"status":200,"tid":"124449073880304","timestamp":1735674626}
{"function":"log_server_request","level":"INFO","line":2794,"method":"GET","msg":"request","params":{},"path":"/completion.js","remote_addr":"127.0.0.1","remote_port":40232,"status":200,"tid":"124449073875568","timestamp":1735674626}
{"function":"log_server_request","level":"INFO","line":2794,"method":"GET","msg":"request","params":{},"path":"/favicon.ico","remote_addr":"127.0.0.1","remote_port":40206,"status":404,"tid":"124449073865952","timestamp":1735674626}
{"function":"launch_slot_with_data","level":"INFO","line":884,"msg":"slot is processing task","slot_id":0,"task_id":0,"tid":"10437056","timestamp":1735674650}
{"function":"update_slots","level":"INFO","line":1885,"msg":"slot progression","n_past":0,"num_prompt_tokens_processed":67,"slot_id":0,"task_id":0,"tid":"10437056","timestamp":1735674650}
{"function":"update_slots","level":"INFO","line":1910,"msg":"kv cache rm [p0, end)","p0":0,"slot_id":0,"task_id":0,"tid":"10437056","timestamp":1735674650}
{"function":"print_timings","level":"INFO","line":313,"msg":"prompt eval time     =    1207.91 ms /    67 tokens (   18.03 ms per token,    55.47 tokens per second)","n_tokens_second":55.46761684626032,"num_prompt_tokens_processed":67,"slot_id":0,"t_prompt_processing":1207.912,"t_token":18.028537313432835,"task_id":0,"tid":"10437056","timestamp":1735674671}
{"function":"print_timings","level":"INFO","line":327,"msg":"generation eval time =   19379.16 ms /   148 runs   (  130.94 ms per token,     7.64 tokens per second)","n_decoded":148,"n_tokens_second":7.637071500946687,"slot_id":0,"t_token":130.94024324324323,"t_token_generation":19379.156,"task_id":0,"tid":"10437056","timestamp":1735674671}
{"function":"print_timings","level":"INFO","line":337,"msg":"          total time =   20587.07 ms","slot_id":0,"t_prompt_processing":1207.912,"t_token_generation":19379.156,"t_total":20587.068,"task_id":0,"tid":"10437056","timestamp":1735674671}
{"function":"update_slots","level":"INFO","line":1721,"msg":"slot released","n_cache_tokens":215,"n_ctx":2048,"n_past":214,"n_system_tokens":0,"slot_id":0,"task_id":0,"tid":"10437056","timestamp":1735674671,"truncated":false}
{"function":"log_server_request","level":"INFO","line":2794,"method":"POST","msg":"request","params":{},"path":"/completion","remote_addr":"127.0.0.1","remote_port":42656,"status":200,"tid":"124449073885152","timestamp":1735674671}
{"function":"launch_slot_with_data","level":"INFO","line":884,"msg":"slot is processing task","slot_id":0,"task_id":151,"tid":"10437056","timestamp":1735674704}
{"function":"update_slots","level":"INFO","line":1885,"msg":"slot progression","n_past":201,"num_prompt_tokens_processed":40,"slot_id":0,"task_id":151,"tid":"10437056","timestamp":1735674704}
{"function":"update_slots","level":"INFO","line":1910,"msg":"kv cache rm [p0, end)","p0":201,"slot_id":0,"task_id":151,"tid":"10437056","timestamp":1735674704}
{"function":"print_timings","level":"INFO","line":313,"msg":"prompt eval time     =     758.59 ms /    40 tokens (   18.96 ms per token,    52.73 tokens per second)","n_tokens_second":52.72954489129805,"num_prompt_tokens_processed":40,"slot_id":0,"t_prompt_processing":758.588,"t_token":18.9647,"task_id":151,"tid":"10437056","timestamp":1735674744}
{"function":"print_timings","level":"INFO","line":327,"msg":"generation eval time =   39239.08 ms /   298 runs   (  131.67 ms per token,     7.59 tokens per second)","n_decoded":298,"n_tokens_second":7.594470175738334,"slot_id":0,"t_token":131.67475503355703,"t_token_generation":39239.077,"task_id":151,"tid":"10437056","timestamp":1735674744}
{"function":"print_timings","level":"INFO","line":337,"msg":"          total time =   39997.67 ms","slot_id":0,"t_prompt_processing":758.588,"t_token_generation":39239.077,"t_total":39997.665,"task_id":151,"tid":"10437056","timestamp":1735674744}
{"function":"update_slots","level":"INFO","line":1721,"msg":"slot released","n_cache_tokens":539,"n_ctx":2048,"n_past":538,"n_system_tokens":0,"slot_id":0,"task_id":151,"tid":"10437056","timestamp":1735674744,"truncated":false}
{"function":"log_server_request","level":"INFO","line":2794,"method":"POST","msg":"request","params":{},"path":"/completion","remote_addr":"127.0.0.1","remote_port":40612,"status":200,"tid":"124449073889920","timestamp":1735674744}
{"function":"launch_slot_with_data","level":"INFO","line":884,"msg":"slot is processing task","slot_id":0,"task_id":452,"tid":"10437056","timestamp":1735674811}
{"function":"update_slots","level":"INFO","line":1885,"msg":"slot progression","n_past":538,"num_prompt_tokens_processed":24,"slot_id":0,"task_id":452,"tid":"10437056","timestamp":1735674811}
{"function":"update_slots","level":"INFO","line":1910,"msg":"kv cache rm [p0, end)","p0":538,"slot_id":0,"task_id":452,"tid":"10437056","timestamp":1735674811}
{"function":"print_timings","level":"INFO","line":313,"msg":"prompt eval time     =     505.63 ms /    24 tokens (   21.07 ms per token,    47.47 tokens per second)","n_tokens_second":47.46506867602124,"num_prompt_tokens_processed":24,"slot_id":0,"t_prompt_processing":505.635,"t_token":21.068125,"task_id":452,"tid":"10437056","timestamp":1735674835}
{"function":"print_timings","level":"INFO","line":327,"msg":"generation eval time =   23840.98 ms /   174 runs   (  137.02 ms per token,     7.30 tokens per second)","n_decoded":174,"n_tokens_second":7.298358007865365,"slot_id":0,"t_token":137.01712068965517,"t_token_generation":23840.979,"task_id":452,"tid":"10437056","timestamp":1735674835}
{"function":"print_timings","level":"INFO","line":337,"msg":"          total time =   24346.61 ms","slot_id":0,"t_prompt_processing":505.635,"t_token_generation":23840.979,"t_total":24346.613999999998,"task_id":452,"tid":"10437056","timestamp":1735674835}
{"function":"update_slots","level":"INFO","line":1721,"msg":"slot released","n_cache_tokens":736,"n_ctx":2048,"n_past":735,"n_system_tokens":0,"slot_id":0,"task_id":452,"tid":"10437056","timestamp":1735674835,"truncated":false}
{"function":"log_server_request","level":"INFO","line":2794,"method":"POST","msg":"request","params":{},"path":"/completion","remote_addr":"127.0.0.1","remote_port":41724,"status":200,"tid":"124449073894704","timestamp":1735674835}
{"function":"launch_slot_with_data","level":"INFO","line":884,"msg":"slot is processing task","slot_id":0,"task_id":629,"tid":"10437056","timestamp":1735674839}
{"function":"update_slots","level":"INFO","line":1885,"msg":"slot progression","n_past":614,"num_prompt_tokens_processed":137,"slot_id":0,"task_id":629,"tid":"10437056","timestamp":1735674839}
{"function":"update_slots","level":"INFO","line":1910,"msg":"kv cache rm [p0, end)","p0":614,"slot_id":0,"task_id":629,"tid":"10437056","timestamp":1735674839}
{"function":"print_timings","level":"INFO","line":313,"msg":"prompt eval time     =    2568.46 ms /   137 tokens (   18.75 ms per token,    53.34 tokens per second)","n_tokens_second":53.339396634089404,"num_prompt_tokens_processed":137,"slot_id":0,"t_prompt_processing":2568.458,"t_token":18.747868613138685,"task_id":629,"tid":"10437056","timestamp":1735674863}
{"function":"print_timings","level":"INFO","line":327,"msg":"generation eval time =   21007.26 ms /   146 runs   (  143.89 ms per token,     6.95 tokens per second)","n_decoded":146,"n_tokens_second":6.9499785764530255,"slot_id":0,"t_token":143.88533561643834,"t_token_generation":21007.259,"task_id":629,"tid":"10437056","timestamp":1735674863}
{"function":"print_timings","level":"INFO","line":337,"msg":"          total time =   23575.72 ms","slot_id":0,"t_prompt_processing":2568.458,"t_token_generation":21007.259,"t_total":23575.716999999997,"task_id":629,"tid":"10437056","timestamp":1735674863}
{"function":"update_slots","level":"INFO","line":1721,"msg":"slot released","n_cache_tokens":897,"n_ctx":2048,"n_past":896,"n_system_tokens":0,"slot_id":0,"task_id":629,"tid":"10437056","timestamp":1735674863,"truncated":false}
{"function":"log_server_request","level":"INFO","line":2794,"method":"POST","msg":"request","params":{},"path":"/completion","remote_addr":"127.0.0.1","remote_port":41724,"status":200,"tid":"124449073894704","timestamp":1735674863}
{"function":"launch_slot_with_data","level":"INFO","line":884,"msg":"slot is processing task","slot_id":0,"task_id":778,"tid":"10437056","timestamp":1735674879}
{"function":"update_slots","level":"INFO","line":1885,"msg":"slot progression","n_past":814,"num_prompt_tokens_processed":123,"slot_id":0,"task_id":778,"tid":"10437056","timestamp":1735674879}
{"function":"update_slots","level":"INFO","line":1910,"msg":"kv cache rm [p0, end)","p0":814,"slot_id":0,"task_id":778,"tid":"10437056","timestamp":1735674879}
{"function":"print_timings","level":"INFO","line":313,"msg":"prompt eval time     =    2206.94 ms /   123 tokens (   17.94 ms per token,    55.73 tokens per second)","n_tokens_second":55.73332825842865,"num_prompt_tokens_processed":123,"slot_id":0,"t_prompt_processing":2206.938,"t_token":17.94258536585366,"task_id":778,"tid":"10437056","timestamp":1735674942}
{"function":"print_timings","level":"INFO","line":327,"msg":"generation eval time =   61224.03 ms /   400 runs   (  153.06 ms per token,     6.53 tokens per second)","n_decoded":400,"n_tokens_second":6.533382932598764,"slot_id":0,"t_token":153.06006250000002,"t_token_generation":61224.025,"task_id":778,"tid":"10437056","timestamp":1735674942}
{"function":"print_timings","level":"INFO","line":337,"msg":"          total time =   63430.96 ms","slot_id":0,"t_prompt_processing":2206.938,"t_token_generation":61224.025,"t_total":63430.963,"task_id":778,"tid":"10437056","timestamp":1735674942}
{"function":"update_slots","level":"INFO","line":1721,"msg":"slot released","n_cache_tokens":1337,"n_ctx":2048,"n_past":1336,"n_system_tokens":0,"slot_id":0,"task_id":778,"tid":"10437056","timestamp":1735674942,"truncated":false}
{"function":"log_server_request","level":"INFO","line":2794,"method":"POST","msg":"request","params":{},"path":"/completion","remote_addr":"127.0.0.1","remote_port":58052,"status":200,"tid":"124449073899472","timestamp":1735674942}
{"function":"launch_slot_with_data","level":"INFO","line":884,"msg":"slot is processing task","slot_id":0,"task_id":1181,"tid":"10437056","timestamp":1735674959}
{"function":"update_slots","level":"INFO","line":1885,"msg":"slot progression","n_past":1228,"num_prompt_tokens_processed":144,"slot_id":0,"task_id":1181,"tid":"10437056","timestamp":1735674959}
{"function":"update_slots","level":"INFO","line":1910,"msg":"kv cache rm [p0, end)","p0":1228,"slot_id":0,"task_id":1181,"tid":"10437056","timestamp":1735674959}
{"function":"print_timings","level":"INFO","line":313,"msg":"prompt eval time     =    2854.28 ms /   144 tokens (   19.82 ms per token,    50.45 tokens per second)","n_tokens_second":50.4505161017727,"num_prompt_tokens_processed":144,"slot_id":0,"t_prompt_processing":2854.282,"t_token":19.821402777777777,"task_id":1181,"tid":"10437056","timestamp":1735674991}
{"function":"print_timings","level":"INFO","line":327,"msg":"generation eval time =   29497.10 ms /   181 runs   (  162.97 ms per token,     6.14 tokens per second)","n_decoded":181,"n_tokens_second":6.136196023595809,"slot_id":0,"t_token":162.9674143646409,"t_token_generation":29497.102,"task_id":1181,"tid":"10437056","timestamp":1735674991}
{"function":"print_timings","level":"INFO","line":337,"msg":"          total time =   32351.38 ms","slot_id":0,"t_prompt_processing":2854.282,"t_token_generation":29497.102,"t_total":32351.384,"task_id":1181,"tid":"10437056","timestamp":1735674991}
{"function":"update_slots","level":"INFO","line":1721,"msg":"slot released","n_cache_tokens":1553,"n_ctx":2048,"n_past":1552,"n_system_tokens":0,"slot_id":0,"task_id":1181,"tid":"10437056","timestamp":1735674991,"truncated":false}
{"function":"log_server_request","level":"INFO","line":2794,"method":"POST","msg":"request","params":{},"path":"/completion","remote_addr":"127.0.0.1","remote_port":52270,"status":200,"tid":"124449073904384","timestamp":1735674991}
{"function":"launch_slot_with_data","level":"INFO","line":884,"msg":"slot is processing task","slot_id":0,"task_id":1365,"tid":"10437056","timestamp":1735674991}
{"function":"update_slots","level":"INFO","line":1885,"msg":"slot progression","n_past":1458,"num_prompt_tokens_processed":112,"slot_id":0,"task_id":1365,"tid":"10437056","timestamp":1735674991}
{"function":"update_slots","level":"INFO","line":1910,"msg":"kv cache rm [p0, end)","p0":1458,"slot_id":0,"task_id":1365,"tid":"10437056","timestamp":1735674991}
{"function":"print_timings","level":"INFO","line":313,"msg":"prompt eval time     =    2362.26 ms /   112 tokens (   21.09 ms per token,    47.41 tokens per second)","n_tokens_second":47.41218374591811,"num_prompt_tokens_processed":112,"slot_id":0,"t_prompt_processing":2362.262,"t_token":21.091625,"task_id":1365,"tid":"10437056","timestamp":1735675022}
{"function":"print_timings","level":"INFO","line":327,"msg":"generation eval time =   28648.18 ms /   174 runs   (  164.64 ms per token,     6.07 tokens per second)","n_decoded":174,"n_tokens_second":6.0736842619670774,"slot_id":0,"t_token":164.64471264367816,"t_token_generation":28648.18,"task_id":1365,"tid":"10437056","timestamp":1735675022}
{"function":"print_timings","level":"INFO","line":337,"msg":"          total time =   31010.44 ms","slot_id":0,"t_prompt_processing":2362.262,"t_token_generation":28648.18,"t_total":31010.442,"task_id":1365,"tid":"10437056","timestamp":1735675022}
{"function":"update_slots","level":"INFO","line":1721,"msg":"slot released","n_cache_tokens":1744,"n_ctx":2048,"n_past":1743,"n_system_tokens":0,"slot_id":0,"task_id":1365,"tid":"10437056","timestamp":1735675022,"truncated":false}
{"function":"log_server_request","level":"INFO","line":2794,"method":"POST","msg":"request","params":{},"path":"/completion","remote_addr":"127.0.0.1","remote_port":52270,"status":200,"tid":"124449073904384","timestamp":1735675022}
{"function":"launch_slot_with_data","level":"INFO","line":884,"msg":"slot is processing task","slot_id":0,"task_id":1542,"tid":"10437056","timestamp":1735675046}
{"function":"update_slots","level":"INFO","line":1885,"msg":"slot progression","n_past":1654,"num_prompt_tokens_processed":116,"slot_id":0,"task_id":1542,"tid":"10437056","timestamp":1735675046}
{"function":"update_slots","level":"INFO","line":1910,"msg":"kv cache rm [p0, end)","p0":1654,"slot_id":0,"task_id":1542,"tid":"10437056","timestamp":1735675046}
{"function":"print_timings","level":"INFO","line":313,"msg":"prompt eval time     =    2500.56 ms /   116 tokens (   21.56 ms per token,    46.39 tokens per second)","n_tokens_second":46.38953452101206,"num_prompt_tokens_processed":116,"slot_id":0,"t_prompt_processing":2500.564,"t_token":21.55658620689655,"task_id":1542,"tid":"10437056","timestamp":1735675072}
{"function":"print_timings","level":"INFO","line":327,"msg":"generation eval time =   23161.64 ms /   146 runs   (  158.64 ms per token,     6.30 tokens per second)","n_decoded":146,"n_tokens_second":6.303527091091493,"slot_id":0,"t_token":158.6413424657534,"t_token_generation":23161.636,"task_id":1542,"tid":"10437056","timestamp":1735675072}
{"function":"print_timings","level":"INFO","line":337,"msg":"          total time =   25662.20 ms","slot_id":0,"t_prompt_processing":2500.564,"t_token_generation":23161.636,"t_total":25662.199999999997,"task_id":1542,"tid":"10437056","timestamp":1735675072}
{"function":"update_slots","level":"INFO","line":1721,"msg":"slot released","n_cache_tokens":1916,"n_ctx":2048,"n_past":1915,"n_system_tokens":0,"slot_id":0,"task_id":1542,"tid":"10437056","timestamp":1735675072,"truncated":false}
{"function":"log_server_request","level":"INFO","line":2794,"method":"POST","msg":"request","params":{},"path":"/completion","remote_addr":"127.0.0.1","remote_port":40656,"status":200,"tid":"124449073909104","timestamp":1735675072}
{"function":"launch_slot_with_data","level":"INFO","line":884,"msg":"slot is processing task","slot_id":0,"task_id":1691,"tid":"10437056","timestamp":1735675077}
{"function":"update_slots","level":"INFO","line":1885,"msg":"slot progression","n_past":1871,"num_prompt_tokens_processed":63,"slot_id":0,"task_id":1691,"tid":"10437056","timestamp":1735675077}
{"function":"update_slots","level":"INFO","line":1910,"msg":"kv cache rm [p0, end)","p0":1871,"slot_id":0,"task_id":1691,"tid":"10437056","timestamp":1735675077}
{"function":"update_slots","level":"INFO","line":1682,"msg":"slot context shift","n_cache_tokens":2048,"n_ctx":2048,"n_discard":1023,"n_keep":1,"n_left":2046,"n_past":2047,"n_system_tokens":0,"slot_id":0,"task_id":1691,"tid":"10437056","timestamp":1735675098}
{"function":"print_timings","level":"INFO","line":313,"msg":"prompt eval time     =    1456.04 ms /    63 tokens (   23.11 ms per token,    43.27 tokens per second)","n_tokens_second":43.26819066849354,"num_prompt_tokens_processed":63,"slot_id":0,"t_prompt_processing":1456.035,"t_token":23.111666666666668,"task_id":1691,"tid":"10437056","timestamp":1735675110}
{"function":"print_timings","level":"INFO","line":327,"msg":"generation eval time =   30900.46 ms /   178 runs   (  173.60 ms per token,     5.76 tokens per second)","n_decoded":178,"n_tokens_second":5.760432977443212,"slot_id":0,"t_token":173.59806179775282,"t_token_generation":30900.455,"task_id":1691,"tid":"10437056","timestamp":1735675110}
{"function":"print_timings","level":"INFO","line":337,"msg":"          total time =   32356.49 ms","slot_id":0,"t_prompt_processing":1456.035,"t_token_generation":30900.455,"t_total":32356.49,"task_id":1691,"tid":"10437056","timestamp":1735675110}
{"function":"update_slots","level":"INFO","line":1721,"msg":"slot released","n_cache_tokens":1089,"n_ctx":2048,"n_past":1088,"n_system_tokens":0,"slot_id":0,"task_id":1691,"tid":"10437056","timestamp":1735675110,"truncated":true}
{"function":"log_server_request","level":"INFO","line":2794,"method":"POST","msg":"request","params":{},"path":"/completion","remote_addr":"127.0.0.1","remote_port":56988,"status":200,"tid":"124449073913872","timestamp":1735675110}
{"function":"launch_slot_with_data","level":"INFO","line":884,"msg":"slot is processing task","slot_id":0,"task_id":1872,"tid":"10437056","timestamp":1735675129}
{"function":"update_slots","level":"INFO","line":1885,"msg":"slot progression","n_past":0,"num_prompt_tokens_processed":1096,"slot_id":0,"task_id":1872,"tid":"10437056","timestamp":1735675129}
{"function":"update_slots","level":"INFO","line":1910,"msg":"kv cache rm [p0, end)","p0":0,"slot_id":0,"task_id":1872,"tid":"10437056","timestamp":1735675129}
{"function":"print_timings","level":"INFO","line":313,"msg":"prompt eval time     =   20600.85 ms /  1096 tokens (   18.80 ms per token,    53.20 tokens per second)","n_tokens_second":53.20169344485237,"num_prompt_tokens_processed":1096,"slot_id":0,"t_prompt_processing":20600.848,"t_token":18.796394160583944,"task_id":1872,"tid":"10437056","timestamp":1735675181}
{"function":"print_timings","level":"INFO","line":327,"msg":"generation eval time =   30627.04 ms /   174 runs   (  176.02 ms per token,     5.68 tokens per second)","n_decoded":174,"n_tokens_second":5.681253469972486,"slot_id":0,"t_token":176.01749425287358,"t_token_generation":30627.044,"task_id":1872,"tid":"10437056","timestamp":1735675181}
{"function":"print_timings","level":"INFO","line":337,"msg":"          total time =   51227.89 ms","slot_id":0,"t_prompt_processing":20600.848,"t_token_generation":30627.044,"t_total":51227.89200000001,"task_id":1872,"tid":"10437056","timestamp":1735675181}
{"function":"update_slots","level":"INFO","line":1721,"msg":"slot released","n_cache_tokens":1270,"n_ctx":2048,"n_past":1269,"n_system_tokens":0,"slot_id":0,"task_id":1872,"tid":"10437056","timestamp":1735675181,"truncated":true}
{"function":"log_server_request","level":"INFO","line":2794,"method":"POST","msg":"request","params":{},"path":"/completion","remote_addr":"127.0.0.1","remote_port":43390,"status":200,"tid":"124449073918640","timestamp":1735675181}
{"function":"log_server_request","level":"INFO","line":2794,"method":"GET","msg":"request","params":{},"path":"/","remote_addr":"127.0.0.1","remote_port":50770,"status":200,"tid":"124449073923408","timestamp":1735675217}
{"function":"log_server_request","level":"INFO","line":2794,"method":"GET","msg":"request","params":{},"path":"/completion.js","remote_addr":"127.0.0.1","remote_port":50778,"status":200,"tid":"124449073928176","timestamp":1735675217}
{"function":"log_server_request","level":"INFO","line":2794,"method":"GET","msg":"request","params":{},"path":"/index.js","remote_addr":"127.0.0.1","remote_port":50770,"status":200,"tid":"124449073923408","timestamp":1735675217}
{"function":"log_server_request","level":"INFO","line":2794,"method":"GET","msg":"request","params":{},"path":"/json-schema-to-grammar.mjs","remote_addr":"127.0.0.1","remote_port":50782,"status":200,"tid":"124449073928544","timestamp":1735675217}
{"function":"log_server_request","level":"INFO","line":2794,"method":"GET","msg":"request","params":{},"path":"/favicon.ico","remote_addr":"127.0.0.1","remote_port":50782,"status":404,"tid":"124449073928544","timestamp":1735675217}
{"function":"launch_slot_with_data","level":"INFO","line":884,"msg":"slot is processing task","slot_id":0,"task_id":2049,"tid":"10437056","timestamp":1735675228}
{"function":"update_slots","level":"INFO","line":1885,"msg":"slot progression","n_past":0,"num_prompt_tokens_processed":68,"slot_id":0,"task_id":2049,"tid":"10437056","timestamp":1735675228}
{"function":"update_slots","level":"INFO","line":1910,"msg":"kv cache rm [p0, end)","p0":0,"slot_id":0,"task_id":2049,"tid":"10437056","timestamp":1735675228}
{"function":"log_server_request","level":"INFO","line":2794,"method":"GET","msg":"request","params":{},"path":"/","remote_addr":"127.0.0.1","remote_port":48250,"status":200,"tid":"124449073929616","timestamp":1735675232}
{"function":"log_server_request","level":"INFO","line":2794,"method":"GET","msg":"request","params":{},"path":"/index.js","remote_addr":"127.0.0.1","remote_port":48250,"status":200,"tid":"124449073929616","timestamp":1735675232}
{"function":"log_server_request","level":"INFO","line":2794,"method":"GET","msg":"request","params":{},"path":"/completion.js","remote_addr":"127.0.0.1","remote_port":48260,"status":200,"tid":"124444750169360","timestamp":1735675232}
{"function":"log_server_request","level":"INFO","line":2794,"method":"GET","msg":"request","params":{},"path":"/json-schema-to-grammar.mjs","remote_addr":"127.0.0.1","remote_port":48276,"status":200,"tid":"124444750174032","timestamp":1735675232}
{"function":"print_timings","level":"INFO","line":313,"msg":"prompt eval time     =    1252.95 ms /    68 tokens (   18.43 ms per token,    54.27 tokens per second)","n_tokens_second":54.27178832725569,"num_prompt_tokens_processed":68,"slot_id":0,"t_prompt_processing":1252.953,"t_token":18.425779411764704,"task_id":2049,"tid":"10437056","timestamp":1735675257}
{"function":"print_timings","level":"INFO","line":327,"msg":"generation eval time =   27321.03 ms /   215 runs   (  127.07 ms per token,     7.87 tokens per second)","n_decoded":215,"n_tokens_second":7.869395558315498,"slot_id":0,"t_token":127.07456279069767,"t_token_generation":27321.031,"task_id":2049,"tid":"10437056","timestamp":1735675257}
{"function":"print_timings","level":"INFO","line":337,"msg":"          total time =   28573.98 ms","slot_id":0,"t_prompt_processing":1252.953,"t_token_generation":27321.031,"t_total":28573.984,"task_id":2049,"tid":"10437056","timestamp":1735675257}
{"function":"update_slots","level":"INFO","line":1721,"msg":"slot released","n_cache_tokens":283,"n_ctx":2048,"n_past":282,"n_system_tokens":0,"slot_id":0,"task_id":2049,"tid":"10437056","timestamp":1735675257,"truncated":false}
{"function":"log_server_request","level":"INFO","line":2794,"method":"POST","msg":"request","params":{},"path":"/completion","remote_addr":"127.0.0.1","remote_port":48246,"status":200,"tid":"124449073928944","timestamp":1735675257}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug medium severity
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ipkpjersi