Refined advanced questions

[wongjingping]

Refined advanced questions after eye-balling through all the ones that one of the sqlcoder-7b variants currently fail in.
Updated api_runner to output num_tokens to know when we're hitting the max tokens limit.

Updated GPT-4 turbo results:

python3 main.py \
  -db postgres \
  -q data/instruct_advanced_postgres.csv \
  -o results/openai_gpt4_turbo_advanced.csv \
  -g oa \
  -f prompts/prompt_openai.md \
  -m gpt-4-0125-preview \
  -c 0 \
  -p 20
Preparing questions...
Using all questions from data/instruct_advanced_postgres.csv
Correct so far: 41/64 (64.06%): 100%|████████████████████████████████████████████████████████████████████████████████████████████████| 64/64 [00:50<00:00,  1.27it/s]
                 query_category  num_rows  mean_correct  mean_error_db_exec
0         instructions_cte_join        16        0.8125              0.0625
1       instructions_cte_window         8        0.3750              0.2500
2        instructions_date_join        16        0.5625              0.1875
3  instructions_string_matching         8        0.8750              0.0000
4            keywords_aggregate         8        0.8750              0.0000
5                keywords_ratio         8        0.2500              0.1250
Average correct rate: 0.64

[rishsriv]

Thanks for updating these, and making the instructions clearer!