sqlfluff gives E716: Key not present in Dictionary: "end_line_no" error

[ferdinandyb]

Information

VIM version

VIM - Vi IMproved 9.1 (2024 Jan 02, compiled Jul 16 2024 13:46:55)
Included patches: 1-589

Operating System: WSL2 running Ubuntu 22.04

What went wrong

I'm constantly getting this error:

Error detected while processing function <SNR>275_VimCloseCallback[11]..<SNR>275_VimExitCallback[22]..function <SNR>275_VimCloseCallback[11]..<SNR>275_VimExitCallback[15]..<lambda>42[1]..<SNR>272_ExitCallback[28]..<SNR>217_HandleExit[24]..<lambda>35[1]..ale#semver#RunWithVersionCheck[8]..<lambda>44[1]..ale_linters#sql#sqlfluff#Handle:
line   17:
E716: Key not present in Dictionary: "end_line_no"
E116: Invalid arguments for function add

Reproducing the bug

Just opened an sql file with ALE enabled.

:ALEInfo

Expand

Current Filetype: sql Available Linters: ['sqlfluff', 'sqlint', 'sqllint'] Linter Aliases: 'sqllint' -> ['sql-lint'] Enabled Linters: ['sqlfluff', 'sqlint', 'sqllint'] Ignored Linters: [] Suggested Fixers: 'pgformatter' - A PostgreSQL SQL syntax beautifier 'remove_trailing_lines' - Remove all blank lines at the end of a file. 'sqlfluff' - Fix SQL files with sqlfluff. 'sqlfmt' - Fix SQL files with sqlfmt. 'sqlformat' - Fix SQL files with sqlformat. 'trim_whitespace' - Remove all trailing whitespace characters at the end of every line.

Linter Variables:
" Press Space to read :help for a setting
let g:ale_sql_sqlfluff_executable = 'sqlfluff'
let g:ale_sql_sqlfluff_options = ''

Global Variables:
" Press Space to read :help for a setting
let g:ale_cache_executable_check_failures = 1
let g:ale_change_sign_column_color = 0
let g:ale_command_wrapper = ''
let g:ale_completion_delay = v:null
let g:ale_completion_enabled = 0
let g:ale_completion_max_suggestions = v:null
let g:ale_disable_lsp = 1
let g:ale_echo_cursor = 1
let g:ale_echo_msg_error_str = 'Error'
let g:ale_echo_msg_format = '%linter% - %code: %%s'
let g:ale_echo_msg_info_str = 'Info'
let g:ale_echo_msg_warning_str = 'Warning'
let g:ale_enabled = 1
let g:ale_fix_on_save = 1
let g:ale_fixers = {'go': ['gofumpt'], 'vue': ['prettier', 'eslint'], 'sh': ['shfmt'], '*': ['remove_trailing_lines', 'trim_whitespace'], 'javascript': ['prettier', 'eslint'], 'python': ['black', 'ruff']}
let g:ale_history_enabled = 1
let g:ale_info_default_mode = 'preview'
let g:ale_history_log_output = 1
let g:ale_keep_list_window_open = 0
let g:ale_lint_delay = 200
let g:ale_lint_on_enter = 1
let g:ale_lint_on_filetype_changed = 1
let g:ale_lint_on_insert_leave = 1
let g:ale_lint_on_save = 1
let g:ale_lint_on_text_changed = 'normal'
let g:ale_linter_aliases = {}
let g:ale_linters = {'javascript': ['eslint'], 'python': ['ruff', 'mypy']}
let g:ale_linters_explicit = 0
let g:ale_linters_ignore = {}
let g:ale_list_vertical = 0
let g:ale_list_window_size = 10
let g:ale_loclist_msg_format = '%linter% - %code: %%s'
let g:ale_max_buffer_history_size = 20
let g:ale_max_signs = -1
let g:ale_maximum_file_size = v:null
let g:ale_open_list = 0
let g:ale_pattern_options = v:null
let g:ale_pattern_options_enabled = v:null
let g:ale_root = {}
let g:ale_set_balloons = 0
let g:ale_set_highlights = 1
let g:ale_set_loclist = 1
let g:ale_set_quickfix = 0
let g:ale_set_signs = 1
let g:ale_sign_column_always = 0
let g:ale_sign_error = 'E'
let g:ale_sign_info = 'I'
let g:ale_sign_offset = 1000000
let g:ale_sign_style_error = 'E'
let g:ale_sign_style_warning = 'W'
let g:ale_sign_warning = 'W'
let g:ale_sign_highlight_linenrs = 0
let g:ale_type_map = {}
let g:ale_use_neovim_diagnostics_api = 0
let g:ale_use_global_executables = v:null
let g:ale_virtualtext_cursor = 'all'
let g:ale_warn_about_trailing_blank_lines = 1
let g:ale_warn_about_trailing_whitespace = 1

Command History:

(executable check - success) sqlfluff
(finished - exit code 0) ['sh', '-c', '''sqlfluff'' --version']

<<>>
sqlfluff, version 3.1.1
<<>>

(executable check - failure) sqlint
(executable check - failure) sql-lint
(finished - exit code 1) ['/bin/zsh', '-c', '''sqlfluff'' lint --config ''/home/bferdinandy/amp/mlops/prepaid_engine_2024/.sqlfluff'' --format json ''/tmp/vFQSDXV/20/prae_02a_create_base_table_proc.sql''']

<<>>
[{"filepath": "/tmp/vFQSDXV/20/prae_02a_create_base_table_proc.sql", "violations": [{"start_line_no": 27, "start_line_pos": 1, "code": "PRS", "description": "Line 27, Position 1: Found unparsable section: 'CREATE OR REPLACE PROCEDURE .prae_featur...'", "name": "", "warning": false}, {"start_line_no": 27, "start_line_pos": 32, "code": "TMP", "description": "Undefined jinja template variable: 'prae_schema'", "name": "", "warning": false}, {"start_line_no": 38, "start_line_pos": 25, "code": "TMP", "description": "Undefined jinja template variable: 'prae_schema'", "name": "", "warning": false}, {"start_line_no": 39, "start_line_pos": 17, "code": "TMP", "description": "Undefined jinja template variable: 'prae_schema'", "name": "", "warning": false}, {"start_line_no": 297, "start_line_pos": 14, "code": "TMP", "description": "Undefined jinja template variable: 'prae_dlh_pipeline_name'", "name": "", "warning": false}, {"start_line_no": 298, "start_line_pos": 14, "code": "TMP", "description": "Undefined jinja template variable: 'prae_dlh_pipeline_instance_id'", "name": "", "warning": false}], "statistics": {"source_chars": 10426, "templated_chars": 10312, "segments": 79, "raw_segments": 77}, "timings": {"templating": 0.006912617012858391, "lexing": 0.002570675002061762, "parsing": 0.004416504001710564, "linting": 0.0056342479947488755, "AL01": 6.0574995586648583e-05, "AL02": 1.4798002666793764e-05, "AL03": 1.0238989489153028e-05, "AL04": 8.766990504227579e-06, "AL05": 7.234004442580044e-06, "AL06": 7.163995178416371e-06, "AL08": 6.863003363832831e-06, "AL09": 6.442991434596479e-06, "AM01": 6.342001142911613e-06, "AM02": 6.6929933382198215e-06, "AM03": 6.69300789013505e-06, "AM04": 6.722999387420714e-06, "AM05": 6.722999387420714e-06, "AM06": 6.632995791733265e-06, "AM07": 6.8829976953566074e-06, "CP01": 6.903006578795612e-06, "CP02": 6.532995030283928e-06, "CP03": 6.4820051193237305e-06, "CP04": 6.622998625971377e-06, "CP05": 6.4820051193237305e-06, "CV01": 6.422997103072703e-06, "CV02": 7.414011633954942e-06, "CV03": 7.734997780062258e-06, "CV04": 9.217998012900352e-06, "CV05": 6.6020002122968435e-06, "CV06": 3.582799399737269e-05, "CV07": 5.273999704513699e-05, "CV08": 8.104994776658714e-06, "CV09": 9.807699825614691e-05, "CV10": 8.786999387666583e-06, "CV11": 8.025002898648381e-06, "JJ01": 4.749999789055437e-05, "LT01": 0.0021670990099664778, "LT02": 0.0016506569954799488, "LT03": 2.1792002371512353e-05, "LT04": 1.2674005120061338e-05, "LT06": 9.75799048319459e-06, "LT07": 1.060000795405358e-05, "LT08": 8.857008651830256e-06, "LT09": 7.89501063991338e-06, "LT10": 7.344002369791269e-06, "LT11": 7.134003681130707e-06, "LT12": 7.493300654459745e-05, "LT13": 2.2061998606659472e-05, "RF01": 1.1262003681622446e-05, "RF02": 8.946997695602477e-06, "RF03": 8.706992957741022e-06, "RF04": 8.03500006441027e-06, "RF05": 7.754992111586034e-06, "RF06": 7.0929963840171695e-06, "ST01": 7.715003448538482e-06, "ST02": 7.843991625122726e-06, "ST03": 7.394002750515938e-06, "ST04": 7.794995326548815e-06, "ST05": 7.494003511965275e-06, "ST07": 7.0829992182552814e-06, "ST08": 7.794995326548815e-06, "ST09": 6.571994163095951e-06, "TQ01": 1.026899553835392e-05}}]
<<>>

[nospam2998]

The json output from your linter does indeed appear to not have any end_line_no key of your violation dict.

Would applying this patch make things look correct?

diff --git a/ale_linters/sql/sqlfluff.vim b/ale_linters/sql/sqlfluff.vim
index 4381e1ab..ec26512b 100644
--- a/ale_linters/sql/sqlfluff.vim
+++ b/ale_linters/sql/sqlfluff.vim
@@ -55,9 +55,11 @@ function! ale_linters#sql#sqlfluff#Handle(buffer, version, lines) abort
             call add(l:output, {
             \   'filename': l:json.filepath,
             \   'lnum': l:violation.start_line_no,
-            \   'end_lnum': l:violation.end_line_no,
+            \   'end_lnum': get(l:violation, 'end_line_no',
+            \       l:violation.start_line_no),
             \   'col': l:violation.start_line_pos,
-            \   'end_col': l:violation.end_line_pos,
+            \   'end_col': get(l:violation, 'end_line_pos',
+            \       l:violation.start_line_pos),
             \   'text': l:violation.description,
             \   'code': l:violation.code,
             \   'type': 'W',

I assume a proper fix should likely just leave out the end values if they are not present?

[ferdinandyb]

On Tue Sep 10, 2024 at 18:10, nospam2998 ***@***.***> wrote: The json output from your linter does indeed appear to not have any end_line_no key of your violation dict. Would applying this patch make things look correct? ```diff diff --git a/ale_linters/sql/sqlfluff.vim b/ale_linters/sql/sqlfluff.vim index 4381e1ab..ec26512b 100644 --- a/ale_linters/sql/sqlfluff.vim +++ b/ale_linters/sql/sqlfluff.vim @@ -55,9 +55,11 @@ function! ale_linters#sql#sqlfluff#Handle(buffer, version, lines) abort call add(l:output, { \ 'filename': l:json.filepath, \ 'lnum': l:violation.start_line_no, - \ 'end_lnum': l:violation.end_line_no, + \ 'end_lnum': get(l:violation, 'end_line_no', + \ l:violation.start_line_no), \ 'col': l:violation.start_line_pos, - \ 'end_col': l:violation.end_line_pos, + \ 'end_col': get(l:violation, 'end_line_pos', + \ l:violation.start_line_pos), \ 'text': l:violation.description, \ 'code': l:violation.code, \ 'type': 'W', ```

Thanks this works!

I assume a proper fix should likely just leave out the end values if they are not present?

I'm not sure, but I'll run with the patch until there is a fix :)

[nospam2998]

I'm not sure, but I'll run with the patch until there is a fix :)

I was under the impression that the patch provided should unblock you, allowing you to proceed with submitting a PR for your issue...

[ferdinandyb]

I was under the impression that the patch provided should unblock you, allowing you to proceed with submitting a PR for your issue...

Right :) Could you assign me this issue so I can track it as a task? I'll probably not have free time for this for a couple weeks, but I'll be able to get to it at some point. Thanks!