* origin/master: (297 commits)
driver: Follow-up to grandparent: Have all test suite entry points declare the mock $region_highlight.
Use the new, unreleased zsh 'memo=' feature to remove only our own entries from $region_highlight.
driver: Stop re-declaring $region_highlight. It's unneeded.
docs: regexp highlighter: Fix a wrong associative array name in the example.
docs: Fix obs-repository link
tests: Fix a wrong value of $PREBUFFER in a test, and add checks to prevent this from recurring.
test harness: Fix use of an undefined variable in an error message.
'main': Don't progress the $in_redirection staller while $in_param.
tests: Add an XFail test for issue #712.
'main': Highlight the parentheses of array assignments as reserved words.
CI += zsh-5.8
main: Add tests for arithmetic expansion
main: Add arithmetic substitution highlighting
changelog.md: Restore vertical whitespace before section headers.
'main': Fix issue #677, concerning multiline aliases.
changelog: Update through HEAD.
'main': Further optimize argument parsing.
'main': Optimize a hot path.
tests: Add a performance testing script, for measuring the performance of the 'main' highlighter on a large file.
changelog: Update through HEAD.
test harness: Print the expected-v.-actual on every failure, not just upon cardinality failures.
Document ZSH_HIGHLIGHT_MAXLENGTH.
'main': Fix the last commit's bug concerning parameter elision not happening in redirects in command position.
'main': Add a test for parameter elision not happening in redirects in command position.
'main': Fix regression in zsh 5.3.1 and older: all precmd hooks later than z-sy-h would be aborted.
changelog += WARN_NESTED_VAR fixes (#727, #731)
'main': Fix a regression caused by the great-grandparent commit's WARN_NESTED_VAR fix.
'main': Don't run `_zsh_highlight_main__type` on every non-command word.
'make perf': Show only a cumulative datum per highligher, rather than per test file.
'main': Don't trip WARN_NESTED_VAR.
'main': Follow-up to previous: Document the version number, and deduplicate some option letters.
'main': precommands += strace
editorconfig: Fix Makefile settings
Fix typo
Bump copyright years.
driver: Fix "_zsh_highlight:3: read-only variable: ret" warnings when POSIX_BUILTINS is set.
tests: Add a test for the infinite loop fixed by each of the last two commits.
'main': Fix expansion of positional parameters in `_zsh_highlight_main_highlighter__try_expand_parameter`.
'main': Fix an infinite loop.
'main': precommands += ionice(1) (from util-linux)
driver: Simplify initialization of $zsyh_user_options in the fallback codepath.
driver: Make sure we don't change the return value in a called function.
'main': Make logic more robust. No functional change.
'main': Break out an anonymous function into a named function.
Fix typos in comments.
main: Add test for issue #713
'main': Support the 'env' precommand.
test harness: Fix the pretty-printer's padding implementation.
Revert "test harness: Rewrite the columnar pretty-printer without external tools." and "travis: Remove bsdmainutils since column(1) has been removed, three commits ago."
changelog: Update through HEAD.
'main': Correctly highlight '&&' and '||' inside '[[ … ]]' conditions.
'main': Highlight reserved words following assignments as errors.
tests: Add tests for issue #461.
test harness: Output the time information to the same place the test name was printed to.
test harness: Stringify values in a more readable manner.
tests: Add a unit test for a path specified with mixed quoting.
tests: Add a test for issue #498, which has already been fixed.
tests: Test that global qualifiers and command substitutions aren't evaluated.
'main': Don't consider path_prefix in alias expansions.
'main': Add a test for aliases to AUTO_CD directories.
'main': Let AUTO_CD directories be highlighted with their own style.
'main': Add an auxiliary variable for readability.
'main': In command position, do not highlight directories (unless AUTO_CD is set) and non-executable files.
'main': Extend tests to capture the current behaviour.
'main': Add an XFail test for issue #202.
'main': Highlight errors from the EQUALS option.
'main': Let the type determination ignore global aliases when it ignores regular ones.
'main': Add a regression test for parameters that expand to global aliases.
'main': Enable the zsh/parameter codepath of global aliases highlighting.
changelog: Update through HEAD.
travis: Remove bsdmainutils since column(1) has been removed, three commits ago.
'main': Highlight global aliases
tests: Record current behaviour on global aliases.
test harness: Rewrite the columnar pretty-printer without external tools.
test harness: Fix an issue with the pretty-printed $expected_region_highlight/$region_highlight diffing.
'main': Support the "close file descriptor" and "coproc" redirection syntaxes
tests: Add a test for the "close file descriptor" and "coproc" redirection syntaxes
tests: Fix the test added in the last commit.
tests: Add a test for issue #705, concerning continuation lines.
test harness: Let tests fail early by exiting non-zero or by setting a flag.
test harness: Print the test name when $skip_test is set.
test harness: Remove a bogus check.
test harness: Fix $skip_test support, broken yesterday.
travis: Install bsdmainutils to provide column(1).
test harness: When the cardinality check fails, pretty-print \$expected_region_highlight and \$region_highlight.
test harness: Don't leak options from test files to the test harness.
test harness: Fix test failures under zsh 5.0.8 and older.
'main': Fix a bug manifesting under zsh 5.2 and older.
'main': Don't highlight arithmetic expansions as command substitutions.
tests: Add a test documenting the current state, prior to introducing #704.
test harness: Change cardinality check semantics
test harness: No-op change to minimize the next diff.
'main': Document additional meanings of the 'S' $braces_stack flag.
'main': When the redirection operator '>&' or '<&' is followed by a positive integer, do not consider that as a filename; it's always a file descriptor.
'main': Add $last_arg for "lookbehind".
noop: Clarify comment.
'main': Honour the MULTIOS option when applying the 'globbing' style.
'main': Document what $in_redirection is currently used for.
'main': The optimized cmdsubst input syntax doesn't glob.
changelog: Fix markup.
...
The error message uses «$1», but it is always unset at that point.
Furthermore, the NO_UNSET (-u) option is in effect. Therefore, when the
error message was to be printed, zsh's NO_UNSET error message was
printed instead:
.
(anon):20: 1: parameter not set
That error message was printed to stderr, and was caught by the caller
within the test harness:
.
Bail out! On './highlighters/main/test-data/path_prefix3.zsh': output on stderr
Thus, the bug being fixed would not caused false positive or false
negative test results, but only wrong error messages.
Follow-up to 0.7.1-71-ge6eea1f, "test harness: Don't leak options from
test files to the test harness".
% repeat 3 { zsh -f tests/test-zprof.zsh main | tee … | grep -w _zsh_highlight | head -n1 }
18) 1 26895.86 26895.86 100.00% 6.35 6.35 0.02% _zsh_highlight
19) 1 27399.11 27399.11 100.00% 5.52 5.52 0.02% _zsh_highlight
19) 1 27027.58 27027.58 100.00% 5.66 5.66 0.02% _zsh_highlight
----
This commit has been rebased. The above statistics were measured after
the rebase. The below statistics had been measured before the rebase.
num calls time self name
-----------------------------------------------------------------------------------
1) 3 25689.17 8563.06 98.15% 18422.01 6140.67 70.38% _zsh_highlight_main_highlighter_highlight_list
2) 32390 5706.13 0.18 21.80% 2315.68 0.07 8.85% _zsh_highlight_main_highlighter_highlight_argument
19) 1 26173.33 26173.33 100.00% 5.27 5.27 0.02% _zsh_highlight
Interestingly, if I make the change in this diff to
_zsh_highlight_main_highlighter_highlight_double_quote —
> diff --git a/highlighters/main/main-highlighter.zsh b/highlighters/main/main-highlighter.zsh
> index da6ab2b..bb17618 100644
> --- a/highlighters/main/main-highlighter.zsh
> +++ b/highlighters/main/main-highlighter.zsh
> @@ -1462,10 +1462,13 @@ _zsh_highlight_main_highlighter_highlight_double_quote()
> local i j k ret style
> reply=()
>
> - for (( i = $1 + 1 ; i <= $#arg ; i += 1 )) ; do
> + (( i = $1 ))
> + while (( ++i <= $#arg )); do
> + i=${arg[(ib.i.)[\"\`\$\\\\${histchars[1]}]]}
> (( j = i + start_pos - 1 ))
> (( k = j + 1 ))
> case "$arg[$i]" in
> + ("") break;;
> ('"') break;;
> ('`') saved_reply=($reply)
> _zsh_highlight_main_highlighter_highlight_backtick $i
— it actually makes things measurably slower (!), even on input that has
a large number of pasted double-quoted strings: on «BUFFER=": ${(r.8*1500..foo"bar".):-}"»
the slowdown is (1123.24ms / 1091.06ms = 1.0295). Therefore, I won't be
committing that change.
% git co HEAD^ && repeat 3 { zsh -f tests/test-zprof.zsh main | tee … | grep -w _zsh_highlight | head -n1 }
HEAD is now at 64e3651 'main': Optimize a hot path.
19) 1 28765.13 28765.13 100.00% 5.57 5.57 0.02% _zsh_highlight
19) 1 28566.46 28566.46 100.00% 5.91 5.91 0.02% _zsh_highlight
19) 1 28248.12 28248.12 100.00% 5.57 5.57 0.02% _zsh_highlight
----
This commit has been rebased. The above statistics were measured after
the rebase. The below statistics had been measured before the rebase.
Before this patch:
num calls time self name
-----------------------------------------------------------------------------------
1) 3 33410.81 11136.94 98.51% 19277.07 6425.69 56.84% _zsh_highlight_main_highlighter_highlight_list
19) 1 33916.21 33916.21 100.00% 5.27 5.27 0.02% _zsh_highlight
With this patch:
num calls time self name
-----------------------------------------------------------------------------------
1) 3 27167.49 9055.83 98.17% 18754.77 6251.59 67.77% _zsh_highlight_main_highlighter_highlight_list
19) 1 27674.40 27674.40 100.00% 5.39 5.39 0.02% _zsh_highlight
And if test-zprof.zsh is changed to not set interactivecomments:
num calls time self name
-----------------------------------------------------------------------------------
1) 13360 36029.12 2.70 83.56% 30304.23 2.27 70.28% _zsh_highlight_main_highlighter_highlight_argument
21) 1 43117.76 43117.76 100.00% 4.52 4.52 0.01% _zsh_highlight
num calls time self name
-----------------------------------------------------------------------------------
1) 13360 14782.89 1.11 68.12% 9163.42 0.69 42.23% _zsh_highlight_main_highlighter_highlight_argument
21) 1 21699.93 21699.93 100.00% 4.17 4.17 0.02% _zsh_highlight
% git co HEAD^ && repeat 3 { zsh -f tests/test-zprof.zsh main | tee … | grep -w _zsh_highlight | head -n1 }
HEAD is now at f1948df tests: Add a performance testing script, for measuring the performance of the 'main' highlighter on a large file.
19) 1 34378.97 34378.97 100.00% 5.43 5.43 0.02% _zsh_highlight
19) 1 34058.34 34058.34 100.00% 5.50 5.50 0.02% _zsh_highlight
19) 1 34364.80 34364.80 100.00% 5.36 5.36 0.02% _zsh_highlight
The great-grandparent commit, "a3ae74 'main': Fix the last commit's bug
concerning parameter elision not happening in redirects in command
position.", is not added because it's not a change with respect
to 0.7.1.
In those versions of zsh, «[[ -o nosuchoption ]]» is regarded as
a syntax error. In newer zsh versions, it merely returns non-zero
(specifically, it returns 3, unlike «[[ -o unsetoption ]]» which
returns 1).
Fixes#732.
Fixes#733.
Without this patch, `_zsh_highlight` was invoked by add-zle-hook-widget
with `$?` being non-zero (see add-zle-hook-widget:48-52). Since
`_zsh_highlight` preserves `$?` from its caller's point of view,
add-zle-hook-widget saw a non-zero exit code from `_zsh_highlight` and
did not run any the remaining zle-line-pre-redraw hooks.
See https://github.com/zsh-users/zsh-syntax-highlighting/issues/579#issuecomment-623576907.
As described in the last commit's log message, ${parameter_name_pattern]
explicitly matches positional parameters but ${parameters[$MATCH]}
expands to nothing in that case (when, e.g., [[ $MATCH == '1' ]]; note
this is equality of strings, not integers).
As a side effect, this removes the dependency on the zsh/parameter
module for expanding parameters.
On the test case, the behaviour was as follows:
+highlighters/main/main-highlighter.zsh:733> _zsh_highlight_main_highlighter__try_expand_parameter '$1'
+highlighters/main/main-highlighter.zsh:432> local arg='$1'
+highlighters/main/main-highlighter.zsh:433> unset reply
+highlighters/main/main-highlighter.zsh:439> local -a match mbegin mend
+highlighters/main/main-highlighter.zsh:440> local MATCH
+highlighters/main/main-highlighter.zsh:440> integer MBEGIN MEND
+highlighters/main/main-highlighter.zsh:441> local parameter_name
+highlighters/main/main-highlighter.zsh:442> local -a words
+highlighters/main/main-highlighter.zsh:443> [[ '$' != \$ ]]
+highlighters/main/main-highlighter.zsh:446> [[ 1 == { ]]
+highlighters/main/main-highlighter.zsh:449> parameter_name=1
+highlighters/main/main-highlighter.zsh:451> [[ none == none ]]
+highlighters/main/main-highlighter.zsh:451> zmodload -e zsh/parameter
+highlighters/main/main-highlighter.zsh:452> [[ ${parameter_name} -regex-match ^${~parameter_name_pattern}$ ]]
+highlighters/main/main-highlighter.zsh:453> [[ '' != *special* ]]
+highlighters/main/main-highlighter.zsh:456> case array-special (*array*|*assoc*)
+highlighters/main/main-highlighter.zsh:458> words=( '$1' )
+highlighters/main/main-highlighter.zsh:469> reply=( '$1' )
There are two problems here:
- In terms of _zsh_highlight_main_highlighter__try_expand_parameter's
pre- and postconditions, the expansion of the word «$1» (line 733)
included that same word (line 469).
That happened because word-to-be-expanded is passed to
_zsh_highlight_main_highlighter__try_expand_parameter as its first
positional parameter, and in this case the word happened to be «$1».
- Furthermore, the exclusion of special parameters (line 453) false
negatived. That happened because $parameter_name_pattern explicitly
allows positional parameters, but ${parameters[(e)1]} expands to
nothing. This will be fixed in the next commit.
Not a regression from 0.7.1.
Before this commit, if the value didn't begin with a dollar sign,
_zsh_highlight_main_highlighter__try_expand_parameter() would return 1
by accident.¹ Tweak the input validation to make this behaviour
explicit. No functional change.
¹ Specifically, it would return 1 because ${parameter_name}'s value
would be the empty string and ${parameter_name_pattern} wouldn't match
that.
Daniel Shahaf
Nuri Jung
Il Harper
Matthew Martin
Daniel Shahaf
Dimitris Apostolou