feat: support basic regex analysers in XML

[alecthomas]

The <analyse> element contains a regex to match against the input, and a score if the pattern matches.

The scores of all matching patterns for a lexer are summed.

Replaces #815, #813 and #826.

[gandarez]

Why scores are added instead of early returned? It won't work unless you implement a control flow and the developer decides if it needs to be added or not.

For example C# Aspx

if csharpAspxAnalyzerPageLanguageRe.MatchString(text) {
	return 0.2
}

if csharpAspxAnalyzerScriptLanguageRe.MatchString(text) {
	return 0.15
}

return 0

[gandarez]

Pygments ex1 and ex2

[alecthomas]

Summing the scores seems like a generally more useful approach to me. For the two PRs you've sent it makes no sense to early exit. If a file has both #include < and using namespace, those are strong signals.

If you'd like different behaviour, send a PR. I could see an extra attribute like single="true" being useful.

[gandarez]

I think would be better if the attribute lives in the root node of analyse. So my suggestion is to change like this. What do you think?

<config>
  <analyse single="true">
    <regex value="(?m)^\s*#include &lt;" score="0.1">
    <regex value="(?m)^\s*#ifn?def " score="0.1">
  </analyse>
</config>

[alecthomas]

Yeah great idea!