Skip to content

x/text/language: Match detects wrong language #49176

@eyudkin

Description

@eyudkin

What version of Go are you using (go version)?

$ go version
go1.17.2

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
darwin_amd64

What did you do?

Im trying to resolve a locale language from the "accept-language" http header: en-GB;q=1.0, fr-DE;q=0.9, fr-CA;q=0.8, en-DE;q=0.7

by using golang.org/x/text/language:

m := language.NewMatcher([]language.Tag{language.English, language.French})
desired, _, _ := language.ParseAcceptLanguage("en-GB;q=1.0, fr-DE;q=0.9, fr-CA;q=0.8, en-DE;q=0.7")
tag, i, conf := m.Match(desired...)
fmt.Println("case B", tag, i, conf) // fr-u-rg-dezzzz instead of en-u-rg-gbzzzz

What did you expect to see?

en-u-rg-gbzzzz

What did you see instead?

fr-u-rg-dezzzz

Other comments

Actually this is reproducible at playground: https://play.golang.org/p/puiT36mYjiU
Please notice that english lang has been detected correctly if we remove en-DE from the end of the string: en-GB;q=1.0, fr-DE;q=0.9, fr-CA;q=0.8 (I dont understand how adding ofher low-priority language affects french lang detection but it works)

Realizations in other programming languages

I wrote similar code for other languages to compare the outputs and both of them detected "en" instead of "fr" so probably we have a bug in golang.org/x/text/language package.

javascript

 var parser = require('accept-language-parser');
 
 var language = parser.pick(['en', 'fr'], 'en-GB;q=1.0, fr-DE;q=0.9, fr-CA;q=0.8, en-DE;q=0.7', { loose: true });
 
 console.log(language); // en

java

 import java.util.Arrays;
 import java.util.List;
 import java.util.Locale;
 
 public class MyClass {
     public static void main(String args[]) {
        List<Locale> locales = Arrays.asList(new Locale("en"),new Locale("fr"));
        List<Locale.LanguageRange> list = Locale.LanguageRange.parse("en-GB;q=1.0, fr-DE;q=0.9, fr-CA;q=0.8, en-DE;q=0.7");
        Locale locale = Locale.lookup(list,locales);
      
       System.out.println(locale); // en
     }
 }

Metadata

Metadata

Assignees

No one assigned

    Labels

    NeedsInvestigationSomeone must examine and confirm this is a valid issue and not a duplicate of an existing one.

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions