MIME type issue with Qt 6.8 from VCPKG 2.6 branch

[daschuer]

Bug Description

Testing fails in windows using the latest VCPKG environment with QT 6.8.2:

2025-03-04T20:30:29.3269963Z info [Qt mainThread] Opened file "D:/a/mixxx/mixxx/src/test/id3-test-data/cover-test-ffmpeg-aac.m4a" using provider "Microsoft Media Foundation 10.0.17763.2989"
2025-03-04T20:30:29.3271881Z warning [Qt mainThread] Using type "aac" instead of "mp3" according to the detected MIME type QMimeType("audio/x-aac") of file "D:/a/mixxx/mixxx/src/test/id3-test-data/cover-test-png.mp3"
2025-03-04T20:30:29.3273897Z warning [Qt mainThread] Using type "aac" instead of "mp3" according to the detected MIME type QMimeType("audio/x-aac") of file "D:/a/mixxx/mixxx/src/test/id3-test-data/cover-test-png.mp3"
2025-03-04T20:30:29.3275755Z info [Qt mainThread] Opened file "D:/a/mixxx/mixxx/src/test/id3-test-data/cover-test-png.mp3" using provider "Microsoft Media Foundation 10.0.17763.2989"
2025-03-04T20:30:29.3277649Z warning [Qt mainThread] Using type "aac" instead of "mp3" according to the detected MIME type QMimeType("audio/x-aac") of file "D:/a/mixxx/mixxx/src/test/id3-test-data/cover-test-png.mp3"
2025-03-04T20:30:29.3279652Z info [Qt mainThread] Opened file "D:/a/mixxx/mixxx/src/test/id3-test-data/cover-test-png.mp3" using provider "Microsoft Media Foundation 10.0.17763.2989"
2025-03-04T20:30:29.3281538Z warning [Qt mainThread] Using type "aac" instead of "mp3" according to the detected MIME type QMimeType("audio/x-aac") of file "D:/a/mixxx/mixxx/src/test/id3-test-data/cover-test-png.mp3"
2025-03-04T20:30:29.3283371Z info [Qt mainThread] Opened file "D:/a/mixxx/mixxx/src/test/id3-test-data/cover-test-png.mp3" using provider "Microsoft Media Foundation 10.0.17763.2989"
2025-03-04T20:30:29.3285306Z D:\a\mixxx\mixxx\src\test\soundproxy_test.cpp(128): error: The difference between expected[i] and actual[i] is 0.07663196325302124, which exceeds kMaxDecodingError, where
2025-03-04T20:30:29.3286409Z expected[i] evaluates to 0.3890499472618103,
2025-03-04T20:30:29.3286902Z actual[i] evaluates to 0.31241798400878906, and
2025-03-04T20:30:29.3287431Z kMaxDecodingError evaluates to 0.0099999997764825821.
2025-03-04T20:30:29.3287976Z i=0 Decoding mismatch after seeking backward

My assumption is that it uses the wrong decoder which than fails in the seek test:

Here is the full test run:
https://github.com/daschuer/mixxx/actions/runs/13662091636/job/38195374089

Version

2.6-alpha

OS

Windows 10

[JoergAtGithub]

I guess this has the same origin as: #14392

[JoergAtGithub]

Up to Qt6.7.2 the MIME type reference from Freedesktop was used: https://github.com/qt/qtbase/tree/6.7.2/src/corelib/mimetypes/mime/packages
Starting with Qt6.7.3 the MIME type reference from Apache Tika is used instead: https://github.com/qt/qtbase/tree/6.7.3/src/corelib/mimetypes/3rdparty

[daschuer]

Here is the commit in question:
qt/qtbase@3571012

[daschuer]

And here are the upstream bugs:
https://bugreports.qt.io/browse/QTBUG-131109
https://bugreports.qt.io/browse/QTBUG-133916

[JoergAtGithub]

I think the cleanest solution would be to add a VCPKG feature to the qt-base port, that enforces the shared-mime-info from Freedesktop.

[daschuer]

I am preparing a shared-mime-info port for VCPKG. It is however unclear how Qt will pick it up in the macOs and Windows case.

[daschuer]

I am out of luck and need help:
https://github.com/daschuer/vcpkg/actions/runs/13721231626
It work nice on Linux and also on macOS however the Windows build fails

"C:\mixxx-vcpkg\downloads\tools\python\python-3.12.7-x64-1\python.exe" "C:/mixxx-vcpkg/downloads/tools/meson-1.6.1-6779de/meson.py" "--internal" "msgfmthelper" "--msgfmt=C:/mixxx-vcpkg/vcpkg_installed/x64-windows/tools/gettext/bin/msgfmt.EXE" "--datadirs=C:/mixxx-vcpkg/buildtrees/shared-mime-info/src/2.4-b756cadafd.clean/data/." "../src/2.4-b756cadafd.clean/data/freedesktop.org.xml.in" "data/freedesktop.org.xml" "xml" "../src/2.4-b756cadafd.clean/data/../po"
C:/mixxx-vcpkg/vcpkg_installed/x64-windows/tools/gettext/bin/msgfmt.EXE: cannot locate ITS rules for ../src/2.4-b756cadafd.clean/data/freedesktop.org.xml.in

The equivalent macOS call works:

[1/2] /usr/local/bin/python3 /Users/runner/mixxx-vcpkg/downloads/tools/meson-1.6.1-6779de/meson.py --internal msgfmthelper --msgfmt=/usr/local/bin/msgfmt --datadirs=/Users/runner/mixxx-vcpkg/buildtrees/shared-mime-info/src/2.4-b756cadafd.clean/data/. ../src/2.4-b756cadafd.clean/data/freedesktop.org.xml.in data/freedesktop.org.xml xml ../src/2.4-b756cadafd.clean/data/../po

I have tried to override GETTEXTDATADIRS but the meson helper also overrides it.

Is there an issue with \ vs / or the drive letter in windows?

@JoergAtGithub can you investigate how msgfmt needs to be called on windows locally. Thank you.

[JoergAtGithub]

I tried to build your shared-mime-info branch localy nd ran into the same problem as on CI. Regarding the path syntax, the only thing I notice is that data/freedesktop.org.xml does not start with ./ , but I think that should work too.
To my understanding, it is looking for some .its files related to data/freedesktop.org.xml, which is not found.
The only .its I found is .\data\its\shared-mime-info.its and a copy of it in the tests directory.

[daschuer]

Can you try to do the msgfmt.EXE call manually?
It is something like:

set GETTEXTDATADIRS=C:/mixxx-vcpkg/buildtrees/shared-mime-info/src/2.4-b756cadafd.clean/data
msgfmt.exe  ../src/2.4-b756cadafd.clean/data/freedesktop.org.xml.in data/freedesktop.org.xml xml ../src/2.4-b756cadafd.clean/data/../po

Can you reproduce the error as well?
The puzzle is to adjust the call that it not fails.

[JoergAtGithub]

I tried

"D:/vcpkg/packages/gettext_x64-windows/tools/gettext/bin/msgfmt.exe" -v --xml --template "D:/vcpkg/buildtrees/shared-mime-info/src/2.4-b756cadafd.clean/data/freedesktop.org.xml.in" -d "D:/vcpkg/buildtrees/shared-mime-info/src/2.4-b756cadafd.clean/data/../po" -o "D:/vcpkg/buildtrees/shared-mime-info/x64-windows-dbg/data/freedesktop.org.xml"

what seems right to me according the online help of msgfmt.exe and got:

D:\vcpkg\packages\gettext_x64-windows\tools\gettext\bin\msgfmt.exe: ITS-Regeln f"ur D:/vcpkg/buildtrees/shared-mime-info/src/2.4-b756cadafd.clean/data/freedesktop.org.xml.in k"onnen nicht gefunden werden

msgfmt.exe --help
Aufruf: D:\vcpkg\packages\gettext_x64-windows\tools\gettext\bin\msgfmt.exe [OPTION] Datei.po ...

Bin"aren Katalog mit Meldungen aus der textuellen "Ubersetzung erstellen.

Notwendige Argumente f"ur Optionen in Langform sind auch f"ur die Kurzform
notwendig. Dies gilt in gleicher Weise f"ur optionale Argumente.

Angaben zu Eingabedateien:
  Datei.po ...                  Eingabedateien
  -D, --directory=VERZ        VERZ der Liste der Eingabedateien hinzuf"ugen
Wenn die Eingabedatei >>-<< ist, wird die Standardeingabe gelesen.

Art der Verarbeitung:
  -j, --java                  Java-Modus: eine ResourceBundle-Klasse f"ur
                                Java erstellen
      --java2                 wie --java, aber Java2 (JDK 1.2 oder h"oher)
                                annehmen
      --qt                    C#-Modus: eine .dll-Datei f"ur .NET erstellen
      --csharp-resources      C#-Ressourcenmodus: eine .resources-Datei f"ur
                                .NET erstellen
      --tcl                   Tcl-Modus: eine .msg-Datei f"ur tcl/msgcat
                                erstellen
      --qt                    Qt-Modus: eine .qm-Datei f"ur Qt erstellen
      --desktop               Desktopeintrag-Modus: erzeugt eine
                                .desktop-Datei
      --xml                   XML-Modus: eine XML-Datei erstellen

Angaben zu Ausgabedateien:
  -o, --output-file=DATEI      Ausgabe in die angegebene DATEI schreiben
      --strict                Datei streng nach Uniforum-Format erstellen
Ergebnisse werden in die Standardausgabe geschrieben, wenn >>-<< angegeben ist.

Angaben zu Ausgabedateien im Java-Modus:
  -r, --resource=RESSOURCE    Name der Ressource
  -l, --locale=LOCALE         Name der Spracheinstellung, entweder >>Sprache<<
                               (z.B. >>de<<) oder >>Sprache_LAND" (z.B. >>de_DE<<)
      --source                erstellt eine .java-Datei anstelle einer
                                .class-Datei
  -d VERZEICHNIS              Basis-Verzeichnis der Verzeichnis-Hierarchie
                                mit den Klassen
Der Name der Klasse wird durch das Anh"angen des Namens der Spracheinstellung
an den Namen der Ressource festgelegt, abgetrennt durch einen Unterstrich
(>>_<<). Die Option >>-d<< ist notwendig. Die Klasse wird unter dem angegebenen
Verzeichnis geschrieben.

Angaben zu Ausgabedateien im C#-Modus:
  -r, --resource=RESSOURCE    Name der Ressource
  -l, --locale=LOCALE         Name der Spracheinstellung, entweder >>Sprache<<
                               (z.B. >>de<<) oder >>Sprache_LAND" (z.B. >>de_DE<<)
  -d VERZEICHNIS              Basis-Verzeichnis der von der
                                Spracheinstellung abh"angigen .dll-Dateien
Die Optionen >>-l<< und >>-d<< sind notwendig. Die .dll-Datei wird in ein
Unterverzeichnis des angegebenen Verzeichnisses geschrieben, dessen Name von
der Spracheinstellung abh"angig ist.

Angaben zu Ausgabedateien im Tcl-Modus:
  -l, --locale=LOCALE         Name der Spracheinstellung, entweder >>Sprache<<
                               (z.B. >>de<<) oder >>Sprache_LAND" (z.B. >>de_DE<<)
  -d VERZEICHNIS              Basis-Verzeichnis der .msg-Kataloge mit
                                Meldungen
Die Optionen >>-l<< und >>-d<< sind notwendig. Die .msg-Datei wird in das
angegebene Verzeichnis geschrieben.

Optionen f"ur Desktopeintrag-Modus:
  -l, --locale=LOCALE         Name der Spracheinstellung, entweder >>Sprache<<
                               (z.B. >>de<<) oder >>Sprache_LAND" (z.B. >>de_DE<<)
  -o, --output-file=DATEI      Ausgabe in die angegebene DATEI schreiben
  --template=VORLAGE          eine als Vorlage zu verwendende .desktop-Datei
  -d VERZEICHNIS              Basisverzeichnis der .po-Dateien
  -kWORD, --keyword=WORT      zus"atzliches Schl"usselwort, nach dem gesucht
                                wird.
  -k, --keyword               es wird nicht nach den vorgegebenen
                                Schl"usselw"ortern gesucht
Die Optionen -l, -o und --template sind obligatorisch. Falls -D angegeben
ist, werden Dateien aus dem Verzeichnis anstelle der Befehlszeilenargumente
gelesen.

Optionen f"ur XML-Modus:
  -l, --locale=LOCALE         Name der Spracheinstellung, entweder >>Sprache<<
                               (z.B. >>de<<) oder >>Sprache_LAND" (z.B. >>de_DE<<)
  -L, --language=NAME         die angegebene XML-Sprache erkennen
  -o, --output-file=DATEI      Ausgabe in die angegebene DATEI schreiben
  --template=VORLAGE          eine als Vorlage zu verwendende XML-Datei
  -d VERZEICHNIS              Basisverzeichnis der .po-Dateien
Die Optionen -l, -o und --template sind obligatorisch. Falls -D angegeben
ist, werden Dateien aus dem Verzeichnis anstelle der Befehlszeilenargumente
gelesen.

Syntax der Eingabedatei:
  -P, --properties-input      Eingabedateien folgen der .properties-Syntax
                                von Java
      --stringtable-input     Eingabedateien folgen der
                                NeXTstep/GNUstep-Syntax f"ur .strings

"Uberpr"ufung der Eingabedatei:
  -c, --check                 alle "Uberpr"ufungen durchf"uhren; umfasst:
                                --check-format, --check-header,
                                --check-domain
      --check-format          sprachabh"angige Formatelemente "uberpr"ufen
      --check-header          Existenz und Inhalt des Kopfeintrags
                                verifizieren
      --check-domain          auf Konflikte zwischen Bereich-Anweisung und
                                die Option --output-file hin "uberpr"ufen
  -C, --check-compatibility   sicherstellen, dass sich GNU msgfmt wie X/Open
                                msgfmt verh"alt
      --check-accelerators[=ZEICHEN]
                                Vorhandensein des Tastatur-Accelerators
                                 f"ur Men"ueintr"age "uberpr"ufen
  -f, --use-fuzzy             auch ungenaue "Ubersetzungen (>>fuzzy<<)
                                verwenden

Details zur Ausgabe:
      --no-convert            Meldungen nicht in UTF-8 umkodieren
      --no-redundancy         Formatzeichenketten-Anweisungsmakros
                                <inttypes.h> gem"ass ISO C 99 nicht
                                vorexpandieren
  -a, --alignment=ZAHL        Ausgaben an ZAHL (Vorgabe: 1) Bytegrenze
                               ausrichten
      --endianness=BYTEREIHENFOLGE
                              32-Bit-Zahlen in der angegebenen                                Bytereihenfolge (>>big<< oder >>little<<, Standard
                                von Plattform abh"angig)
      --no-hash               Bin"ardatei soll keine Hash-Tabelle enthalten

Informative Ausgabe:
  -h, --help                   diese Hilfe anzeigen und beenden
  -V, --version                Versionsnummer anzeigen und beenden
      --statistics            Statistik zu den "Ubersetzungen anzeigen
  -v, --verbose               mehr Hinweise w"ahrend der Verarbeitung geben

Melden Sie Fehler im Fehlererfassungssystem auf <https://savannah.gnu.org/projects/gettext>
oder per E-Mail an <[email protected]> (jeweils auf Englisch).

[JoergAtGithub]

The problem seems to be the XML output format, which reqiures a .its rules file, which is not there, with binary output it works:

D:\vcpkg\buildtrees\shared-mime-info\x64-windows-dbg> & "D:/vcpkg/packages/gettext_x64-windows/tools/gettext/bin/msgfmt.exe" -v  --template "D:/vcpkg/buildtrees/shared-mime-info/src/2.4-b756cadafd.clean/data/freedesktop.org.xml.in" -o "D:/vcpkg/buildtrees/shared-mime-info/x64-windows-dbg/data/freedesktop.org.mo" "D:\vcpkg\buildtrees\shared-mime-info\src\2.4-b756cadafd.clean\po\de.po"
888 "ubersetzte Meldungen.

https://www.gnu.org/software/gettext/manual/html_node/ITS-Rules.html

There is https://gitlab.freedesktop.org/xdg/shared-mime-info/-/blob/master/data/its/shared-mime-info.its?ref_type=heads but this is not accepted by gettext, independent of path and filename.

[daschuer]

But there is an "its" file in the "its" folder.

My conclusion is that there must be an issue with the call or the msgfmt.exe windows build, because everything is in shape on macOS using the same files.

[JoergAtGithub]

The documentation of gettext says:
If your XML file is not of one of the types covered by the system-wide installed *.its files, you need a particular *.its file and a corresponding *.loc file (see Preparing Rules for XML Internationalization). Furthermore you need to store these files in a directory parent_dir/its/ and set the environment variable GETTEXTDATADIRS to include parent_dir. More generally, the value of GETTEXTDATADIRS should be a colon-separated list of directory names.

So I guess that on macOS there is a system-wide installed *.its file for plain XML

[daschuer]

The required files are in the its folder in the source tree. I have verified this by on Linux by removing them from the system and replace the source tree files by empty files. Build fails. Once they are populated again build succeeds. My conclusion is that there is a bug in msgfmt.exe for windows.

In my latest attempt I have tried to use undocumented GETTEXTDATADIR (Without S) to override the directory with a relative path to get around a possible drive letter issue, but it still fails. See:
daschuer/vcpkg@1a84162

Internally it uses "/" to build pathes. So passing other pathes with "/" seems to be OK.

Are you able to single step through msgfmt.exe?

[JoergAtGithub]

I'm pretty sure now, that the msgfmt.exe from VCPKG does not work in XML mode, and also that it never worked in former versions of the VCPKG port.
This is because I tested the shared-mime-info port with various gettext builds. I tried older versions from VCPKG and they showed the same failure. Than I downloaded Windows binaries 0.24 and 0.22.5 from https://github.com/mlocati/gettext-iconv-windows/tree/main extracted the zip to D:\gettext0.22.5a-iconv1.17-shared-64, removed the gettext dependecy from th vcpkg.json and added
vcpkg_add_to_path("D:/gettext0.22.5a-iconv1.17-shared-64/bin") to the portfile.cmake. Both binaries worked out of the box. As well for the manual called msgfmt.exe as for .\vcpkg.exe install shared-mime-info:

PS D:\vcpkg> .\vcpkg.exe remove shared-mime-info
The following packages will be removed:
    shared-mime-info:x64-windows
Removing 1/1 shared-mime-info:x64-windows
PS D:\vcpkg> .\vcpkg.exe install shared-mime-info --no-binarycaching
Computing installation plan...
The following packages will be built and installed:
    shared-mime-info:[email protected]
Detecting compiler hash for triplet x64-windows...
Compiler found: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.43.34808/bin/Hostx64/x64/cl.exe
Installing 1/1 shared-mime-info:[email protected]...
Building shared-mime-info:[email protected]...
-- Found Python version '3.12.7 at D:/vcpkg/downloads/tools/python/python-3.12.7-x64-1/python.exe'
-- Using meson: D:/vcpkg/downloads/tools/meson-1.6.1-6779de/meson.py
-- Using cached xdg-shared-mime-info-2.4.tar.gz
-- Cleaning sources at D:/vcpkg/buildtrees/shared-mime-info/src/2.4-b756cadafd.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source D:/vcpkg/downloads/xdg-shared-mime-info-2.4.tar.gz
-- Using source at D:/vcpkg/buildtrees/shared-mime-info/src/2.4-b756cadafd.clean
-- Using cached msys2-mingw-w64-x86_64-pkgconf-1~2.3.0-1-any.pkg.tar.zst
-- Using cached msys2-msys2-runtime-3.5.4-2-x86_64.pkg.tar.zst
-- Using msys root at D:/vcpkg/downloads/tools/msys2/21caed2f81ec917b
-- Configuring x64-windows-dbg
-- Getting CMake variables for x64-windows
-- Configuring x64-windows-dbg done
-- Configuring x64-windows-rel
-- Configuring x64-windows-rel done
-- Package x64-windows-dbg
-- Package x64-windows-rel
-- Fixing pkgconfig file: D:/vcpkg/packages/shared-mime-info_x64-windows/share/pkgconfig/shared-mime-info.pc
-- Fixing pkgconfig file: D:/vcpkg/packages/shared-mime-info_x64-windows/debug/share/pkgconfig/shared-mime-info.pc
-- Installing: D:/vcpkg/packages/shared-mime-info_x64-windows/share/shared-mime-info/copyright
-- Performing post-build validation
Elapsed time to handle shared-mime-info:x64-windows: 12 s
shared-mime-info:x64-windows package ABI: 0d67c5bafcf35d7610cb9eca77438be218fec0f32c2be3deb7fd233f6842e547
Total install time: 12 s
shared-mime-info provides pkg-config modules:

  # Freedesktop common MIME database
  shared-mime-info

PS D:\vcpkg>

[JoergAtGithub]

As we only use the standalone tool msgfmt.exe from gettext and not the library itself, I recommend, that we download the zip from https://github.com/mlocati/gettext-iconv-windows/tree/main in the mixxxdj/VCPKG gitHub action and call msgfmt.exe from this instead of the VCPKG version until the upstream bug #44299 is fixed.

[daschuer]

Great finding. Do you have an idea what causes the issue?

[JoergAtGithub]

This VCPKG port is very complicated. The working workflow setup at https://github.com/mlocati/gettext-iconv-windows/blob/main/.github/workflows/build.yml is complicated too. Not easy to compare. But replacing msgfmt.exe and the DLLs next to it in VCPKGs .\installed\x64-windows\tools\gettext\bin folder fixes the issue. It's something specific to the tools and XML.

[daschuer]

Got it working:
https://github.com/daschuer/vcpkg/actions/runs/13800724512
The "dirent" dependency was missing.

The next step is to instruct QT to use the shared-mime-info port. Do you have an idea how this can work?
I think embedd it as resource is reasonable. Ideas?

[daschuer]

Here is the full build:
https://github.com/daschuer/vcpkg/actions/runs/13805940455

[daschuer]

Luckily they have implemented a backdoor
qt/qtbase@ed2f80b