qml-autoreqprov - Automatic generation of Provides and Requires for QML imports ====== QML is a user interface specification and programming language. Each QML file can have import statements at the top, which can either reference files or directories directly or modules by identifier and version (major.minor). If any of those import statements can't be satisfied, loading fails. For imports which are provided by other packages, this maps naturally to Requires statements in RPM packages, which enforce that all imports are satisfied on package installation. Direct file/directory imports are ignored here, as those are usually contained within a single package and thus not relevant for inter-package dependencies. TLDR for packagers ------------------ Packages with system-wide QML modules get Provides like `qt5qmlimport(QtQuick.Controls.2) = 15` automatically. Imports in .qml files map to RPM requires like `qt5qmlimport(QtQuick.Controls.2) >= 15`. This can be disabled with `%global %_disable_qml_requires 1` in .spec files. It's important to check that all dependendencies are fulfilled, as in some cases a needed `qmlimport` Provides is missing. See the "Internal and private exports" section for how to deal with that. How the QML engine imports modules ---------------------------------- For each module import, the QML engine looks into the import cache to find any suitable export with identical identifier and major version and same/higher minor version. If there is no match, it goes through the QML import path (with a system-wide default) in order with the module identifier and version appended in various ways and reads the qmldir file inside. If the qmldir file mentions plugins, those are loaded and it can register the exported types. If the import can't be satisfied (due to a version mismatch), the search continues. Mapping to RPM capabilities --------------------------- The goal is that all imports for .qml files within a package are satisfied by RPM dependencies of that package. This is achieved by adding Provides to packages which satisfy a specific module import and Requires to the packages with QML files inside. The capability has to include both the full module identifier and version. As modules with a different major version are pretty much independent, the major version is part of the capabilities' name and the minor version is used as the capabilities' version. Additionally, the system import paths are specific to a Qt major version, this is also included. The end result are QML modules providing capabilities like: `Provides: qt5qmlimport(QtQuick.Controls.2) = 15` On the import side, packages with QML files get requirements like: `Requires: qt5qmlimport(QtQuick.Controls.2) >= 13` for a statement like `import QtQuick.Controls 2.13`. The `>=` is there so that modules with a higher minor version satisfy it as well. With Qt 6, unversioned QML import statements got introduced which import the highest available major version (which IMO does not make sense...). Representing those in RPM requires using separate unversioned capabilities alongside the versioned ones: `Provides: qt6qmlimport(QtQuick.Controls)` `Requires: qt6qmlimport(QtQuick.Controls)` Generating Requires from .qml files ----------------------------------- As can be seen, mapping QML import statements to RPM requires is straightforward, and even made easier by using the `qmlimportscanner` tool from qtdeclarative combined with `jq` to convert its JSON output with some filtering directly into the capabilities format for RPM. There is one tricky part though: QML files are (intentionally) not tied to any specific version of Qt, while QML modules are (though the Qt version specific import paths and binary plugins). So when generating the list of required imports, those are tied to a specific Qt major version. Currently this is automatically detected by looking at which versions of qmlimportscanner are installed. If multiple versions are found, the requires scanner aborts. This can be overwritten by setting a variable in the .spec file (unfortunately not possible per subpackage): `%global __qml_requires_opts --qtver 5` Currently, only .qml files directly part of the package are handled, so if those are part of a resources file embedded into an executable or library, they will not be read. Making this possible needs more research and effort. Generating Provides from qmldir files ------------------------------------- Every module installed into the QML import path contains a `qmldir` file with metainformation. They usually contain a `module` line which specifies the identifier (those which don't are ignored) and a list of exports and plugins. Just the module identifier is not enough to generate the capability, major and corresponding minor version(s) are also needed. For each major version, only the highest minor version is stored, as it also satisfies imports with a lower minor version. Handling direct exports are easy, as they mention the major and minor version directly, which combined with the Qt version (derived from the location the qmldir file is installed to) results in a capability. For plugins, it's not as easy though, those actually have to be loaded to get their registrations. While `qtdeclarative` provides a tool called `qmlplugindump`, which lists all exported types in a QML format, it uses the QML engine for loading plugins, which requires the module identifier and a version. In addition to that, it just doesn't work at all sometimes and does not provide all necessary information (like pure module exports, which just bump the available minor version without exporting any new type revisions). To get a list of all versioned exports made by a plugin, a new tool called `qmlpluginexports` specifically for that was written. It uses private API to load a plugin without specifying a version and then iterates through all known types to get their versions. It also handles lazy registration using QQmlModuleRegistration and qmlRegisterModule by implementing the underlying modules, which overrides the symbols in the Qt libraries (symbols in executables have higher priority than public symbols from shared objects) to store the information and then forwarding the call to the Qt library. As loading a plugin also triggers loading of all dependencies, it's possible that those register their own exports as well. So only exports including the module identifier from the `qmldir` file are used and others are ignored. For instance, the plugin for `QtQuick.Controls.2` also registers `QtQuick.Controls.impl.2`. Internal and private exports ---------------------------- The automatic generation of import requirements uses every module import in installed .qml files. In some cases, those imports are not for system-wide modules, but for imports provided by code in shared libraries and executables, usable only in .qml files loaded by those. Provides for those can't be generated, but the requirement will be, which makes the package unsatisfiable due to missing dependencies. How to deal with that depends on the export itself. RPM dependencies are inter-package, so if an export is only used within a package (for instance, an application with its UI), it's fine to filter it out from requires and provides like this: ``` %global __requires_exclude (org.kde.private.kcms.kwin.effects)|(org.kde.kcms.kwinrules) %global __provides_exclude (org.kde.private.kcms.kwin.effects)|(org.kde.kcms.kwinrules) ``` Truly private exports are usually not found by the generation of provided capabilities, so the latter is normally not necessary. The opposite case is when an application or library registers exports for use by other packages (e.g. Plasma applets) which aren't available as system wide QML imports. Those shouldn't simply be filtered out, as other packages actually make use of it. Instead, the capability has to be provided manually, e.g. ``` Provides: qt5qmlimport(org.kde.plasma.configuration.2) = 0 Provides: qt5qmlimport(org.kde.plasma.plasmoid.2) = 0 ``` qtdeclarative-imports-provides ------------------------------ The qml-autoreqprov scripts need qmlimportscanner from qtdeclarative to generate Requires and for `qmldir` files with `plugin` lines `qmlpluginexports` is required. The latter needs qtdeclarative to be built. This is a problem, because qtdeclarative provides important exports, which have to be available as RPM capabilities as well. Making qml-autoreqprov and its dependencies available during build of qtdeclarative would cause a build cycle. To work around that, a "stub" package installs qtdeclarative during build and runs the provides generator for each relevant qtdeclarative package, to map the exports to the corresponding package. TODO ---- Is special treatment for baselibs.conf needed? How do .qmlc files relate to this? RPM doesn't seem to merge "foo >= 10" and "foo >= 11", so there are redundant requirements generated. As the generator is run for each file separately, it's not directly possible to work around that.
