Digikam/Tutorials/Setup of digiKam for Windows compatibility

    From KDE UserBase Wiki
    Revision as of 12:00, 13 March 2023 by Fotograaf (talk | contribs) (→‎Step 2 - Fix which metadata is written into files: To better integrate with Darktable, you should also fix the Xmp.lr:hierarchicalSubject exactly like the Xmp.dc.subject)
    (diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

    Introduction

    It is safe to say that digiKam (7.4) is the most powerful image metadata management photo gallery software. Unless you use command-line tools, there is no more universal program to read and write any metadata that you can imagine than digiKam (it even allows you to define new metadata namespaces to read and write data to, in case something is missing). As it is free for everyone, open-source, writes metadata back to files and runs on multiple operating systems – digiKam is truly future-proof tool to manage all your image and video metadata (people face tags, descriptive tags, location, captions, star ratings and more).

    Unfortunately, as digiKam tries to be (and can be) cross-compatible with everything at once, the default digiKam settings cannot satisfy everyone. This tutorial provides a suggestion on how to best configure digiKam for new users or people who have managed their photos using any of the Microsoft software, including Windows built-in metadata tools.

    "But I do not use Windows!"

    Even if you never use Windows yourself, the main reasons to be Microsoft compatible are because Microsoft uses actually quite good in-file metadata storage format and there is a great chance that someone you know will not use digiKam, but due to Microsoft compatibility will still have access to the embedded metadata. (Even simple Windows Search will find photos by Microsoft compatible tags.)


    The recommended configuration

    Of course, you can use digiKam with default settings, but here are the reasons why you may want to follow this tutorial and change the Metadata configuration.

    The Advantages:

    • Less garbage in the files – the default configuration writes the same metadata multiple times under many metadata fields. While it can be good for ultimate compatibility outside digiKam, it practically guarantees that any other software will read and modify only part of those fields. As a result, if another software is involved, your image will carry both updated metadata as well as old metadata that was originally created by digiKam but not recognized by the other software that did the latest edit (for example OneDrive, Lightroom, etc.)
    • More robust – This configuration keeps your digiKam tagged images cross-compatible with Microsoft OneDrive and Windows so you can correctly use and edit metadata even on Windows computers where digiKam is not installed (for example if you send files to your relatives). In addition a lot of major photo metadata software, including online services will correctly read Microsoft compatible tags, because it is one of the most common image metadata formats embedded into image files.
    • This configuration will correctly preserve hierarchical tag structures, without creating lots of duplicate tags (when the same information is written in many metadata fields, other software, for example, Microsoft Windows may interpret the metadata from default digiKam configuration as multiple separate tags)

    The Drawbacks:

    • By disabling Read and Write to less common metadata fields, you are also stopping digiKam from correctly importing such metadata. But it is not very likely to find files that exclusively use the less common metadata fields which we are disabling – usually, the same information will be duplicated in another enabled field. (Plus it is possible to individually disable writing certain metadata fields while still reading them by un-checking “Unify read and write” in the Advanced Metadata settings).

    Note

    You do NOT have to change default digiKam settings to import all your data if you are switching to digiKam from Windows Photo Gallery (WLPG) or Microsoft OneDrive. BUT without these changes, you will get a mess with cross-compatibility in case you will later view or edit your digiKam tagged files in any Microsoft software.


    What you need to know

    1. You have to re-check the Metadata settings every time you update digiKam to a new version, before you resume using digiKam as usual. Currently (digiKam 7.4) you do need to repeat the manual Metadata setting changes after each digiKam update.
    2. There may be bugs, special exceptions or just format changes that could break cross-compatibility that we try to achieve (from digiKam to Microsoft programs). The good news is that such metadata issues are almost always resolvable either by tools built in into digiKam or in the worst cases by other programs, where ExifTool is usually the ultimate solution to just about any metadata problem (see one old example here).
    3. If you are migrating from some Microsoft metadata editing software, there is a dedicated additional tutorial to help specifically WLPG, OneDrive and other Microsoft software users - how-to Switch from Microsoft OneDrive or WLPG to digiKam.
    4. The statement “This configuration keeps your digiKam tagged images cross-compatible with Microsoft” does include some limitations:
      • If you save people face tags in digiKam it will automatically create descriptive tags for each person in a hierarchy “People/<face tag name>”. This is not the same as WLPG behavior, but still shows-up nicely in any Microsoft software.
      • If you save people face tags in digiKam it will also create "MWG regions" which follow the main standard to save face tags and performs the same function as Microsoft People Tags. Microsoft software will always display correctly people tags saved in digiKam (and digiKam always reads Microsoft created People Tags), but you can not edit digiKam created face tags in Microsoft software!
      • Trully cross-compatible changes (edit in digKam or Microsoft at any time) are:
        • selecting/deselecting descriptive tags,
        • changing Title in digiKam == changing Caption in WLPG,
        • changing star rating.
    5. If you encounter some problems while following this tutorial – please add a “Warning” box to the tutorial to warn everyone about a potential issue and consider filing a bug report at https://bugs.kde.org/describecomponents.cgi?product=digikam

    Never edit digiKam saved Face Tags in Microsoft programs!

    digiKam saves faces as MWG and Microsoft regions, but Microsoft will only edit "Microsoft regions". Because digiKam prioritizes "MWG regions", any changes you make in Microsoft software will be DISCARDED. (But you can always use Microsoft software to tag a new image that did not previously have any face tags - digiKam always correctly reads not-conflicting people tags from Microsoft)"


    Change the settings

    Here are all the steps to correctly set up your digiKam for the recommended configuration.

    Updated digiKam? - Re-check the settings!

    Do not forget to re-check the Metadata settings every time you update your digiKam! Some settings can be reset to defaults.


    Step 1 – Configure the basics

    Before you begin with the details it is good to have the basics right:

    1. Check the folder and file Properties of your photo collection using a basic file browser. Check that:
      1. Either all your image files can be modified (“NOT Read-Only”) – necessary if you want to store all your tags and metadata inside the image files (this is the typical approach for the best metadata preservation).
      2. Or in case you plan to only use sidecar files for all your media (more information about “Sidecar” files further below), then you may choose to set your files to “Read-Only” (but not folders).
    2. Select the right database type for digiKam. According to the documentation, all database options are good but have different pros and cons:
      1. SQLite is effortless to configure, but if you have more than about 100 000 files in your collection it can become a performance bottleneck. Note that you can always migrate to another database type later, so no need to worry about this.
      2. MySQL Internal requires more work to set up, but should pay off in terms of performance in large photo collections. You will actually need to follow the 1st provided “download” link and install “MariaDB Server” (you can leave the root password empty for MariaDB and Disable Networking) and then click “Find” and locate the 1st required .exe file in the MariaDB installation folder.
      3. MySQL Server is only used if multiple computers with digiKam will access the same shared photo library. Again, you can migrate to this type of database at a later point.

    Finally when you choose which folders with images and videos digiKam will manage, consider that:

    1. You should NOT add your true main collection to digiKam before you complete "Step 2" and decide if you want to Set up digiKam to ignore RAW files. But you can add some test folder(s) to initialize digiKam and run some tests.
    2. Consider ticking the “Monitor the albums for external changes” setting. It should make digiKam more intuitive to use, but could also impact performance.

    Note - Re-initialize database

    When you are done with your testing and want to switch to the true photo collection you can always delete all digiKam database files from the folder where it is stored. However, when deleting the digKam database files you also reset the settings to ignore RAW files (this setting needs to be re-applied).


    Beware loss of metadata at later times

    Later, when you already have been working on your true photo collection with digiKam, you better would not re-initialize the digiKam database the hard way as decribed above. Some of the metadata, in particular that related to face recognition, is not stored in the images or .xmp sidecar files, but only in the digiKam database. Examples are the information on ignored face regions or the flag whether an image already has been scanned for faces.


    Step 2 - Fix which metadata is written into files

    The main Metadata settings are under SettingsConfigure digiKamMetadataAdvanced. Here we will set DigiKam to not-write to various unnecessary Metadata fields. (For simplicity we are also disabling reading of those fields, but you are welcome to disable only the writing by un-checking the “Unify read and write”.

    • Start with Caption (aka Comments): here deselect all namespaces except for “Xmp.dc.description” and “Exif.Image.ImageDescription”. AND using Move Up button raise the “Exif.Image.ImageDescription” field to the 1st priority (this is important to stop WLPG from overwriting your Caption by duplicating Title text). Remember that Microsoft WLPG confuses Title & Caption, so it is safer to only use either “Title” or “Caption” but not both at the same time. See “What tags to use” for more info.
    • Under Color label – leave only “Xmp.xmp.Label” (it is enough)
    • Under Rating – leave only “Xmp.xmp.Rating” and “Xmp.MicrosoftPhoto.Rating”.
    • Under Tags we have a more complex case (see screenshot):

    The most important setting

    If you will make only one change to the settings, then fix the Xmp.dc.subject under Tags . This is crucial for preserving tag hierarchy between digiKam, Windows and other software.
      • Here we fix the main Descriptive Tag cross-compatibility issue with Windows, WLPG, OneDrive and other Microsoft software [for reference see this and this]:
        1. Disable the default: "Xmp.dc.subject" namespace (it cannot be modified)
        2. Add new namespace and configure it as:
    Name: Xmp.dc.subject (substitute for the existing one)
    Spec_options: TAG_XMPBAG
    Separator: /
    Set Tags Path: YES <== this is the FIX to the main problem!

    Do the same thing for Xmp.lr.hierarchicalSubject - disable the original and create a new entry. Follow the steps like Xmp.dc.subject above. For 'name' enter 'Xmp.lr.hierarchicalSubject'. Reason is that Darktable uses these, and if Digikam does not read/write them too, they may get out of sync.

      • Leave enabled "Xmp.Micosoft Photo.LastKeywordXMP" namespace (as it is used by WLPG)
      • Then disable all other namespaces (unless you have a specific reason to keep some of them enabled)
    • Finally under Title – leave only “Xmp.dc.title”. Remember that due to Title & Caption confusion in Microsoft WLPG, it is safer to only use either “Title” or “Caption” but not both at the same time.

    Step 3 - Other important metadata settings

    Here we check that Metadata is actually written into files, and not just left in the digiKam database.

    • Under SettingsConfigure digiKamMetadataBehavior make sure that all Information is written into the Metadata.
    • There is also “Use lazy synchronization” option which I recommend to enable to improve performance and avoid re-writing each file multiple times for every little edit that you make. It also can help to detect accidental metadata changes.
    • Under Sidecars the basic idea is to allow digiKam to write sidecar files, but only for media files that can not contain metadata directly (most useful for video files or any file that is set to read-only). According to the documentation, the option “Read from sidecar files” means "digiKam will only read the sidecar while ignoring the embedded metadata.”

    Note

    Some conservative users actually prefer to configure digiKam to exclusively store metadata in .xmp sidecar files and never make any changes to the image files that normally can contain XMP metadata. Sidecar approach is the safest against data corruption (it is even possible to delete .xmp sidecars using file manager and instantly erase all metadata changes), but it is also the least cross-compatible approach and makes the metadata only visible in correctly configured digiKam. On the other hand, when metadata is separate from an image file, even a naïve user is less likely to accidentally spread sensitive metadata on the Internet, so it is the best approach for privacy.


    • Finally for Rotation it is preferable to choose lossless data rotation instead of using the orientation metadata flag. (The main reason is Microsoft – MS still does not respect the orientation flags and it causes issues such as your pictures may appear on a side when viewed in Windows as well as problems with rotated face tag regions)

    Fix Tree view

    This setting is not essential, but if you do not have ticks in the ViewInclude Album Sub-Tree and ViewInclude Tag Sub-Tree, then the tree structures in the left panel will not behave as most users would expect – highly recommended to make this change if you are setting up digiKam for a new user.

    Unfortunately in digiKam (7.4) the Albums tree still has a limitation that only one tree entry can be selected at the same time - https://bugs.kde.org/show_bug.cgi?id=269340. (it is possible to select multiple tree entries in the Tags tree, but not in the Albums tree.)

    Personalize appearance

    In addition I would suggest to experiment with settings for digiKam's

    • appearance in SettingsThemes,
    • thumbnails in SettingsConfigure digiKamViewsIcons and
    • sorting in ViewSort … to get to an optimal look and feel for digiKam.
    Example of my current appearance settings for thumbnails.

    RAW file handling

    If you have “digital negatives” (RAW files) together with the usual .JPG image files in your collection there is always a concern about how to best handle these files. Here are some of the common options:

    Show All
    Most software by default just shows both JPG and RAW files as equals. This is the worst option because it makes it very cumbersome to view, sort or tag your pictures. Remember that when you do need to see both JPG and RAW files together, there are always normal file browsers that are simple and reliable.
    Grouping RAWs under JPGs
    More advanced software offers to automatically recognize which RAW files belong to which JPGs and apply all operations on the “groups” of files. This is a valid approach IF it is easy to use and robust. DigiKam can do grouping, but there are a few things to consider if you choose to use it:
    • It is not possible to permanently enable grouping in digiKam (7.4) – grouping must be manually enabled every time by selecting all photos that should be grouped, then right-clicking one thumbnail and selecting GroupGroup Selected By Filename.

    Important

    the Group function is hidden, not visible unless you select more than one photo!
    • As grouping is not always enabled, it is difficult to explain how to use grouping to infrequent users. And even geeks will not appreciate the required extra manual steps.
    • Finally the grouping by Filename is very simplistic in digiKam 7.4. If you make any changes to the filename of the JPG file, for example, add “_cropped” or “_altColor” to the filename, it will no longer be grouped with a corresponding RAW file (so you will again see that RAW file during slideshow and will need to tag it separately.
    Ignoring RAWs
    As digiKam is not a true competitor to a dedicated RAW processor, I prefer to use digiKam only to manage metadata in the photo collection. Then it is optimal to block digiKam from reading RAW files and only deal with JPGs and videos. It means:
    • Simplicity – even inexperienced users will not get distracted by the RAW format duplicates of every image. And the limitations in grouping of RAW+JPG in digiKam are not a problem.
    • Perfect consistency – if you do not try to sync RAW file metadata with the JPGs then all potential metadata mismatch problems go away. (for example if you import RAW+JPG files where only JPGs have been tagged by another program)
    • As RAW files are ignored, they never get modified and there are no additional .xmp sidecar files when you make changes to the Metadata – which is good for backups and performance.
    • The downside – to move images between folders, you would have to use a generic file browser, because in “Ignore” mode digiKam will normally not do any operations with RAW files.
    • Also you will need to remove RAW files which are left after you delete unwanted JPGs. This is not just digiKam issue as I personally use multiple image viewers for sorting pictures which also leave RAW files behind. My preferred solution for leftover RAW files is this simple Python executable script that automatically removes all orphaned RAW files: "CleanUp_RAW_files.py"
    "Ignoring RAWs" configuration is very convenient to use if you understand what it does.
    If you... Show All Grouping RAWs under JPGs Ignoring RAWs
    Move complete album All files & folders are moved All files & folders are moved All files & folders are moved
    Move file by file All files can be moved All files can be moved ONLY "JPG" files will be moved
    Delete file by file All files can be deleted All files can be deleted ONLY "JPG" files will be deleted

    Set up digiKam to ignore RAW files

    My recommendation is to stop digiKam from reading RAW files and make any changes to RAW files in other software. The full list of RAW file extensions that digiKam can load is given in the documentation [link].

    To disable all RAW files (or any other file extensions) go to SettingsConfigure digiKamViewsMime Types and under “Image Files” add this list with all file extensions that need to be ignored by digiKam:

    -rw2 -crw -cr2 -nef -orf -raf -rwl -pef -ptx -x3f -dcr -kdc -dc2 -k25 -srf -arw -mrw -mdc -raw -dng -bay -erf -fff -mos -pxn –rdc

    What tags to use

    Even when all the metadata settings are correct, to ensure best cross-compatibility it is still important to use the tags in a compatible way. For example, Windows does not differentiate between “Title”, “Caption” or “Comments” so if you use all of these fields simultaneously, some Microsoft software can overwrite part of what you wrote and cause permanent metadata loss [source].

    Here are the strongly-recommended; safe-to-use metadata fields (notice, almost the same as in WLPG):

    • Descriptive Tags organized in tag hierarchies – tags are good, but tag hierarchy greatly enhances the usefulness of predefined tags. With the recommended settings, tag hierarchy is fully supported and cross-compatible with Microsoft tools.
    • Face Tags or people tags are excellent to specify who is who in the photo. Supported and useful. Note you can as well tag pets or other non-human points of interest using face regions.
    • Edit “Title” field – this field is effectively the primary filed for all Comments and Captions despite the name of the field. (In WLPG this data is editable as Caption. More details below.)
    • Secondary: “Caption” field can be used if you really have to. Microsoft software will show it as “Comment” in file properties, but for example WLPG cannot edit this field.
    • Star rating – fully supported and a more robust alternative to Colors or Flag labels.
    • GPS geotag data.
    • Names (Author) and Copyright under the Information – Rights in the right digiKam sidebar.
    • Plus you can rename the file, but be careful in case your image has a corresponding RAW file. After re-naming, RAW file should still match the beginning of the new filename.
    • Finally it is useful to keep meaningful folder names that can contain some basic “Caption” and start with a relevant date in “YYYY-MM-DD” format (to have some control over your files outside dedicated photo management software like digiKam).

    For more information how to actually efficiently apply tags in digiKam see “Tagging and Face Tags” tutorial.

    For all other Metadata fields and types – use at your own risk. Everything will work in digiKam, but may be hard to access for other software.

    Title, Captions and Comments confusion in Windows

    This paragraph explains in-depth how the recommended Metadata configuration affects Title, Caption and Comments. In short – it results in a nice match with Windows 10, but if you are migrating from Microsoft WLPG or OneDrive, then you may want to read on.

    The screenshot and table below shows how Metadata will appear in:

    1. Windows 10
    2. digiKam 7.4
    3. obsolete Microsoft WLPG.
    Where your information appears in different environments using recommended Metadata settings
    1 in Windows 2 in digiKam 3 in Microsoft WLPG
    Title saved in digiKam is Title Title Caption
    Caption saved in digiKam is Comments & Subject Caption NOT visible
    Caption saved in WLPG is Title & Subject Title (*) Caption

    As you can see from the table:

    • Title” is the only field that reliably appears in all 3 compared environments
    • Caption” in digiKam will be “Comment” in Windows, but is not visible in the old WLPG. And as long as digiKam uses recommended Metadata configuration, this data is safe from corruption by Microsoft WLPG.
    • The “Subject” property in Windows is unreliable and unnecessary. Depending on which program saves the data last it will contain duplicate of digiKam’s “Title” or “Caption”. This field is linked to “Xmp.dc.description” (and possibly also other namespaces).
    • (*) special case – if you create Caption in WLPG (or OneDrive?), on import to digiKam it will be duplicated in digiKam as Title and also as Caption. As the relevant information is just duplicated, it makes almost no practical difference. (If you really want to avoid this, you can disable reading of the “Xmp.dc.description” namespace under Advanced settings of “Caption”, but it is important to keep writing to the “Xmp.dc.description” namespace to be able to overwrite any obsolete “Xmp.dc.description” information.)

    Default configuration warning

    If you do Not use recommended Metadata configuration in digiKam, any time someone edits “Caption” in WLPG, it would overwrite Both “Title” and “Caption” in digiKam, which can lead to the “Caption” metadata loss. (OneDrive was not tested, but could have the same risk.)


    Future improvement suggestions

    No doubt we would like faster, more robust and intuitive way to configure digiKam for an average user. Here are some suggestions on how to make digiKam configuration better:

    1. Macro – as a temporary measure, someone could write a script to modify digiKam settings using one executable file. The advantage would be that anyone could read and modify what the script does and therefore make a custom one-click setup script for their unique configuration.
    2. Configuration presets – just as many other programs (for example see Blender keymap presets), digiKam could offer users a choice which Metadata configuration preset to use already during initial configuration. This way there could be multiple ready-made profiles depending on the use case; depending on which other software user migrates from or uses in parallel.
    3. Simplified digiKam – just as there is ShowFoto as a standalone part of digiKam for image editing, there could be a simplified gallery viewer and manager suitable for non-geeks so that everyone could use the metadata features without complex setup tutorials and editing of configurations.

    The good news is – digiKam is open source, so anyone can pick up a challenge and make the improvements we all need. Meanwhile digiKam works already, so happy tagging after the setup!


    About this tutorial

    This tutorial was originally written for digiKam 7.4 in 2021-12. You are very welcome to edit, update and Upgrade this tutorial page to make it better and keep it up to date. No attribution or copyrights on this page (all illustrations are original as well)!