Nepomuk/nl: Difference between revisions

    From KDE UserBase Wiki
    (Updating to match new version of source page)
    (Updating to match new version of source page)
     
    (3 intermediate revisions by the same user not shown)
    Line 1: Line 1:
    <languages />  
    <languages />  


    =Nepomuk=
    =Semantic Search=


    Het doel van deze pagina is niet ieder detail van de technologie van Nepomuk duidelijk te maken, maar een kort overzicht met voorbeelden te geven en de visie erachter te delen. Daarnaast worden er verwijzingen gegeven naar relevante informatie op het web.
    The aim of this page is not to fully explain KDE's Semantic Search technology and every detail, but to give a short overview, some examples, share the vision behind it and link to relevant information on the web.
     
    == Baloo is the next generation of semantic search ==
     
    From KDE Applications 4.13 onwards, the '[https://community.kde.org/Baloo Baloo]' file indexing and file search framework replaces Nepomuk. Read [http://dot.kde.org/2014/02/24/kdes-next-generation-semantic-search details on the changes for Applications 4.13 here]. Semantic Search no longer uses a single, big database, but separate, specialized databases for each type of data. The new search databases are in <tt>$HOME/.local/share/baloo</tt>. If you upgraded to KDE Applications 4.13 from an earlier KDE release, you can delete <tt>$KDEHOME/share/apps/nepomuk</tt>.


    == Korte uitleg ==
    == Korte uitleg ==


    Zoals in de [[Special:myLanguage/Glossary#Nepomuk|Woordenlijst]] wordt aangegeven, gaat Nepomuk over het classificeren, de organisatie en presentatie van gegevens.
    As the [[Special:myLanguage/Glossary#Nepomuk|Glossary]] mentions, Semantic Search is about classification, organisation and presentation of data. It is not an application, but a component which can be used by developers within applications.
    Het is geen toepassing, maar een component die door ontwikkelaars worden gebruikt binnen toepassingen.


    <span class="mw-translate-fuzzy">
    === Try out in Dolphin ===
    === Proberen ===
    For example, the [[Special:myLanguage/Dolphin|Dolphin]] file manager makes use of Search. In KDE Applications versions prior to 4.13 Semantic Search must be enabled from <menuchoice>System settings -> Desktop search</menuchoice>. The information sidebar of Dolphin (<menuchoice>Control -> Panels -> Information</menuchoice>, or press <keycap>F11</keycap>) presents information extracted by Search about the selected file, and also allows you to assign tags, ratings and comments to files. This information is then stored and indexed by Search. You can then search for metadata using the navigation bar in Dolphin. Click to <menuchoice>Find</menuchoice>, or press <keycap>Ctrl+F</keycap>, and search for file names or file contents.
    [[Special:myLanguage/Dolphin|Dolphin]] is een van de applicaties die gebruik maakt van Nepomuk. Om het te laten werken moeten Nepomuk and Strigi geactiveerd worden in <menuchoice>System settings -> Desktop search</menuchoice>. The informatie-zijbalk van Dolphin laat u tags, waarderingen en opmerkingen aan bestanden toevoegen. Deze informatie wordt opgeslagen in Nepomuk en geïndexeerd door Strigi. U kunt dan zoeken naar metadata met de navigatiebalk in Dolphin. Schrijf "nepomuksearch:/" gevolgd door zoektermen.
    </span>


    <!-- info about nepomuksearch:/ deleted per its developer: "Actually nepomuksearch:/ is an internal thing and should not be entered by the user." its developer -->
    <!-- info about nepomuksearch:/ deleted per its developer: "Actually nepomuksearch:/ is an internal thing and should not be entered by the user." its developer -->
    Line 20: Line 21:
    ==Functionaliteiten==
    ==Functionaliteiten==


    Nepomuk biedt verschillende 'lagen' van functionaliteit aan applicaties. De eerste en meest eenvoudige hiervan is handmatig taggen, waarderingen commentaar toevoegen, zoals gebruikt in Dolphin. Dit helpt u om uw bestanden sneller te vinden, maar is ook veel werk.
    KDE's Semantic Search offers several 'layers' of functionality to applications. The first and most simple of those is manual tagging, rating and commenting, as used in Dolphin. This helps you to find your files faster, but is also a lot of work.


    <span class="mw-translate-fuzzy">
    To make finding files containing text easier, Search offers a second functionality: indexing the text of files. You can find files by entering some words which you know are in there, or just (part of) their title.
    Om het vinden van bestanden met tekst te vergemakkelijken, biedt Nepomuk een tweede functionaliteit: het indexeren van de tekst in bestanden. Hiervoor maakt het gebruik van een technologie genaamd [[Special:myLanguage/Glossary#Strigi|Strigi]]. U kunt nu ook bestanden te vinden door het invoeren van een aantal woorden waarvan u weet dat ze er in staan, of gewoon (een deel van) hun titel.
    </span>


    De derde laag is zeer complex, en de reden waarom Nepomuk werd opgevat als een onderzoeksproject van verschillende bedrijven en universiteiten in de Europese Unie. Dit is waar u moeilijke woorden als 'semantische desktop' en 'ontologieën' tegenkomt. Kort samengevat gaat het over context en relaties.
    The third layer is a very complex one, and the reason why the underlying technology, Nepomuk, was conceived as a research project of several companies and universities in the European union. This is where you will find difficult words like 'semantic desktop' and 'ontologies'. Basically, it is about context and relationships.


    === Indexing files ===
    === Indexing files ===


    Strigi does not index every file on the hard drive. Its default configuration in most Linux distributions excludes some common patterns for backup files and configuration directories, and it only indexes certain directories in your home folder. You can change this in <menuchoice>System Settings -> Desktop Search -> Desktop Query -> Customize index folders… -> Folders to index</menuchoice>.
    Search does not index every file on the hard drive. Its default configuration in most Linux distributions excludes some common patterns for backup files and configuration directories. You can change this in <menuchoice>System Settings -> Desktop Search</menuchoice>. Add folders to be excluded. If you want to turn off indexing of files entirely, just add your home folder there.
     
    Note that Strigi as of KDE 4.7 does not follow symbolic links ({{bug|208602}}). Up to KDE 4.9, even if the user selects folders under the symbolic link for indexing ({{bug|287593}}), content will not been indexed. The specific bug has been fixed. Thus, the user will be able to see the symbolic links, but she/he will not be allowed to mark any of them for indexing. (A symbolic link is a file that "points" to another file or directory; Dolphin displays symbolic links in ''italic''.) You must find the path to the actual directory (in '''Dolphin''', select the file, choose <menuchoice>Properties -> General -> Points to</menuchoice>) and tell Strigi to index that.


    In '''System Settings''' you can also control whether Strigi indexes files on removable media such as USB drives and CD-ROMs.
    In '''System Settings''' you can also control whether Search indexes files on removable media such as USB drives and CD-ROMs. In KDE Applications 4.13 this is not available, removable media are not indexed. Future versions are planned to reintroduce this functionality.


    == Voorbeelden ==
    == Voorbeelden ==


    Laat me proberen uit te leggen wat Nepomuk biedt door twee voorbeelden.
    Let me try to explain what Semantic Search offers using two examples. These features are not available fully yet - the base is there but application developers need to integrate this in their applications.


    === Verbindingen ===
    === Verbindingen ===
    Line 44: Line 41:
    Stel dat u twee weken geleden een foto hebt gekregen van een vriend. U heeft het ergens op uw computer opgeslagen. Maar hoe kan het bestand nu teruggevonden worden? Als u de locatie niet meer kunt herinneren, is dat een probleem.
    Stel dat u twee weken geleden een foto hebt gekregen van een vriend. U heeft het ergens op uw computer opgeslagen. Maar hoe kan het bestand nu teruggevonden worden? Als u de locatie niet meer kunt herinneren, is dat een probleem.


    Daarbij kan Nepomuk u helpen. U weet dat het bestand afkomstig was van die vriend, maar uw computer weet dat niet. Nepomuk kan die relatie wel herinneren. Zoeken naar de naam van die vriend zal daardoor de foto terugvinden!
    Now Semantic Search aims to help you. You know this file came from that friend of yours, your computer does not know. Search, however, can remember this relationship. Searching on the name of your friend will therefore pop up the photo!


    Een andere mogelijke relatie is tussen een web pagina waarvan u tekst gekopieerd hebt en het document waarin u die tekst geplakt heeft, of twee beelden die de dezelfde auto bevatten. Dergelijke relaties kunnen soms worden geëxtraheerd uit de bestanden zelf (u zou foto's kunnen analyseren en zien wie of wat er op staat) of worden geleverd door de betrokken applicaties (zoals in het bovenstaande e-mail voorbeeld). Dit deel van Nepomuk is nog steeds in ontwikkeling, en moet worden geïntegreerd in applicaties, dus u kunt verwachten dat het nog een paar jaar duurt voor het echt tot bloei komt.
    Another potential relationship is between a web page you copied text from and the document you pasted it into, or two images showing the same car. Such relations can sometimes be extracted from the files themselves (you could analyze photos and see who or what is on there) or supplied by the applications involved (as in the above email example). This part of Search is still under heavy development, and needs integration in applications, so you can expect it to take a few more years to really shine.


    All in all, this part of Nepomuk is about making search smart. Think about how Google tries to be smart with your searches: when you search for a hotel and a city name, it shows above the website results a google map showing hotels in the city you mentioned! It might even suggest a better name in case you made a spelling mistake. Google also tries to put the most relevant information on top of the list of results, using complex calculations on relationships (links) between websites. Nepomuk will be able to offer such smart results and order them on relevancy using relationship information.
    All in all, this part of Semantic Search is about making search smart. Think about how Google tries to be smart with your searches: when you search for a hotel and a city name, it shows above the website results a google map showing hotels in the city you mentioned! It might even suggest a better name in case you made a spelling mistake. Google also tries to put the most relevant information on top of the list of results, using complex calculations on relationships (links) between websites. Semantic Search will be able to offer such smart results and order them on relevancy using relationship information.


    ===Context===
    ===Context===


    These relationships can not only help you while searching for files, but also have an influence on applications and what information they present. Note that this way of using Nepomuk is still more a vision than reality! Many of the components are in place, but it is not yet integrated in applications and the desktop as a whole.
    These relationships can not only help you while searching for files, but also have an influence on applications and what information they present. Note that this way of using Search is still more a vision than reality! Many of the components are in place, but it is not yet integrated in applications and the desktop as a whole.


    So here an example of bringing context awareness to your desktop could help you work more efficiently.
    So here an example of bringing context awareness to your desktop could help you work more efficiently.
    Line 60: Line 57:
    Het zou leuk zijn als alles een beetje beter te ordenen is, niet?
    Het zou leuk zijn als alles een beetje beter te ordenen is, niet?


    Enter '[[Special:myLanguage/Glossary#Activities|activities]]'. These have been introduced in [[Special:myLanguage/Plasma|Plasma]], and currently offer different 'desktops'. They are a bit like virtual desktops, except that the desktop itself changes, not the set of applications. Different widgets, background, things like that. Of course, since KDE 4.3, each virtual desktop can have its own activity, bringing the two in sync.
    Enter '[[Special:myLanguage/Glossary#Activities|activities]]'. These have been introduced in [[Special:myLanguage/Plasma|Plasma]], and currently offer different 'desktops'. They are a bit like virtual desktops, except that the desktop itself changes, not the set of applications. Different widgets, background, things like that. Of course, since Plasma 4.3, each virtual desktop can have its own activity, bringing the two in sync.


    If applications and desktop were aware of activities, you could create an activity for each of the tasks you regularly work on. So if you often have to change a spreadsheet with prices, you create an activity for that: put a Folder View (or several) widget on the desktop, add a calculator and a todo-widget to keep track of what you still have to change. Maybe even an email folder widget showing the mails with questions regarding these prices spreadsheets!
    If applications and desktop were aware of activities, you could create an activity for each of the tasks you regularly work on. So if you often have to change a spreadsheet with prices, you create an activity for that: put a Folder View (or several) widget on the desktop, add a calculator and a todo-widget to keep track of what you still have to change. Maybe even an email folder widget showing the mails with questions regarding these prices spreadsheets!
    Line 72: Line 69:
    Of course, the above is mostly relevant to people working behind their computer in the office or at home. A gamer or a casual user would probably not use these activities much.
    Of course, the above is mostly relevant to people working behind their computer in the office or at home. A gamer or a casual user would probably not use these activities much.


    Note that the scenario described above is still years away from reality. Much of the basic infrastructure for this in KDE is in place but much is still left.
    The scenario described above is already partially implemented in the Activities but much work is still left.


    ==Frequently Asked Questions==
    ==Frequently Asked Questions==
    Line 80: Line 77:
    ;What is the Nepomuk Semantic Desktop, and the Strigi Desktop File Indexer?
    ;What is the Nepomuk Semantic Desktop, and the Strigi Desktop File Indexer?


    :The Nepomuk Semantic Desktop is the foundation of the all the other modules of the Nepomuk infrastructure. It provides a way to organize, annotate and build relationships among the data (not only file name and content, but for example which applications used a certain file, or how it is tagged). A number of KDE applications and workspaces use this basic infrastructure to deliver features such as email tagging ([[Special:myLanguage/KMail|KMail]]) or activity setup (Plasma).  
    : Nepomuk and Strigi are technologies part of what delivers the abilities of the Semantic Search in KDE. Both are not used directly in the latest generation of KDE's Semantic Search ([http://dot.kde.org/2014/02/24/kdes-next-generation-semantic-search details]), however their successors share much of their code and concepts. Semantic Search provides a way to organize, annotate and build relationships among the data (not only file name and content, but for example which applications used a certain file, or how it is tagged). A number of KDE applications and workspaces use this basic infrastructure to deliver features such as email tagging ([[Special:myLanguage/KMail|KMail]]) or activity setup (Plasma).  


    :On the other hand, the Strigi Desktop File Indexer is a system to index files so that they can be added to the main Nepomuk repository, a convenient way to use them within Nepomuk without adding any file manually. Also, applications such as [[Special:myLanguage/Dolphin|Dolphin]] can then search for files basing on content, name, or other meta-data (e.g. tags) associated to indexed files. Such an indexer can also index non-text files, such as PDFs, by accessing the meta-data contained in these files (author, publication information, etc.). Some KDE components ship additional "analyzers" for more file types. Nepomuk can be fully functional without the use of the File Indexer, which is an additional (and optional) component.
    :The file indexing allows applications such as [[Special:myLanguage/Dolphin|Dolphin]] to search for files based on content, name, or other meta-data (e.g. tags) associated to indexed files. Such an indexer can also index non-text files, such as PDFs, by accessing the meta-data contained in these files (author, publication information, etc.). Some KDE components ship additional "analyzers" for more file types.


    ; Why do we need both Akonadi and Nepomuk?  Aren't they doing the same thing?
    ; Why do we need both Akonadi and Semantic Search?  Aren't they doing the same thing?


    :There is often a good deal of confusion about Akonadi and Nepomuk. [http://cmollekopf.wordpress.com/2013/02/13/kontact-nepomuk-integration-why-data-from-akonadi-is-indexed-in-nepomuk/ Why data from Akonadi is indexed in Nepomuk] explains exactly why we have both, and what their roles are.  Don't miss the additional information from [http://en.opensuse.org/User:Wstephenson Will Stephenson] in the Comments section.
    : In short, Akonadi provides a cache of PIM data like calendar items, contacts and email, which is used by applications like KMail and Korganizer but also the calendar build in Plasma. Semantic Search plugs in Akonadi to provide search functionality. How Baloo offers search is actually up to the application. In case of KDE PIM, Xapian is used to provide indexing and search.


    ;How can I disable the semantic desktop?
    ;How can I disable the semantic desktop?


    :Most of the times, the easiest way is to disable file indexing, which is usually, among the Nepomuk components, the heavier in resource usage (although many optimizations were included in the 4.7 release which has reduced resource usage). This is done by unchecking <menuchoice>Enable Nepomuk File Indexer</menuchoice> in the [[Special:myLanguage/System_Settings/Search_Desktop|Desktop Search]] section of [[Special:myLanguage/System Settings|System Settings]]. In case you want to turn off all semantic features, uncheck <menuchoice>Enable Nepomuk Semantic Desktop</menuchoice>. Notice that this will turn off search in [[Special:myLanguage/Dolphin|Dolphin]] as well.
    : File indexing can be disabled by adding the users' home folder to the <menuchoice>System Settings -> Desktop Search -> Do not search in these locations list</menuchoice>. The other functionality is part of the applications that use it and thus can't be disabled without crippling these applications. For example, to not have any search in KMail you'll have to simply remove KMail...
     
    In versions of the KDE Applications before 4.13, Semantic Search would have components running separate from applications. This functionality could be disabled by unchecking <menuchoice>Enable Nepomuk File Indexer</menuchoice> in the [[Special:myLanguage/System_Settings/Search_Desktop|Desktop Search]] section of [[Special:myLanguage/System Settings|System Settings]]. In case you want to turn off all semantic features, uncheck <menuchoice>Enable Nepomuk Semantic Desktop</menuchoice>. Notice that this will turn off search in [[Special:myLanguage/Dolphin|Dolphin]] as well.
     
    :Notice that with the latter option some programs who use Semantic Search for meta-data will offer reduced functionality: for example [[Special:myLanguage/KMail|KMail]] will not be able to tag mail, or Plasma activities will not offer additional features such as icons, or program data information.
     
    ;Baloo/Semantic Search is eating 100% CPU! What do I do?


    :Notice that with the latter option some programs who use Nepomuk for meta-data will offer reduced functionality: for example [[Special:myLanguage/KMail|KMail]] will not be able to tag mail, or Plasma activities will not offer additional features such as icons, or program data information.  
    :Just wait. Certain files are very hard or even impossible to Index. At the moment, this includes for example text files of over 50 megabyte. When Search finds these, it will try for a fixed time. When it fails, it will try to find out what file is broken and disable indexing it in the future. As it indexes files in batches of about 40, it has to find the problematic file by indexing that bunch in parts: first half/second half, index problematic half in pieces again, until the file is found. This can take up to 30 minutes of heavy cpu usage. Unfortunately, while Baloo will not start to index a new batch of 40 files while on battery power, it continues to determine the broken file while on battery. This behaviour has been fixed in in KDE Applications 4.13.1 (it will stop indexing immediately when the power cord is unplugged) and the time the search for each file can take has been reduced to about 10 minutes. The Semantic Search team is working on improving the indexing tools to handle more difficult files.


    ;Why do I have nepomukservicestub processes even though I've disabled Nepomuk?
    ;Why do I have nepomukservicestub processes even though I've disabled Nepomuk?
    Line 108: Line 111:
    ;The virtuoso-t process hangs at 100% CPU.
    ;The virtuoso-t process hangs at 100% CPU.


    :Virtuoso-t is a key component of the Nepomuk infrastructure and in some occasions the commands sent by the other components end up taking too much time (hence showing the effect of 100% CPU). Sebastian Trüg (the lead developer of Nepomuk) has fixed most of these problems in 4.7.1 or newer.
    :Virtuoso-t is a key component of the old Semantic Search infrastructure and in some occasions the commands sent by the other components end up taking too much time (hence showing the effect of 100% CPU).
     
    Virtuoso is no longer used by Semantic Search starting the Applications 4.13 release.


    ;Sometimes Nepomuk consumes too much RAM.
    ;Sometimes Nepomuk consumes too much RAM.
    Line 114: Line 119:
    :Many of these problems have been fixed, in other cases however the developers are unable to reproduce the issues correctly. In this case, providing examples and test cases to [http://bugs.kde.org/ bug reports] increase the chances to get these bugs fixed.
    :Many of these problems have been fixed, in other cases however the developers are unable to reproduce the issues correctly. In this case, providing examples and test cases to [http://bugs.kde.org/ bug reports] increase the chances to get these bugs fixed.


    ;Nepomuk re-indexes files at startup.
    ;Search accesses the disk too much on startup.:


    :This bug has been fixed in 4.7.0 versions. Now Nepomuk just "scans" for changes, without indexing anything.
    :A throttling mechanism implemented in the file indexer, versions after KDE SC 4.8 should no longer have this issue.


    ;Nepomuk accesses the disk too much on startup.:
    ;My Search database has been corrupted. How do I clean it?


    :In 4.7 and newer this problem has been lessened thanks to a throttling mechanism implemented in the file indexer.
    :In the extreme case your database is really corrupted and all other attempts have failed, you can delete the <tt>$KDEHOME/share/apps/nepomuk</tt> directory (where <tt>$KDEHOME</tt> is usually <tt>.kde</tt> or <tt>.kde4</tt> in your home directory) while Nepomuk is not running. The database will be cleared, but you will also lose existing information such as tags, ratings and comments.
     
    ;My Nepomuk database has been corrupted. How do I clean it?
     
    :In the extreme case your database is really corrupted and all other attempts have failed, you can delete the <tt>$KDEHOME/share/apps/nepomuk</tt> directory (where <tt>$KDEHOME</tt> is usually <tt>.kde</tt> or <tt>.kde4</tt>) while Nepomuk is not running. The database will be cleared, but you will also lose existing information such as tags, ratings and comments.


    == Advanced troubleshooting ==
    == Advanced troubleshooting ==
    If Nepomuk has trouble indexing a file, in a terminal try running
    {{Input|1=nepomukindexer ''/path/to/file''}}
    and see if there's any useful output. If the file cannot be indexed, the indexer will print an error, but you will only see that error if you have debug messages for Nepomuk enabled (kdebugdialog). If the error is reproducible, please file a bug and upload the file along with the error.
    [http://kdeatopensuse.wordpress.com/2011/11/09/debugging-nepomukvirtuosos-cpu-usage/ This blog post] explains how to turn on debugging output from the [[Glossary#Nepomuk|Nepomuk]] service. It also shows how to debug the CPU usage of '''Nepomuk''' and its storage backend.
    [http://vhanda.in/blog/2012/02/virtuoso-going-crazy-/ This blog post] explains how to extract useful information for bug reports about the CPU usage of the '''Virtuoso''' backend.


    == Delen en privacy ==  
    == Delen en privacy ==  
    Line 144: Line 137:
    == External links ==
    == External links ==


    The new Search technology (post KDE Applications 4.13):
    * [http://dot.kde.org/2014/02/24/kdes-next-generation-semantic-search user information article on the dot]
    * [http://community.kde.org/Baloo Developer information on community.kde.org]
    * [http://en.wikipedia.org/wiki/Semantic_desktop Wikipedia - Semantic Desktop]
    The old Search technology:
    * [http://techbase.kde.org/Projects/Nepomuk Nepomuk pages for developers on KDE TechBase]
    * [http://techbase.kde.org/Projects/Nepomuk Nepomuk pages for developers on KDE TechBase]
    * [http://nepomuk.kde.org/discover/user NEPOMUK KDE site]
    * [http://nepomuk.kde.org/discover/user NEPOMUK KDE site]
    * [http://en.wikipedia.org/wiki/Semantic_desktop Wikipedia - Semantic Desktop]
    * [http://en.wikipedia.org/wiki/NEPOMUK_(framework)  Wikipedia - NEPOMUK Framework]  
    * [http://en.wikipedia.org/wiki/NEPOMUK_(framework)  Wikipedia - NEPOMUK Framework]  
    * [http://nepomuk.semanticdesktop.org/nepomuk/ NEPOMUK website]
    * [http://nepomuk.semanticdesktop.org/nepomuk/ NEPOMUK website]

    Latest revision as of 05:10, 9 May 2018

    Semantic Search

    The aim of this page is not to fully explain KDE's Semantic Search technology and every detail, but to give a short overview, some examples, share the vision behind it and link to relevant information on the web.

    Baloo is the next generation of semantic search

    From KDE Applications 4.13 onwards, the 'Baloo' file indexing and file search framework replaces Nepomuk. Read details on the changes for Applications 4.13 here. Semantic Search no longer uses a single, big database, but separate, specialized databases for each type of data. The new search databases are in $HOME/.local/share/baloo. If you upgraded to KDE Applications 4.13 from an earlier KDE release, you can delete $KDEHOME/share/apps/nepomuk.

    Korte uitleg

    As the Glossary mentions, Semantic Search is about classification, organisation and presentation of data. It is not an application, but a component which can be used by developers within applications.

    Try out in Dolphin

    For example, the Dolphin file manager makes use of Search. In KDE Applications versions prior to 4.13 Semantic Search must be enabled from System settings -> Desktop search. The information sidebar of Dolphin (Control -> Panels -> Information, or press F11) presents information extracted by Search about the selected file, and also allows you to assign tags, ratings and comments to files. This information is then stored and indexed by Search. You can then search for metadata using the navigation bar in Dolphin. Click to Find, or press Ctrl+F, and search for file names or file contents.


    Functionaliteiten

    KDE's Semantic Search offers several 'layers' of functionality to applications. The first and most simple of those is manual tagging, rating and commenting, as used in Dolphin. This helps you to find your files faster, but is also a lot of work.

    To make finding files containing text easier, Search offers a second functionality: indexing the text of files. You can find files by entering some words which you know are in there, or just (part of) their title.

    The third layer is a very complex one, and the reason why the underlying technology, Nepomuk, was conceived as a research project of several companies and universities in the European union. This is where you will find difficult words like 'semantic desktop' and 'ontologies'. Basically, it is about context and relationships.

    Indexing files

    Search does not index every file on the hard drive. Its default configuration in most Linux distributions excludes some common patterns for backup files and configuration directories. You can change this in System Settings -> Desktop Search. Add folders to be excluded. If you want to turn off indexing of files entirely, just add your home folder there.

    In System Settings you can also control whether Search indexes files on removable media such as USB drives and CD-ROMs. In KDE Applications 4.13 this is not available, removable media are not indexed. Future versions are planned to reintroduce this functionality.

    Voorbeelden

    Let me try to explain what Semantic Search offers using two examples. These features are not available fully yet - the base is there but application developers need to integrate this in their applications.

    Verbindingen

    Stel dat u twee weken geleden een foto hebt gekregen van een vriend. U heeft het ergens op uw computer opgeslagen. Maar hoe kan het bestand nu teruggevonden worden? Als u de locatie niet meer kunt herinneren, is dat een probleem.

    Now Semantic Search aims to help you. You know this file came from that friend of yours, your computer does not know. Search, however, can remember this relationship. Searching on the name of your friend will therefore pop up the photo!

    Another potential relationship is between a web page you copied text from and the document you pasted it into, or two images showing the same car. Such relations can sometimes be extracted from the files themselves (you could analyze photos and see who or what is on there) or supplied by the applications involved (as in the above email example). This part of Search is still under heavy development, and needs integration in applications, so you can expect it to take a few more years to really shine.

    All in all, this part of Semantic Search is about making search smart. Think about how Google tries to be smart with your searches: when you search for a hotel and a city name, it shows above the website results a google map showing hotels in the city you mentioned! It might even suggest a better name in case you made a spelling mistake. Google also tries to put the most relevant information on top of the list of results, using complex calculations on relationships (links) between websites. Semantic Search will be able to offer such smart results and order them on relevancy using relationship information.

    Context

    These relationships can not only help you while searching for files, but also have an influence on applications and what information they present. Note that this way of using Search is still more a vision than reality! Many of the components are in place, but it is not yet integrated in applications and the desktop as a whole.

    So here an example of bringing context awareness to your desktop could help you work more efficiently.

    Stel dat u bezig bent met het afmaken van wat notities die u tijdens een vergadering gemaakt heeft. De telefoon gaat en iemand vraagt u om die spreadsheet met prijzen te vinden en aan te passen voor een klant. Een paar onderbrekingen later heeft u uw bureaublad vol met bestanden en vensters...

    Het zou leuk zijn als alles een beetje beter te ordenen is, niet?

    Enter 'activities'. These have been introduced in Plasma, and currently offer different 'desktops'. They are a bit like virtual desktops, except that the desktop itself changes, not the set of applications. Different widgets, background, things like that. Of course, since Plasma 4.3, each virtual desktop can have its own activity, bringing the two in sync.

    If applications and desktop were aware of activities, you could create an activity for each of the tasks you regularly work on. So if you often have to change a spreadsheet with prices, you create an activity for that: put a Folder View (or several) widget on the desktop, add a calculator and a todo-widget to keep track of what you still have to change. Maybe even an email folder widget showing the mails with questions regarding these prices spreadsheets!

    As soon as somebody asks a question about prices, you switch to this activity. Fire up your spreadsheet application. It is aware of your activity so it shows recent price spreadsheets, not the recent list of inventory you were working on in another activity! Kopete, the chat application shows your colleague who knows all about prices, as she is the person you always chat with when working on this activity.

    Ga wanneer u klaar bent terug naar een andere activiteit, en opnieuw passen alle applicaties hun gedrag aan op wat u aan het doen bent.

    The benefits of such an activity-based work flow go further than you might at first expect. It not only helps you find files and contact persons, but also helps in switching tasks itself. The human brain isn't very good at multi-tasking - it takes most people several minutes to get up to speed after switching tasks. Changing the 'environment' helps a lot in speeding this up, even if it's just on the screen. Compare it with getting in the mood for your holiday by packing your bag!

    Of course, the above is mostly relevant to people working behind their computer in the office or at home. A gamer or a casual user would probably not use these activities much.

    The scenario described above is already partially implemented in the Activities but much work is still left.

    Frequently Asked Questions

    The following is taken from a KDE forums post. Please feel free to add/remove/modify details if you have the time!

    What is the Nepomuk Semantic Desktop, and the Strigi Desktop File Indexer?
    Nepomuk and Strigi are technologies part of what delivers the abilities of the Semantic Search in KDE. Both are not used directly in the latest generation of KDE's Semantic Search (details), however their successors share much of their code and concepts. Semantic Search provides a way to organize, annotate and build relationships among the data (not only file name and content, but for example which applications used a certain file, or how it is tagged). A number of KDE applications and workspaces use this basic infrastructure to deliver features such as email tagging (KMail) or activity setup (Plasma).
    The file indexing allows applications such as Dolphin to search for files based on content, name, or other meta-data (e.g. tags) associated to indexed files. Such an indexer can also index non-text files, such as PDFs, by accessing the meta-data contained in these files (author, publication information, etc.). Some KDE components ship additional "analyzers" for more file types.
    Why do we need both Akonadi and Semantic Search? Aren't they doing the same thing?
    In short, Akonadi provides a cache of PIM data like calendar items, contacts and email, which is used by applications like KMail and Korganizer but also the calendar build in Plasma. Semantic Search plugs in Akonadi to provide search functionality. How Baloo offers search is actually up to the application. In case of KDE PIM, Xapian is used to provide indexing and search.
    How can I disable the semantic desktop?
    File indexing can be disabled by adding the users' home folder to the System Settings -> Desktop Search -> Do not search in these locations list. The other functionality is part of the applications that use it and thus can't be disabled without crippling these applications. For example, to not have any search in KMail you'll have to simply remove KMail...

    In versions of the KDE Applications before 4.13, Semantic Search would have components running separate from applications. This functionality could be disabled by unchecking Enable Nepomuk File Indexer in the Desktop Search section of System Settings. In case you want to turn off all semantic features, uncheck Enable Nepomuk Semantic Desktop. Notice that this will turn off search in Dolphin as well.

    Notice that with the latter option some programs who use Semantic Search for meta-data will offer reduced functionality: for example KMail will not be able to tag mail, or Plasma activities will not offer additional features such as icons, or program data information.
    Baloo/Semantic Search is eating 100% CPU! What do I do?
    Just wait. Certain files are very hard or even impossible to Index. At the moment, this includes for example text files of over 50 megabyte. When Search finds these, it will try for a fixed time. When it fails, it will try to find out what file is broken and disable indexing it in the future. As it indexes files in batches of about 40, it has to find the problematic file by indexing that bunch in parts: first half/second half, index problematic half in pieces again, until the file is found. This can take up to 30 minutes of heavy cpu usage. Unfortunately, while Baloo will not start to index a new batch of 40 files while on battery power, it continues to determine the broken file while on battery. This behaviour has been fixed in in KDE Applications 4.13.1 (it will stop indexing immediately when the power cord is unplugged) and the time the search for each file can take has been reduced to about 10 minutes. The Semantic Search team is working on improving the indexing tools to handle more difficult files.
    Why do I have nepomukservicestub processes even though I've disabled Nepomuk?
    It may be a bug. Please file a bug report with a complete description of your problem and the steps to trigger it.
    File indexing of PDF/some other file types doesn't work.
    PDF indexing is a known issue and it's being tracked in bug #231936. If you have issues with other files, open a bug, preferably adding a sample file that shows the problem.
    The program nepomukservicestub crashes at startup.
    A large number of fixes for crashes has been fixed for the 4.7.2 release of the KDE Workspaces and Applications. If you encounter more, please file bugs report with detailed instructions on how to reproduce the problem, as sometimes the developers are unable to trigger them in their test setups.
    The virtuoso-t process hangs at 100% CPU.
    Virtuoso-t is a key component of the old Semantic Search infrastructure and in some occasions the commands sent by the other components end up taking too much time (hence showing the effect of 100% CPU).

    Virtuoso is no longer used by Semantic Search starting the Applications 4.13 release.

    Sometimes Nepomuk consumes too much RAM.
    Many of these problems have been fixed, in other cases however the developers are unable to reproduce the issues correctly. In this case, providing examples and test cases to bug reports increase the chances to get these bugs fixed.
    Search accesses the disk too much on startup.
    A throttling mechanism implemented in the file indexer, versions after KDE SC 4.8 should no longer have this issue.
    My Search database has been corrupted. How do I clean it?
    In the extreme case your database is really corrupted and all other attempts have failed, you can delete the $KDEHOME/share/apps/nepomuk directory (where $KDEHOME is usually .kde or .kde4 in your home directory) while Nepomuk is not running. The database will be cleared, but you will also lose existing information such as tags, ratings and comments.

    Advanced troubleshooting

    Delen en privacy

    There is one thing I need to touch on before pointing to other sources of information: sharing Nepomuk data. It'd be great if your tags, ratings and comments would be shared with others when you send them files. However, if you tagged a contact with a slightly embarrassing tag ('boring in bed') and send that persons contact information to a mutual friend you probably don't want that tag to be send as well...

    This issue is of course being considered and an important subject of research by the Nepomuk researchers. For the time being, these privacy concerns, combined with technical challenges, are the reason Nepomuk context is private. Rest assured the Nepomuk team does all it can to make sure your privacy is respected.

    External links

    The new Search technology (post KDE Applications 4.13):

    The old Search technology: