Revision as of 04:16, 26 January 2011

Collection Scanning

Every time Amarok is displaying a collection the information about tracks and albums needs to come from a source. The source can be an ipod, an internet service or a database. For tracks that are stored on the file system Amarok is using a database to have fast access to the required meta information. At first this information needs to be imported into the database which is usually done by scanning the collection directories for audio files. This process is called collection scanning.

It is useful to understand the scanning process in order to work better with Amarok.

Incremental Scan / Update Collection

The so called incremental scan will check the collection directories for updates. This is usually done every minute but can also be triggered manually by selecting "Update Collection" from the menu.

The incremental scan will just check the modification date of every folder in the collection against the last known modification time. This has a couple of implications: - You can trigger a rescan of one directory by modifying it's time (e.g. via "touch") - If files inside a directory are changed the scanner will not notice that (as changing a file does only update its modification time but not the time of the parent folder). - If the collection folders are on a very slow partition the process of just checking all the modification times can take some time. Usually this information is cached by the operating system but with large collections that might not be possible. In such a case the scanner might appear to scan continuously. With collections above 5000 tracks or when collections are stored on a network drive or an NTFS partition it is recommended to switch off the Watch folders for change option.

Progress bar / scanning time

The progress bar will show the progress of the scanning. Up to 50% the scanner will scan the file system and just buffer the result. Times above 50% indicate that the scanner is committing the results to the database. Usually the second step is much faster than the first so don't be surprised if the progress bar seems to jump. Up to 50% aborting the scan is possible. After 50%, the committing of the files can not be stopped.

The scanning time depend on your disk speed and other factors. Usually the first scan is a lot slower than subsequent scans where the files are cached by the operating system. A scan of 10000 files should take around three minutes on a modern computer. 50000 files should be around 13 minutes.

Backup of collection

With the default settings Amarok is storing all the collection information in a directory called ~.kde/share/apps/amarok/mysqle/ . It can be a good idea to make a backup of this directory from time to time, especially when you didn't enable the writing back of statistics information.

About unique ids

Amarok is tracking files by an id that is either stored in the audio track or computed by the artist, album title and track title meta information. This id helps Amarok to identify tracks that are moved to other locations so that statistics informations (rating, score, playcount) are not lost. Currently Amarok will not import tracks with duplicate unique ids. This leads to the surprising behavior that copied tracks still appear only once in Amarok.

In some circumstances even different tracks can end up with the same unique id. That is very uncommon and usually the result of running years of unstable Amarok releases. Such a problem can be seen by the debug output (start Amarok with the --debug option) while scanning.

About Albums

The scanner can only read single tracks but Amarok will display those sorted by album and compilation (an album without one specific artist). Amarok can't rely on the directory in which the files are located, since directory organizational schemes vary so widely.

The scanner is therefore doing the following: - Tracks without an album artist or an artist (or a composer in case of a classical track) are placed in a compilation. - Tracks that have the compilation flag set or an album artist other than "various artists" will be placed in an album. - Tracks that have the compilation flag set to 0 are placed in a compilation. - Albums called "Live", "Greatest Hits" and a couple of other names are always regarded as an album. - If we end up having tracks with several different artists left over they are placed inside a compilation, else we make one album out of them.

This process is quite complicated and even I was sometimes surprised by the results. However usually the outputs of the scanner can help figuring out why the tracks are sorted as they are. In such a case try executing (on a command line) amarokcollectionscanner -r ~/Music/directory. Look for "compilation" tags and tracks with different "artist" and "albumartist" tags.

@@ Line 1: / Line 1: @@
 == Collection Scanning ==
-Every time Amarok is displaying a collection the information about tracks and albums needs to come from a source.
+Every time '''Amarok''' is displaying a collection the information about tracks and albums needs to come from a source.
 The source can be an ipod, an internet service or a database.
-For tracks that are stored on the file system Amarok is using a database to have fast access to the required meta information.
+For tracks that are stored on the file system '''Amarok''' is using a database to have fast access to the required meta information.
 At first this information needs to be imported into the database which is usually done by scanning the collection directories for audio files.
 This process is called collection scanning.
+<br /><br />
+It is useful to understand the scanning process in order to work better with '''Amarok'''.
-It is useful to understand the scanning process in order to work better with Amarok.
+= Incremental Scan / Update Collection =
-= Incremental Scan / Update Collection =
 The so called incremental scan will check the collection directories for updates.
 This is usually done every minute but can also be triggered manually by selecting "Update Collection" from the menu.
+<br /><br />
 The incremental scan will just check the modification date of every folder in the collection against the last known modification time.
 This has a couple of implications:
 - You can trigger a rescan of one directory by modifying it's time (e.g. via "touch")
-- If files inside a directory are changed the scanner will not notice that (as changing a file does only update it's modification time but not the time of the parent folder).
+- If files inside a directory are changed the scanner will not notice that (as changing a file does only update its modification time but not the time of the parent folder).
-- If the collection folders are on a very slow partition the process of just checking all the modification times can take some time. Usually this information is cached by the operating system but with large collections that might not be possible. In such a case the scanner might appear to scan continuously.
+- If the collection folders are on a very slow partition the process of just checking all the modification times can take some time. Usually this information is cached by the operating system but with large collections that might not be possible. In such a case the scanner might appear to scan continuously. With collections above 5000 tracks or when collections are stored on a network drive or an NTFS partition it is recommended to switch off the <menuchoice>Watch folders for change</menuchoice> option.
-With collections above 5000 tracks or when collections are stored on a network drive or an NTFS partition it is recommended to switch off the "watch folders for change" option.
 = Progress bar / scanning time =
@@ Line 26: / Line 25: @@
 Times above 50% indicate that the scanner is committing the results to the database.
 Usually the second step is much faster than the first so don't be surprised if the progress bar seems to jump.
-Up to 50% aborting the scan is possible. After 50% the commiting of the files can not be stopped.
+Up to 50% aborting the scan is possible. After 50%, the committing of the files can not be stopped.
+<br /><br />
 The scanning time depend on your disk speed and other factors.
 Usually the first scan is a lot slower than subsequent scans where the files are cached by the operating system.
@@ Line 33: / Line 32: @@
 = Backup of collection =
-With the default settings Amarok is storing all the collection information in a directory called ~.kde/share/apps/amarok/mysqle/
+With the default settings '''Amarok''' is storing all the collection information in a directory called ''~.kde/share/apps/amarok/mysqle/'' . It can be a good idea to make a backup of this directory from time to time, especially when you didn't enable the writing back of statistics information.
-It can be a good idea to make a backup of this directory from time to time, especially when you didn't enable the writing back of statistics information.
 = About unique ids =
-Amarok is tracking files by an id that is either stored in the audio track or computed by the artist, album title and track title meta information.
+'''Amarok''' is tracking files by an id that is either stored in the audio track or computed by the artist, album title and track title meta information.
-This id helps Amarok to identify tracks that are moved to other locations so that statistics informations (rating, score, playcount) are not lost.
+This id helps '''Amarok''' to identify tracks that are moved to other locations so that statistics informations (rating, score, playcount) are not lost.
-Currently Amarok will not import tracks with duplicate unique ids.
+Currently '''Amarok''' will not import tracks with duplicate unique ids.
-This leads to the surprising behavior that copied tracks still appear only once in Amarok.
+This leads to the surprising behavior that copied tracks still appear only once in '''Amarok'''.
+<br /><br />
-In some circumstances even different tracks can end up with the same unique id. That is very uncommon and usually the result of running years of unstable Amarok releases.
+In some circumstances even different tracks can end up with the same unique id. That is very uncommon and usually the result of running years of unstable '''Amarok''' releases.
-Such a problem can be seen by the debug output (start Amarok with the --debug option) while scanning.
+Such a problem can be seen by the debug output (start '''Amarok''' with the --debug option) while scanning.
 = About Albums =
-The scanner can only read single tracks but Amarok will display those sorted by album and compilation (an album without one specific artist).
+The scanner can only read single tracks but '''Amarok''' will display those sorted by album and compilation (an album without one specific artist).
-Amarok can't rely on the directory the files are located as several applications will sort tracks by artist and not album.
+'''Amarok''' can't rely on the directory in which the files are located, since directory organizational schemes vary so widely.
+<br /><br />
 The scanner is therefore doing the following:
 - Tracks without an album artist or an artist (or a composer in case of a classical track) are placed in a compilation.
@@ Line 55: / Line 53: @@
 - Albums called "Live", "Greatest Hits" and a couple of other names are always regarded as an album.
 - If we end up having tracks with several different artists left over they are placed inside a compilation, else we make one album out of them.
+<br /><br />
 This process is quite complicated and even I was sometimes surprised by the results.
 However usually the outputs of the scanner can help figuring out why the tracks are sorted as they are.
-In such a case try executing (on a command line) "amarokcollectionscanner -r ~/Music/directory"
+In such a case try executing (on a command line) ''amarokcollectionscanner -r ~/Music/directory''.
 Look for "compilation" tags and tracks with different "artist" and "albumartist" tags.