Niche tech tips are the cornerstone of this website, and here is another. How to upgrade an iTunes/Music library whilst maintaining the database.
I know everyone streams music these days – hence this is likely a very niche tech tip – but I still maintain a large music database on a server. The data in my library is precious (to me) since it tracks my listening habits over 17 years. I use the number of plays, last played, and date added information for generating playlists. Now, some of the tracks in my library date from a time when hard disk space was at a premium and the standard for bit rates was low. I wanted to upgrade these low bit rate versions with 320 kps MP3 versions. However, I wanted to maintain the play counts and all data associated with those files. The only way to upgrade and maintain the data is to replace the file in the filesystem. This post explains the way to do it.
The setup
All the music is stored on a server and I use a mac to control adding and organising the library with Music (previously called iTunes). I worked on a separate mac running Music that was not attached to the library to do most of the steps. This is not essential but made things a lot easier. Both macs are running macOS Monterey 12.4 and the latest Music.
The whole process was made easier/feasible thanks to Copy Tag Info Tracks to Tracks from Doug’s Applescripts.
The idea is to replace the original file on the server with a higher bit rate version. To do this, we need the new file to be named exactly the same as the original. Now, the file name is controlled by the track number (and disc number if there is more than one disc) and the song title. We also need all the tags and artwork to be the same. The file location is determined by the Artist and Album. All of these things can be set by cloning the info from the original file, and this is where Doug’s script comes in.
What needs replacing?
To figure out what should be replaced, I exported the library in XML format and read it into R. I plan to post in the future about how to do this. From here I could generate a list of albums where the average bit rate was lower that 200. For many of these albums I have a lossless copy in FLAC, so it was these albums I worked on. If you happen to have higher bit rate mp3 versions from some other source you can skip the next step.
Making high bit-rate versions
The files need to be converted to mp3 since a) the original file is mp3 and this is what we are replacing and b) Music does not work with FLAC. I copied the lossless albums into a directory on my mac and then, on the command line:
cd path/to/my/albums/
find . -iname '*.flac' -exec bash -c 'D=$(dirname "{}"); B=$(basename "{}"); mkdir "$D/mp3/"; ffmpeg -i "{}" -ab 320k -map_metadata 0 -id3v2_version 3 "$D/mp3/${B%.*}.mp3"' \;\n
As described here this will convert all of the albums into 320 kbps mp3 files and place them in a subfolder within each album’s folder.
Import into Music
This whole directory can be drag-and-dropped into Music. As long as the option “Keep Music media files organised” is set, it should copy the MP3s (ignoring the FLACs) into the Music library. Usually this is in ~/Music/iTunes/iTunes Media/Music/ or ~/Music/Music/Media/ depending on your setup.
As long as the track numbers are correct, we’re good to go to the next step.
Import the original tracks into Music
The original versions from the server can now be imported into Music. Now, using the script, select the originals to copy the tags and then select the new versions and move the tags over. It’s worth just eyeballing things like track durations of original vs new files to check that the files got renamed properly.
The original, low bit rate versions can now be deleted in Music. The simplest way to do this is to sort by bit rate and select everything lower than 320 and delete. You should now have all files organised on your local disk the same way as on the server.
Problem 1
Music gets confused at this point when trashing the files. I think because there are two copies of each file. So in some circumstances, the high bit rate versions are present in Music and the original is gone, however, the high bit rate file is in the trash. The solution here is to drag the files out of the trash and put them somewhere ~/Music/iTunes/iTunes Media/ is fine, and then play track 1 and 2 of the affected album. Music then hoovers them back into their proper location.
Problem 2
The filename may be 09 Shout 1.mp3 rather than 09 Shout.mp3
The cause is that the new version joined the folder of the original and got renamed, when the tag copying was done. If the files have correct album and artist on import, then the new version will have the correct filename while the original gets bumped to the 1.mp3 name. This is best fixed by using the Rename function in Finder on all files in the library. Use option+click to expand all artist folders and album subfolders, select all files, right-click and Rename… then use Replace Text and swap 1.mp3 to .mp3 The only downside to this is that songs whose titles actually end with a space and 1 get renamed, but we will check those when we verify before copying.
Verify before proceeding
Back in the terminal:
cd ~/Music/iTunes/iTunes\ Media/Music/
find . -type d -maxdepth 1 -exec diff -rq {} /Volumes/Multimedia/iTunes/iTunes\ Music/{} ';' | grep "^Only" | grep ".mp3$" | grep -v "._"
Here, /Volumes/Multimedia/iTunes/iTunes\ Music/ is the path to the server versions where the originals are. This command will check each artist folder against the server and report if a file is missing or in either location. The three chained grep commands will find lines that report a file “Only in” one location or the other, it will only report mp3s, and it will ignore the mac droppings.
Any mistakes in naming should get picked up here. If this command yields nothing, it’s a good idea to remove the greps and check that you are actually searching the correct locations!
Any problems in naming need to be fixed and then verified again until this command yields nothing.
Replace the files on the server
I had a backup of the original files so I was happy to just do this next part, but obviously caution is needed here since we will overwrite the original files in this next step.
cd ~/Music/iTunes/iTunes\ Media/Music/
rsync -trv --dry-run . /Volumes/Multimedia/iTunes/iTunes\ Music/
If you are happy with what you see here, remove --dry-run and go for it.
The files on the server are now replaced with high bit rate versions. Woohoo!
Final step
The version of Music that controls the library on the server, doesn’t know (yet) that the files have been replaced. There are several ways to update the library to refresh the connection between Music and the files in the filesystem. The quickest and easiest was to make a playlist of the files that were changed, select them all and to tick/untick them. If you have View > Songs setup with a column displaying bit rate you will see them changed from their meagre 128 or whatever into a fully fledged 320.
Is that everything?
Well, not quite. If you sync music from your library to a phone or something, the device will not update to the new files without prompting. How you trigger this depends a bit on your setup but if you have “Sync only ticked items” selected, you can untick the updated files; sync the phone; tick them again and sync once more. Now you can enjoy the higher bit rate versions on your phone.
One last note is that Music will not rescan the files to determine the audio volume to set the appropriate volume level. If anyone knows how to rescan an entire library to do this, let me know! With an old library the audio levels are likely shot anyway, so I was not too concerned about this. My only tip is that if playback seems quiet for an album, just adjust it for that album, in Music.
Miscellany
Whilst updating these files, I also did a spring clean of metadata. Here are some tools that are very useful for wrangling and organising music files:
- Ben Dodson’s iTunes Artwork Finder which was invaluable for finding 600×600 images for tracks.
- Discogs is essential for figuring out which version of album is which and for locating artwork for obscure releases.
- Doug’s scripts gets another shout out. Besides the script used in the post, he has many useful scripts available. He only asks for $5 for some of the scripts and they are definitely worth it.
id3tagis useful for programmatically setting id3 tags in music files on the command line. Very handy if you have hundreds to do!- This fork of a python script from mervick is great for splitting FLAC into separate files using a CUE file.
—
Part of an occasional series of tech tips.