Move audio from source to destination and ensure format is consistent.

Moves audio files from temporary area to permanent location, formatting along the way.

Background:  I had not used perl before this script so there are better methods than what I have here.  This works for me and is a living script.  That only means that I find myself making changes as I encounter situations that need addressed.

I use tagscanner on a Windows workstation that puts files into a holding area on my MythTV server (Amarok does a good job in Linux).  Tagscanner is really one of the best Windows' applications for audio files.  Extremely flexible and it's free.  The files are renamed into a format of "/Working/Directory/Artist/Album/Track. Title".  This script then starts in the source directory: "/Working/Directory" scanning all artists and albums.  From there this script formats all text to a consistent style and case (Linux bound - case matters).  This allows me to prevent some duplicates due to case and other formatting.  This script will grab album artwork from Amazon if I don't already have any in the source or destination directory.

Also, I prefer FLAC over MP3 but I also strive to have the better MP3 so this script will look for a higher bit rate in an MP3 and keep that over a lower bit rate.  The script gets rid of the MP3 if I have a FLAC of the same file.  For FLAC files, I convert them back to WAV to ensure that all Vorbis metatags have been removed and that the FLAC file's integrity has been checked.  I do the same to APE files.  Once in WAV format, I then convert those back to FLAC, adding only the tags that I am interested in having.

In the script, you will see the option (-a) to add artwork to each file.  I usually don't use that option.  If I convert the FLAC to MP3, I have another script that will add the artwork to the MP3.  This way, the MP3 works on my iPhone with artwork.  Just need Apple to start supporting FLAC (hey - we all have dreams).

Bottom line, put the raw files in "/some/base/directory/artist/album/track title" set the SRC_DIR and DEST_DIR variables in this script and let it run.  The location in the script for these two variables show how this script has evolved.  They push me to rewrite much of this thing.  Fortunately, the locations don't change too often.

Update - June 8 2014 - Just some minor changes to the script.  I started using foobar2000 for media playback in place of WinAMP.  With WinAMP, one can right-click a folder and play the content.  This isn't available in foobar.  Instead, foobar can use a cue file.  With that in mind, I created a sub routine to create a basic cue file when the media is being synced from the source to the destination.

I also modified the directory structure on the destination.  Before, I used /Working/Directory/Artist/Album/Track.  I now use /Working/Directory/Alpha/Artist/Album/Track.  An example is /Media/music/A/Adele/19/Track or Media/music/L/Led Zeppelin/II/Track.  I did this to make accessing the data more quickly when browsing, especially from a smartphone or HTPC remote.

<<< --- syncaudio.pl --->>>

#!/usr/bin/perl
use strict;
use warnings;
use Cwd;
#use Cwd qw(chdir);
use Audio::FLAC::Header;
use MP3::Info;
use MP3::Tag;
use sort '_qsort';
use File::Glob qw(:globally :nocase :glob);
use File::Path;
use File::Copy;
use Encode qw(encode decode);
use RequestSignatureHelper;
use Data::Dumper;
use lib '/home/jwhite/bin/lib';
use My::Stuff qw(Format_Text);
#use utf8;
#use Encode;

require LWP::UserAgent;
use LWP::Simple;        # libwww-perl providing simple HTML get actions
#use LWP::Simple qw($ua get);   # libwww-perl providing simple HTML get actions
use HTML::Entities;
use URI::Escape;
use XML::Simple;

use vars qw($opt_a $opt_r $opt_d $opt_f $opt_n);
use Getopt::Std;

#our $DEBUG = 0;

sub System_Call {
        my @args = @_;
        system (@args);
        if ($? == -1) {
                print "\nFailed to execute: $!\n";
                exit 2;
        }
        elsif ($? & 127) {
                printf "\nChild died with signal %d, %s coredump\n",
                ($? & 127), ($? & 128) ? 'with' : 'without';
                exit 2;
        }
        return $?;
}

sub findError {
        my $xml = shift;

        return undef unless ref($xml) eq 'HASH';

        if (exists $xml->{Error}) { return $xml->{Error}; };

        for (keys %$xml) {
        my $error = findError($xml->{$_});
        return $error if defined $error;
        }

        return undef;
}

sub Get_Album_Cover {
        my $am_artist = "";
        my $am_title = "";
        my $ama_uri = "";
        my $ama_genre = "";
        my $found = 0;
        my $uri = "";

        my $artist = $_[0] or die "\tI don't like the text: $_[0] - $!\n";
        my $album = $_[1] or die "\tI don't like the text: $_[1] - $!\n";
        my $Full_Path = $_[2] or die "\tI don't like the path text: $_[2] - $!\n";
        if (defined $opt_d) { print "# ", $artist, "\\", $album, "\n"; }
        $album =~ s/ Cd\d*//g;
        $album =~ s/ \(.*\)//g;
        $album =~ s/^\(.*\) //g;
        if (defined $opt_d) { print "# ", $artist, "\\", $album, "\n"; }
        if (defined $opt_d) { print "# starting to look for cover on Amazon.com\n"; }

#       $ua->proxy('http','http://192.168.2.1:8080');
        use constant myAWSId            => '<Get From Amazon>';
        use constant myAWSSecret        => '<Get From Amazon>';
        use constant myEndPoint  => 'ecs.amazonaws.com';
        my $helper = new RequestSignatureHelper (
                +RequestSignatureHelper::kAWSAccessKeyId => myAWSId,
                +RequestSignatureHelper::kAWSSecretKey => myAWSSecret,
                +RequestSignatureHelper::kEndPoint => myEndPoint,
        );

        my $request = {
                Service => 'AWSECommerceService',
                AssociateTag => '<Get From Amazon>',
                Operation => 'ItemSearch',
                SearchIndex => 'Music',
                Keywords => $album." ".$artist,
                Sort => 'relevancerank',
                ResponseGroup => 'Images,ItemAttributes,BrowseNodes',
        };
        my $signedRequest = $helper->sign($request);

        my $queryString = $helper->canonicalize($signedRequest);
        my $url = "http://" . myEndPoint . "/onca/xml?" . $queryString;
        if (defined $opt_d) { print "# Query URL: $url\n";}

        my $ua = new LWP::UserAgent();
        my $response = $ua->get($url);
        my $content = $response->content();

        my $xmlParser = new XML::Simple();
        my $xml_doc = $xmlParser->XMLin($content);

        if (defined $opt_d) {
                print "Parsed XML is: " . Dumper($xml_doc) . "\n";
        }

        if ($response->is_success()) {
                my $title = $xml_doc->{Items}->{Request}->{ItemSearchRequest}->{Title};
        } else {
                my $error = findError($xml_doc);
                if (defined $error) {
                        print "Error: " . $error->{Code} . ": " . $error->{Message} . "\n";
                } else {
                        print "Unknown Error!\n";
                }
        }

        if (ref($xml_doc->{Items}->{Item}) ne 'ARRAY') {
                my @tmpArray = ($xml_doc->{Items}->{Item});
                $xml_doc->{Items}->{Item} = \@tmpArray;
        }

        my $k = 0;
        do {
                $am_artist=$xml_doc->{Items}->{Item}->[$k]->{ItemAttributes}->{Artist};
                $am_title=$xml_doc->{Items}->{Item}->[$k]->{ItemAttributes}->{Title};
                $ama_uri=$xml_doc->{Items}->{Item}->[$k]->{LargeImage}->{URL};
                if (defined $ama_uri) {
#                       $ama_genre=$xml_doc->{Items}->{Item}->[$k]->{BrowseNodes}->{BrowseNode}->[0]->{Name};
#                       if (defined $opt_d) {
                                print "# Amazon: found cover for $am_artist/$am_title: $ama_uri \n";
#                               print "# Amazon: Genre: $ama_genre \n";
#                       }
                        $found = 1;
                }
                $k++;
        } until ($found || $k == 10);
        #only search through first 10 matches

        my $image = get $ama_uri if (defined($ama_uri) && $ama_uri ne "");
        if (!defined($image)) {
                if (defined $opt_d) { print("# No image found for $artist/$album\n"); }
                return;
        }
        if ($ama_uri ne "" && length($image) eq "807") {
                if (defined $opt_d) { printf("# this image is blank\n"); }
                $ama_uri = "";
        }

        if (!defined($ama_uri)) {
                $ama_uri = "";
        }

        if ($ama_uri ne "") {
                if (defined $opt_d) { printf("Found: ", $ama_uri, "\n"); }
                $uri = $ama_uri;
        }

        my $jpeg_image = get $uri;
        my $outfile = $Full_Path."/folder.jpg";
        if (defined $opt_d) { printf("OutFile: ", $outfile, "\n"); }
        open(INFO, ">$outfile");

        binmode INFO;
        print INFO $jpeg_image;
        close INFO;
}

sub Convert_Image_Format {
#This function assumes only one image file in the source directory
        my @Src_IMG_Files = <*.{bmp,png}> ;
        my $Num_Src_IMG_Files = @Src_IMG_Files;
        if ($Num_Src_IMG_Files ge 1) {
                my $Src_IMG_File = $Src_IMG_Files[0];
                System_Call ("convert", "$Src_IMG_File", "-quiet", "-quality", "100", "folder.jpg");
                unlink $Src_IMG_File;
        }
}

sub Convert_Wav_To_Flac {
        print "Converting Wav to Flac";
        my $results;
        my @WAVS = <*.wav>;
        foreach my $wav (@WAVS) {
                chomp $wav;
                print ".";
#               $results=System_Call ("shntool", "conv", "-w", "-q", "-o", "flac", "$wav");
                $results = System_Call ("flac", "-5", "--verify", "-f", "-s", "$wav");
                if (!$results) {
                        unlink $wav;
                } else {
                        print "Results: ",$results,"\n";
                        print "There is a problem with: ", $wav, "\n";
                        exit 1;
                }
        }
        print "\n";
}

sub Convert_Flac_To_Wav {
        print "Converting Flac to Wav";
        my $results;
        my @FILES = <*.{flac}> ;
        foreach my $File (@FILES) {
                chomp $File;
                print ".";
#               $results = System_Call("shntool", "conv", "-q", "-o", "wav", "$File");
# I had a condition where shntool could not interpret the WAV file correctly.
                $results = System_Call ("flac", "--verify", "-F", "-f", "-s", "-d", "$File");
                if (!$results) {
                        unlink $File;
                } else {
                        print "Results: ",$results,"\n";
                        print "There is a problem with: ", $File, "\n";
                        exit 1;
                }
        }
        print "\n";
}

sub Convert_Ape_To_Wav {
        print "Converting Ape to Wav";
        my $results;
        my @APEFILES = <*.{ape}> ;
        foreach my $ApeFile (@APEFILES) {
                chomp $ApeFile;
#               my $WavFile = $ApeFile;
#               $WavFile =~ s/\.ape$/\.wav/;
                print ".";
                $results = System_Call("shntool", "conv", "-q", "-o", "wav", "$ApeFile");
# mac does not provide a quiet mode. Still using shntool to convert to wav.
#               $results = System_Call ("mac", "$ApeFile", "$WavFile", "-d");
                if (!$results) {
                        unlink $ApeFile;
                } else {
                        print "Results: ",$results,"\n";
                        print "There is a problem with: ", $ApeFile, "\n";
                        exit 1;
                }
        }
        print "\n";
}

sub Create_CUE {
        my $CUEName = $_[0]."/00.cue";
        my $Artist = $_[1];
        my $Album = $_[2];
        my $i = 1;
        my $Track = 0;
        my @files = sort <$_[0]/*.{mp3,flac}>;
        open (CUEFILE, '>'.$CUEName);
        print CUEFILE "PERFORMER \"".$Artist."\"\n";
        print CUEFILE "TITLE \"".$Album."\"\n";
        foreach my $file (@files) {
                $file = ( split m!/!, $file )[-1];
                my $Title = $file;
                $Title =~ s{\.[^.]+$}{}; #Remove the extension
                $Title =~ s(^\d+[\.\_\-\ ]+)(); #Remove the leading track number
                print CUEFILE "FILE \"".$file."\" WAVE\n";
                $Track = sprintf("%02d", $i);
                print CUEFILE " TRACK ".$Track." AUDIO\n";
                print CUEFILE " TITLE \"".$Title."\"\n";
                print CUEFILE " INDEX 01 00:00:00\n";
                $i++;
        }
        close (CUEFILE);
}

sub Copy_Files {
        my $Artist = $_[0] or exit 1;
        my $Album = $_[1] or exit 1;
        my @Src_JPG_Files = {};
        my @Dest_JPG_Files = {};
        $Artist = decode('UTF-8',$Artist);
        $Album = decode('UTF-8',$Album);
        my $Category = substr($Artist,0,1);
        if ($Category =~ /\d/ ) { $Category = "0" };
        my $Dest_Dir = "/Media/music";
        my $Full_Path = $Dest_Dir."/".$Category."/".$Artist."/".$Album;
        my $Picture = $Full_Path."/folder.jpg";
        my @created = mkpath ($Full_Path);
        Convert_Image_Format;
#       If I have any JPG files in the source location, copy them to the dest.
        @Src_JPG_Files = <*.jpg>;
        my $Num_Src_JPG_Files = @Src_JPG_Files;
        if ($Num_Src_JPG_Files ge 1) {
                my $Src_JPG_File = $Src_JPG_Files[0];
                my $Dest_JPG_File = $Full_Path."/folder.jpg";
                print "Found an Image in the source and copying now\n";
                unless (copy($Src_JPG_File,$Dest_JPG_File)) {
                        print "Cannot copy JPG: ", $Src_JPG_File, "\n";
                }
        }
#       Don't bother getting cover from Amazon if I already have a JPG in the dest.
        @Dest_JPG_Files = bsd_glob( $Full_Path."/*.jpg" );
        my $Num_Dest_JPG_Files = @Dest_JPG_Files;
        if ($Num_Dest_JPG_Files eq 0) { Get_Album_Cover ($Artist, $Album, $Full_Path); }
#       Process MP3 and FLAC files from the source to the destination.
        my @Src_Files = sort <*.{mp3,flac}> ;
        my $Num_Files = @Src_Files;
        if ($Num_Files > 0) {
                my $CP_OK = 1;
                my $New_Track = 0;
                foreach my $Src_File (@Src_Files) {
                        my $Dest_File = $Src_File;
                        if ($Dest_File =~ /^(\d+[\.\_\-\ ]+)(.*)(\.\w+)$/) {
                                my ($Track, $Title, $Ext) = ($1, $2, lc($3));
                                $Track =~ s/^(\d+)(.*)/$1/;
                                if (defined $opt_n) {
                                        $Track = ++$New_Track;
                                }
                                $Track = sprintf("%02d", $Track);
                                $Title = Format_Text ($Title);
                                $Title = decode('UTF-8',$Title);
                                $CP_OK = 1;
                                $Dest_File = $Full_Path."/$Track. $Title$Ext";
                                if ( (-e $Dest_File) && ($Ext eq ".mp3") ) {
                                        my $srcinfo = get_mp3info($Src_File);
                                        my $destinfo = get_mp3info($Dest_File);
                                        if ($srcinfo->{BITRATE} <= $destinfo->{BITRATE}) {
                                                $CP_OK = 0;
                                        }
                                }
                                if ($Ext eq ".mp3") { #Don't copy mp3 if flac exists
                                        my $FLAC_Equiv = $Dest_File;
                                        $FLAC_Equiv =~ s/\.mp3$/\.flac/;
                                        if (-e $FLAC_Equiv) {
                                                $CP_OK = 0
                                        }
                                }
                                if ($CP_OK) {
                                        unless (copy($Src_File,$Dest_File)) {
                                                print "File Cannot be copied. Deleting zero-byte file.\n";
                                                unlink ($Dest_File);
                                        }
                                        if ($Ext eq ".flac") { #Replace mp3 with flac
                                                my $MP3_Equiv = $Dest_File;
                                                $MP3_Equiv =~ s/\.flac$/\.mp3/;
                                                if (-e $MP3_Equiv) {
                                                        print "\tReplacing $Track. $Title.mp3 with $Title.flac \n";
                                                        unlink ($MP3_Equiv);
                                                }
                                        }
                                        if ($Ext eq ".mp3") {
#                                               remove_mp3tag($Dest_File, 'ALL');
                                                my $mp3 = MP3::Tag->new($Dest_File);
                                                $mp3->get_tags;
                                                if (exists $mp3->{ID3v1}) {
                                                        $mp3->{ID3v1}->remove_tag;
                                                }
                                                if (exists $mp3->{ID3v2}) {
                                                        $mp3->{ID3v2}->remove_tag;
                                                }
                                                my $id3v1 = $mp3->new_tag("ID3v1");
                                                $id3v1->all($Title,$Artist,$Album,"","",$Track,"Rock");
                                                $id3v1->write_tag;
                                                my $id3v2 = $mp3->new_tag("ID3v2");
                                                $id3v2->add_frame('TRCK',$Track);
                                                $id3v2->add_frame('TIT2',$Title);
                                                $id3v2->add_frame('TPE1',$Artist);
                                                $id3v2->add_frame('TALB',$Album);
                                                $id3v2->add_frame('TCON',"17");
                                                if ((open(PICFILE, $Picture)) && (defined $opt_a)) {
                                                        my $filesize = (-s PICFILE);
                                                        binmode(PICFILE);
                                                        read(PICFILE, my $imgdata, $filesize);
                                                        close(PICFILE);
                                                        $id3v2->add_frame("APIC",
                                                                                        chr(0x0),
                                                                                        "image/jpeg",
                                                                                        chr(0x0),
                                                                                        "Cover Image",
                                                                                        $imgdata);
                                                }
                                                $id3v2->write_tag;
                                        } else {
                                                my $flac = Audio::FLAC::Header->new($Dest_File);
                                                my $tags = $flac->tags();
                                                $tags->{TRACKNUMBER} = $Track;
                                                $tags->{TITLE} = $Title;
                                                $tags->{ARTIST} = $Artist;
                                                $tags->{ALBUM} = $Album;
#                                               $tags->{'ALBUMARTIST'} = $Artist;
                                                $tags->{GENRE} = "Rock";
                                                if ((-e $Picture) && (defined $opt_a)) {
                                                        System_Call ("metaflac", "--import-picture-from=$Picture",$Dest_File);
                                                }
                                                $flac->write();
                                        }
                                }
                        } else {
                                print "File format not recognized: $Src_File \n";
                                exit 1;
                        }
                }
        }
        Create_CUE $Full_Path,$Artist,$Album;
}

unless (-x "/usr/bin/shntool") {
        print "I need the program shntool to function\n";
        exit 1;
}
unless (-x "/usr/bin/mac") {
        print "I need the program mac to function\n";
        exit 1;
}
unless (-x "/usr/bin/flac") {
        print "I need the program flac to function\n";
        exit 1;
}
unless (-x "/usr/bin/metaflac") {
        print "I need the program metaflac to function\n";
        exit 1;
}

getopts('ardfn');
my $Src_Dir = "/Media/work";
opendir (Src_DH, $Src_Dir) || die "can't opendir $Src_Dir: $!";
chdir Src_DH;

my @Src_Artists = grep { !/^\./ && -d "$_" } sort readdir(Src_DH);
chomp @Src_Artists;
foreach my $Src_Artist (@Src_Artists) {
        chomp $Src_Artist;
        if (! ($Src_Artist eq 'Junk')) {
                my $Dest_Artist = Format_Text ($Src_Artist);
                opendir (Src_Artist_DH, $Src_Artist) || die "can't open $Src_Artist: $!";
                chdir($Src_Artist);
                my @Src_Albums = grep { !/^\./ && -d "$_" } sort readdir(Src_Artist_DH);
                foreach my $Src_Album (@Src_Albums) {
                        print "Processing $Src_Artist / $Src_Album \n";
                        my $Dest_Album = Format_Text ($Src_Album);
                        chdir($Src_Album);
                        if (defined $opt_f) {
                                Convert_Flac_To_Wav;
                                Convert_Ape_To_Wav;
                                Convert_Wav_To_Flac;
                        }
                        Copy_Files ($Dest_Artist, $Dest_Album);
                        chdir "..";
                }
                chdir "..";
                closedir Src_Artist_DH;
        }
}
closedir Src_DH;