Oct 22 2005
Radio Capture and Self Podcasting
For a long time I’ve wanted to capture radio shows on
I got a Radio Shark, hooked it up to my PC, and used its TIVO-like software to record the shows I wanted.
I’ve read that the Mac version of the software will save in a number of formats, but the Windows version only saves as WAV files. I wanted to get these into a format that works well on the iPod – either MP3 or AAC.
These are big files, some of them over 2 hours and gigabytes in length. I needed to compress them. I also wanted to use the iPod’s bookmarking feature so that I could pause a file, jump to another playlist for a while, then come back where I left off – a feature iTunes and the iPod call bookmarking. MP3s don’t do this, only
I found a WAV to AAC converter called
Here’s the process to convert the files:
I unmount then remount the Windows drive from the Linux box (clunky but ensures I have a clean mount), copy the WAVs over into a /tmp directory named after the date, then delete them from the Windows box to keep it from filling up. Next I run the FAAC converter on them, dropping each completed file into a web directory so it will be accessible to iTunes later. Note again that my output file has a .
I’m not sure if ‘basename‘ is a common command in unix. I used it to strip the filename of all extensions and make it easy to rename.
Here’s the script for all of this (sound-convert.sh):
umount /var/netshare/dshare/radio
mount -t cifs //192.168.1.77/radio /var/netshare/dshare/radio -o credentials=/root/.cred2DATE=`date +"%Y-%m-%d"`
echo $DATEmkdir /tmp/$DATE-soundconv
cp /var/netshare/dshare/radio/* /tmp/$DATE-soundconv
rm -f /var/netshare/dshare/radio/*
for i in /tmp/$DATE-soundconv/*.wav
do
j=`basename "$i" .wav`;
echo $j;/usr/local/bin/faac -w -q 150 -c 22000 /tmp/$DATE-soundconv/$j.wav -o /var/www/html/radiofiles/$j.m4b --artist "NPR" --title "$j" --album "$j";
done
I added the script to cron and run it every night at 2am. I end up with a directory full of compressed, bookmarkable radio shows! But, to get them to the iPod, I had to manually drag them into iTunes every few days and synch the device.
Automating the Process
Then iTunes 5 came out with podcast support and I wondered if I could somehow automate the process. It turns out I can! I started with Apple’s instructions for how to create an
http://phobos.apple.com/static/iTunesRSS.html
From the examples at that site, I cut down fields until I got down to the bare minimum that would still function. Note that the site does not say you can use .m4b files, but I just tried it and it worked. Now I needed a method of creating an RSS file that specified my files. I whipped up a brute force script that runs right after the nightly conversions and just echoes every .m4b file in the radiofiles directory surrounded by enough XML formatting to make it a proper RSS Podcast feed.
Here’s that script (podcast-rss-maker.sh):
DATE=`date -R`
echo "< ?xml version=\"1.0\" encoding=\"UTF-8\"?>"
echo "<rss xmlns:itunes=\"http://example.com/DTDs/Podcast-1.0.dtd\" version=\"2.0\">"
echo "<channel>"
echo "<title>Private NPR Collection</title>"
echo "<link>http://192.168.1.88/radiofiles</link>"for i in /var/www/html/blog/radio/*.m4b
doj=`basename "$i" .m4b`;
echo "<item>"
echo " <title>"$j"</title>"
echo " <enclosure url=\"http://192.168.1.88/radiofiles/"$j".m4b\" type=\"audio/x-m4b\"
/>"
echo " <guid>http://192.168.1.88/radiofiles/"$j.m4b"\"</guid>"
echo " <pubdate>"$DATE"</pubdate>"
echo "</item>"done
echo "</channel>"
echo "</rss>"
This will create output that looks like this:
< ?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:itunes="http://example.com/DTDs/Podcast-1.0.dtd" version="2.0">
<channel>
<title>Private NPR Collection</title>
<link>http://192.168.1.88/radiofiles</link>
<item>
<title>APrairieHomeCompanion_Sat_10_01_17_00</title>
<enclosure url="http://192.168.1.88/radiofiles/APrairieHomeCompanion_Sat_10_01_17_0
0.m4b" type="audio/x-m4b" />
<guid>http://192.168.1.88/radiofiles/APrairieHomeCompanion_Sat_10_01_17_00.m4b"
<pubdate>Sat, 22 Oct 2005 11:54:46 -0700</pubdate>
</guid></item>
<item>
<title>APrairieHomeCompanion_Sat_10_08_17_00</title>
<enclosure url="http://192.168.1.88/radiofiles/APrairieHomeCompanion_Sat_10_08_17_0
0.m4b" type="audio/x-m4b" />
<guid>http://192.168.1.88/radiofiles/APrairieHomeCompanion_Sat_10_08_17_00.m4b"
<pubdate>Sat, 22 Oct 2005 11:54:46 -0700</pubdate>
</guid></item>[ ... ]
</channel>
</rss>
Now just add this line to the end of the sound-convert.sh file:
podcast-rss-maker.sh > /var/www/html/radiofiles/podcast.xml
Finally, go to iTunes, Select the Advanced Menu – Subscribe to Podcast…:
Then enter the URL for the podcast.xml file:
Now the listing shows up in your Podcasts Menu in iTunes:
I hope this all works for you too!
Issues
There are a few things I need to clean up and understand better.
First, my RSS file grows without bound as new files are added to the directory. This is a larger problem as I need some way to clean out old files that I’ve already heard. Since the RSS file only contains the files that are present, once I solve the bigger problem, this won’t be a problem for the RSS file anymore.
I don’t really understand how iTunes decides which files to download fully when it subscribes to a new Podcast. I bet it has something to do with the pubdate attribute which I am blasting with the newest date each time I run the script. I could do something smarter and pull the date from the file if it already exists. That would take being smarter though… 🙂
I always mess up the pathing when I add something to cron. Be careful to add absolute paths to the scripts you add to cron.
Please leave comments with ways I could make this better!
UPDATE v 2.0!
Logan Browne mailed me (again!) a much cleaner version of rss-maker.sh (thanks Logan!):
#!/bin/sh
# credit for this script goes to Declan
# http://www.declan.net/?p=160DATE=`date +"%a, %e %b %Y %H:%M:%S %Z"`
URI="http://localhost/~logan/Radio/"
MYDIR="/Users/logan/Sites/Radio/"
TITLE="Logan's Local Feed"echo "< ?xml version=\"1.0\" encoding=\"UTF-8\"?>"
echo "<rss xmlns:itunes=\"http://example.com/DTDs/Podcast-1.0.dtd\" version=\"2.0\">"
echo "<channel>"
echo "<title>"${TITLE}"</title>"
echo "<link>"$URI"</link>"
for i in ${MYDIR}*.mp3
do
j=`basename "$i" .mp3`;
k=`echo $j | sed -f urlencode.sed`;
l="audio/mpeg";# grab the creation date as a string and format it to RFC 2822
DATE=`stat -f "%Sc" -t "%a, %e %b %Y %H:%M:%S %Z" "$i"`echo "<item>"
echo " <title>"$j"</title>"
echo " <enclosure url=\""$URI$k".mp3\" type=\""$l"\" />"
echo " <guid>\""$URI$k".mp3\"</guid>"
echo " <pubdate>"$DATE"</pubdate>"
echo "</item>"done
echo "</channel>"
echo "</rss>"
Here are the contents of the urlencode.sed file to handle special characters:
s/%/%25/g
s/ /%20/g
s/ /%09/g
s/!/%21/g
s/"/%22/g
s/#/%23/g
s/\$/%24/g
s/\&/%26/g
s/'\''/%27/g
s/(/%28/g
s/)/%29/g
s/\*/%2a/g
s/+/%2b/g
s/,/%2c/g
s/-/%2d/g
s/\./%2e/g
s/\//%2f/g
s/:/%3a/g
s/;/%3b/g
s//%3e/g
s/?/%3f/g
s/@/%40/g
s/\[/%5b/g
s/\\/%5c/g
s/\]/%5d/g
s/\^/%5e/g
s/_/%5f/g
s/`/%60/g
s/{/%7b/g
s/|/%7c/g
s/}/%7d/g
s/~/%7e/g
find /var/www/html/radiofiles -mtime +7 -exec rm -f ‘{}’ \;
Add this to your crontab or the RSS script to delete any files older than 7 days in the /var/www/html/radiofiles directory. You will want to add something like ‘-name \*.m4b’ if you have other files in the directory.
Brilliant! This is a teriffic ‘fair use’ hack you’ve come up with – I’m on a Mac, but with my RadioShark and your instructions, including the podcast RSS maker shell script.
It’s only a matter of time before someone mods this technique to grab shows off the TiVo, convert them and dump them onto an iPod via iTunes.
I added a few additional variables to your podcast-rss-maker.sh script to make it easier to move around on my servers. I’ve also added accurate file date stamping and am looking into better handling of RFC 1738 URL encoding.
I’ll send you the script if you’d like.
[…] Radio Capture and Self Podcasting Filed under: Declan, Fun, Tech — Administrator @ 10:07 am […]
I got the radioSHARK working under linux.
http://wiki.linuxquestions.org/wiki/Audio
Here’s another version of the script that is written in Perl. It tries to pull out the ID3 Title and Artist.
—
#!/usr/bin/perl -w
use strict;
use Image::ExifTool qw(ImageInfo);
use HTML::Entities qw(:DEFAULT encode_entities_numeric);
use URI::Escape;
use File::stat;
use POSIX qw(strftime);
# Make an iTunes/RSS(?) feed for a directory of MP3 files.
# Perl version by Jason Luther
# Rewrite of Declan Fleming’s shell script at http://www.declan.net/?p=160;
my $feed_title = ‘Feed Title’;
my $url = ‘http://www.example.com/directory’;
my $dir = ‘/var/www/html/directory’;
print “Content-type: application/rss+xml\n\n”;
print ”;
print “\n\n”;
print “\n”;
print “$feed_title\n”;
print “$url\n”;
opendir(DIR, “$dir”) or die “opendir: $!”;
my @files = grep { /\.mp3$/ } readdir(DIR);
closedir(DIR);
my %mtime_by_file;
foreach my $file (@files) {
my $sb = stat(“$dir/$file”);
my $mtime = $sb->mtime;
$mtime_by_file{$file} = $mtime;
}
my @sorted_files = sort { $mtime_by_file{$b} $mtime_by_file{$a} } @files;
foreach my $file (@sorted_files) {
my $file_url = “$url/” . uri_escape($file);
my $date = strftime “%a, %e %b %Y %H:%M:%S %Z”, localtime $mtime_by_file{$file};
my $title = $file;
my $artist;
my $info = ImageInfo(“$dir/$file”);
if ($info) {
$title = encode_entities_numeric($info->{Title}) if exists $info->{Title};
$artist = encode_entities_numeric($info->{Artist}) if exists $info->{Artist};
}
print “\n”;
print ” $title\n”;
print ” \n”;
print ” $file_url\n”;
print ” $date\n”;
print ” $artist\n” if defined $artist;
print “\n”;
}
print “\n”;
print “\n”;
Thanks Jason!
[…] Declan’s Blog »Blog Archive » Radio Capture and Self Podcasting […]
Declan,
My version of the RadioShark installed on Windows XP does record in the WMA compressed format. Actually, the software lets me choose between 48kbps, 96kbps or 128kbps.
I do not have your problem because my mp3 player is not an iPod, so it will play WMA formatted files. I would like if the RadioShark did give you the mp3 option because one day I might want to make the plung to the world of iPod mania, but until then, the recorded radio programs
in the 96 or 128 speed format works just fine.
I do have a problem with my RadioShark that I have not resolved yet. I usually check my RadioShark files about once a week, and once in a while many of the files are only about 28 kb in size. Sometimes it is every recording since that first one that malfunctioned, but sometimes the computer issue that caused it resolved itself and some of the recorded programs after the malfunctioned recordings are of normal size. ( 26kb, 36kb, etc…)
I usually just restart my P4/2000mhz WinXP computer and it doesn’t happen for weeks.
Good luck, Dino.
Thanks for the great write-up! My situation was slightly different, in that I had to use FTP to transfer my files up to the server, and I used a different library for encoding the audio.
I wrote up my own process (with a link back here) in this (gratuitous blog) article: http://drjason.com/?q=node/85
Jason
[…] then make them available as a Podcast so iTunes can automatically sync them to your iPod.” Link. Clever hack to get all the podcasts that aren’t in the iTunes directory from NPR. Share […]