Archive for August, 2008

Aug 31 2008

Testing My SDBlogger Yahoo Pipe

Published by under Declan,Tech

I’ve been learning Yahoo Pipes to see if I could add more info, specifically the name of the blog, to the SDBlogger Twitter feed that itself is fed from a FriendFeed RSS feed.

Follow my progress at http://twitter.com/declantest

This all started when Matt Browne set up a FriendFeed room for SDBloggers, a gathering of rss feeds from local bloggers’ sites. Matt then set up an RSS to Twitter feed using Twitterfeed.com that took the RSS from the room and posted it as the Twitter user SDBloggers. By following this user, people could get a tweet telling them that something new had been posted on any of the blogs in the room.

Now, there are over 50 different blogs in the room, any of which could alert Twitter that it had a new post. The problem I had was that the tweet looked like this:

I couldn’t tell by looking at the tweet which blog it was referring to. The “(via Blog)” part of the title was generic and the same on every tweet, regardless of the sender. I mentioned to Matt that he should change that. Manager that he is, he tasked me with looking into how to do it… 😉

I knew from the raw HTML on the Friendfeed site that the name of each entry’s blog was available. And when I looked at the RSS feed provided by Friendfeed, all the parts where there. The item.content was made up of a lot of HTML and included some consistent text – specifically “an entry from” followed by an “a” tag that changed per entry, then the closing “a” tag. I just needed to parse the RSS, rip the blog name out of item.content, then replace “(via Blog)” with “(via ” + real blog name in the item.title.

At first I started with a perl script I found called RSS2Twitter. After spending a good part of a day off re-learning perl for the 17th time, I said, “heck, let’s learn something new…” and started playing with Yahoo Pipes.

I’m technical, but I’m not a programmer, so I usually start with some example and hack a piece of code until it does what I want. The initial problem I had with Pipes is that I wasn’t sure what context I was in. I knew I wanted RSS in and modified RSS out, but I was confused how to go about it. I looked at a couple examples and figured out I needed to start with a Fetch Feed object, and to stuff the Friendfeed RSS URL into it. The debug window was a bunch of help, showing me what the output of each object was as I clicked on Refresh.

So, now I had a list of RSS entries. I knew where my pieces were, I just couldn’t figure out how to extract them. I Googled “pipes tutorials strings” and the first link was from Daybarr.com and helped a ton. I needed to copy the item.content to a temporary variable so I could do some work on cutting it up without hurting the original item.content. From Daybarr’s page, I saw to use the “Rename” object and invoke the Copy As method (not sure that’s proper programmer talk) to replicate the value of item.content’s to my new variable called content_holder.

Here’s where it really gets ugly. I hadn’t touched regular expressions in a long time, but I’d need one to isolate the blog name from all the HTML. Another few Googles gave me some tutorials, and I settled on one from Apple. The intro mentioned Reggy, a regex validator that lets you write ugly little regex code in the top pane, paste some text into the bottom pane, and see the results in real time. It really helped:

You can see the regex I used in a Pipes Regex object in the top pane of Reggy above. What I’m doing is finding whatever is in that space, shoving it into the regex variable $5, then replacing the whole of content_holder with $5. It seems kludgy, and it probably is, but it works. 🙂 I use another line in the regex object to strip the generic “(via Blog)” text from the item.title.

Believe it or not, the next Pipes object was the hardest to get my head around. It helps a lot to cursor over an object’s connection points to see what input and output is expected and delivered. I did this for a long time before I read Daybarr’s loop example. The Loop objects consumes items and lets you perform operations on them individually, as opposed to the whole stream of items at once. The Loop object is modular in that it allowed me to plug a String Builder object into it. Once I understood that I had access to each item’s fields, with the blog name stuffed into my content_holder variable, all I had to do was add string parts to the String Builder in the order I wanted them to be concatenated. The last little trick in the Loop object was to assign the results to item.title.

I’m not really clear why everything else just gets passed through untouched, but that’s the context problem I was struggling with. Once I stopped worrying about all the details and concentrated on the parts I knew I wanted to change, I made a lot of progress. It’s not how I normally work. Understanding the boundaries of a system usually helps me know which way to start a solution. Assuming that “it’ll all be ok” doesn’t occur to me naturally.

One last Regex object to strip the content_holder variable out of the resulting RSS, and it’s all clean.

Here’s the Pipe:

I fed this to Twitterfeed.com and it fails validation, BUT after waiting the 30 minute delay, I got results!

Here you can see the name of the blog (in this case it’s mine for testing) right in the tweet!

Ok, it doesn’t seem like that big of a deal, but I learned something new and now I have a blog post I can go back to when I forget it all! 🙂

My regex is probably flawed. If there were a blog with “an entry from” in the title, I bet it would fail. And I didn’t use any “[” or “^” so I’m not very cool.

How would you have gone at this problem differently? How have you learned Pipes?

8 responses so far

Aug 28 2008

Man, I love little cakes

3 responses so far

Aug 28 2008

Thirteen

Published by under Erin,Photography

Wow, 13! Happy Birthday no-longer-Little Girl!

One response so far

Aug 23 2008

Is Schmap a Scam?

Published by under Declan,Photography,San Diego

I take a lot of pictures. I’m an amateur with a decent camera, so some of them are ok. I’ve never seriously pursued getting any of them published, but I’ve thought about it.

So I was pretty thrilled to get this email a few months ago:

:: Schmap: San Diego Photo Short-list
Hi Declan,
I am writing to let you know that one of your photos has
been short-listed for inclusion in the fifth edition of our
Schmap San Diego Guide, to be published late July 2008.
http://www.schmap.com/shortlist/p=35034345554N01/c=SH20282878
Clicking this link will take you to a page where you can:
i) See which of your photos has been short-listed.
ii) Submit or withdraw your photo from our final selection
phase.
iii) Learn how we credit photos in our Schmap Guides.
iv) Browse online or download the fourth edition of our
Schmap San Diego Guide.
While we offer no payment for publication, many
photographers are pleased to submit their photos, as Schmap
Guides give their work recognition and wide exposure, and
are free of charge to readers. Photos are published at a
maximum width of 150 pixels, are clearly attributed, and
link to high-resolution originals at Flickr.
Our submission deadline is Wednesday, 2 July. If you happen
to be reading this message after this date, please still
click on the link above (our Schmap Guides are updated
frequently – photos submitted after this deadline will be
considered for later releases).
Best regards,
Emma Williams,
Managing Editor, Schmap Guides

The link went to a permissions page to use the photo that didn’t seem bad. I Googled around to see if anyone else had dealt with them and looked at their site – and it all looked ok. So I said sure. Hey, I’ve got an ego… 😉

A few weeks later I get this in the mail:

:: Schmap San Diego Fifth Edition: Photo Inclusion
Hi Declan,
I am delighted to let you know that your submitted photo
has been selected for inclusion in the newly released fifth
edition of our Schmap San Diego Guide:
Downtown San Diego
http://www.schmap.com/sandiego/tours_tour2/p=2028D01/i=2028D01_29.jpg
If you like the guide and have a website, blog or personal
page, then please also check out the customizable
widgetized versions of our Schmap San Diego Guide, complete
with your published photo:
http://www.schmap.com/guidewidgets/p=35034345554N01/c=SH20281996
Thanks so much for letting us include your photo – please
enjoy the guide!
Best regards,
Emma Williams,
Managing Editor, Schmap Guides

Pretty cool! Indeed, if you go to http://www.schmap.com/sandiego/tours_tour2/p=2028D01/i=2028D01_29.jpg you will see some text about San Diego with my picture up at the top left.

I still had the feeling that this was too good to be true. The picture is an ok San Diego skyline, but it’s not great. There’s too much water for the frame size and the proportions are kinda dinky. Yep, them are pro terms… 😉 The main thing that was bugging me is that the URL they gave me had a specific URL in it.

So, I went to the root Schmap site and surfed to the San Diego map, explicitly NOT using their URL. As I suspected, I couldn’t find my picture at all.

I’m guessing that Schmap plays on the desire to have one’s picture noticed so that the photographer will link to their site, thus elevating their page rank. Rather than a promotion of my picture, they are using my desire to have my picture used to further their site with that widget.

Scam might be too harsh of a term… This is a form of mutual promotion, but it feels less than honest on Schmap’s part not to include the photo in a general search for San Diego.

21 responses so far

Aug 19 2008

Souplantation Ice Cream

Published by under stuff i didn't eat

And they have sprinkles. I love sprinkles.

One response so far

Aug 18 2008

Red Island Repository Institute – PEI Trip

I just got back from a week on Prince Edward Island in Canada. I was up there for an intense, 5 day introduction to Fedora (the digital library repository, not the RedHat variant).

I haven’t processed the content yet, but I did take a lot of pictures. What encapsulates a trip better than a grown man in an Anne of Green Gables hat and braids?:

Jim Tuttle is so silly… I wouldn’t be caught dead…

Dang digital cameras…

PEI is famous for its lobster, and there is good reason:

They are served almost everywhere. We had these on Monday night:

We also had a few tasty beverages. Here Grant and Jim listen politely to Mike tell a story. The politeness comes from not understanding a thing he was saying due to a very strong Boston accent:

or a lot of beer.

The University of PEI’s University Librarian, Mark Leggott, took us all on a walk to the red sand beach:

which he said was lovely. We had to take his word for it because there were no lights, the moon was covered in cloud, and we were all dizzy from blood loss to mosquitoes.

Thankfully, there was a bar at the end of the walk, so we could hide from insects and Chris could get a drink:

For lunch on Wednesday, we went to the local farmers market:

where three of the locals – Paul, Peter, and Alex demonstrated the East Coast Canadian habit of stopping and standing in the middle of a walk way. I’m not kidding, these people have getting in my way down to a science.

Then we walked the length of the island on a path:

Ok, not really, but the path does stretch from tip to tip. Maybe someday I’ll bring my bike up there.

The locals bath when the sewers back up and spew what can only be raw sewage 6 feet into the air:

Or it was a clever fountain.

This was a small petting zoo for crabs and lobsters. Notice no one putting their hands in:

Jim was then sent home for “disturbing” the local livestock:

Ok, so the water and boats and beautiful:

But what is prettier than a man sized potato?

I’m talking about the one in the back…

On Weds night, we were sitting at dinner and I got the Lensbaby out and got some nice shots:

Here’s Richard Green from Hull:

Peter Binkley from Canadia:

And Mark Leggott again:

The flowers really add to the scene… 😉

On Thursday night, Grant and his daughter brought us down to see Victoria Bay:

Note the famous, ubiquitous red soil.

I got a little bit of the sunset:

Then the rain rolled in:

On Saturday, Jim and I got a car and drove from Charlottetown as far East as possible and looped back.

PEI has many lighthouses:

And pretty, red cliffs:

And pretty streams:

And more light houses:

That one is on East Point, the farthest we could go East.

Jim was too cool for all this:

And I was eaten alive by mosquitoes:

Then we went to see Pineapple Express. All in all, a pretty good day! 🙂

Here are a lot more pictures (Flickr link):

One response so far

Aug 16 2008

Compost?

Published by under Declan,Photography

Canada has too many trash options. What goes in “Compost?” This is too much to think about…

2 responses so far

Aug 14 2008

Eeeevil Haagen-Dasz and Croissants

PEI tempts me yet again!

No responses yet

Aug 14 2008

Blending in on Prince Edward Island

Published by under Declan,Fun,Photography

I think Jim got a better picture, but here I am auditioning for Anne of Green Gables.

Here’s a better image from Jim:

One response so far

Aug 12 2008

Skipping the Cookies

Published by under stuff i didn't eat

Man, those brownies are pretty…

No responses yet

Next »