At QNH we are always looking for ways to put the company name and knowledge out there on the internet. For quite a while we had been pondering the idea of starting a company weblog to share that knowledge with the on-line community. The thing that held us back was the difficulty of getting our engineers to post on this company weblog. Lost of our engineers have their own personal weblogs and we did not want them having to copy-paste each and every relevant blog entry into the company weblog. While trying to come up with a solution to this problem, one of our senior software engineers and architect stumbled upon Yahoo Pipes.

Yahoo Pipes according to Yahoo

"Pipes is a powerful composition tool to aggregate, manipulate, and mashup content from around the web".
Like Unix pipes, simple commands can be combined together to create output that meets your needs:

  • combine many feeds into one, then sort, filter and translate it.
  • geocode your favorite feeds and browse the items on an interactive map.
  • power widgets/badges on your web site.

  • grab the output of any Pipes as RSS, JSON, KML, and other formats.

Getting Started on combining RSS feeds

Go to http://pipes.yahoo.com and sign-up, or use a yahoo account you already own.
We decided not to use a personal account, but opted to creating a new company account. The company account is

Creating a Pipe

Create a new Pipe and open it, and click the "Edit Source" Tab.
The Pipes Editor is shown.

  1. Add a "Fetch Feed" source from the left menu and add some RSS 2.0 feeds as URL's
  2. Add a "Filter" operator from the left
  3. Connect the "Fetch Feed" To The "Filter" by dragging and dropping a connection starting from the fetch feed
  4. Because we only want posts containing the category QNH to be aggregated we configure the filter as follows:
    - Permit items that match All of the following
    - Add a Rule: Item.Category contains QNH
    - Add a Rule: Item.Category contains qnh

    The editor should look something like this

 

 

 

 

 

 

 

 

 

 

If we connect the filter to the Pipe Output we would get all entries of the blogs defined in the Fetch Feed that have a category named QNH.
But now we want to cap the description of every entry to 500 characters and add "..." at the end of the description. We also want to get rid of some illegal characters some blogs create.
These are the steps required.

  1.  Add a Loop Operator.
    - For each item.description in input feed
    - Add a Sub String String (from 0 to 500)
    -     Assign results to item.description
  2. Add a RegEx Operator and add the following rules
    - item.description replace (.*)(\[...\]) with $1
    - item.description replace <pre.*?> with empty (empty field)
    - item.description replace <img.*?> with empty (empty field)
  3. Add a Loop Operator.
    - For each (empty field) in imput feed
    - Add a String Builder String (item.description + ...) and assign the results to the item.description

The part we just created should look like the next picture.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Now we are almost done. But because some weblog engines put the author in the item.dc.publisher or in the item.dc.creator in stead if the item.author.name we need to correct this.

  1. Add a Loop Operator.
    - For each (empty field) in imput feed
    - Add a String Builder String (item.dc.publisher + item.dc.author + item.author.name) and assign the results to the item.author.name
  2. Add a Sort Operator
    -     sort by item.pubDate in descending order
  3. Connect the sort operator to the pipe output

The last part created should now look like

 

 

 

 

 

 

 

 

 

 

 

 

 

 Save the pipe! 

End result


Now we end up with an agregated RSS feed that we can integrate into any part of the organisation.
For example: The feed is featured on the QNH AD&S website http://www.qnh-ads.nl/blog, but also shown on the homepage of our company intranet,

Good times!