Using YQL to build a web app in a few hours

1st November 2009

A couple of years ago I thought about building an ultra-simple site for browsing music videos with ease. The idea being that users may enjoy exploring artist's back catalogues, viewing not just a band's biggest hits, but also their lesser known tracks.

The only functions I deemed necessary for an alpha release were artist search, artist video list and links to similar artists... that was it!

Unfortunately the priority on this project was low, in fact I had forgotten about it altogether until I watched a talk by Christian Heilman of Yahoo at the StackOverflow DevDays conference in London.

Christian's talk detailed how the Yahoo Query Language (YQL) can be used to grab all kinds of Yahoo data using SQL syntax. On the train home I browsed the available tables and noticed the ability to search both musical artists and music videos which triggered my memory of the project I had imagined.

This afternoon I began work on this mini project and it is now ready to be deployed (did I mention it was v.simple?).

Using YQL

My first task was to find YQL queries which would help with my task.

First, the user must be able to search for an artist. The form data will be inserted into this query.

YQL artist search

select * from music.artist.search where keyword='madonna'

Next, after a search result is clicked I need to gather the artist information. The other music tables use the Yahoo artist ID to link data and so I will need to use it as my primary identifier. It is important to note that although YQL looks like SQL it is not as flexible when retrieving data. E.g. you cannot simply search for all musical artists who have released more than 20 records, you must stick to the keys specified which in this case is artist ID.

Get artist info from artist Id

select * from music.artist.id where ids='289282'

This returns information such as artist name and the artist's website.

The next task was to show related artists, so that the user can navigate between related bands and explore etire genres of music.

Related artists

select * from music.artist.similar where id =''289282''

Unfortunately, in YQL, music videos cannot be searched using artist id. You can only search for videos by title. However, a large proportion of web videos are hosted on YouTube anyway. So I created a query to retrieve video clips from the Google owned service rather than Yahoo. There is a ton of info on the web about how to get videos from YouTube so I won't rewrite the same info here.

Finally, I added the results of a Yahoo image search based on artist name to the artist page. This means the user can now browse both videos and photos.

Image search

select * from search.images where query='artist_name' and mimetype like '%jpeg%'

After initially uploading the site I revisited it to add Colorbox, an excellent jQuery lightbox clone to display the images/videos and allow scrolling back and forth.

I then configured some search engine friendly Urls and added a logo which I grabbed from IconFinder

All done. The site is now live at http://vidsly.com

Issues?

The only issues I have spotted so far relate to ambiguous band names return strange images i.e. Eagles, Queen, Boston return mostly non-musical images. Also, many record companies seem to actively remove their artist's content from YouTube or at least disable embedding and so there is sometimes a lack of quality videos.

If you experience any issues or have feature requests feel free to leave a message in the comments.

Edit 03/11/09: Fixed a massive rendering issue with IE7 that made the lists of artists look ridiculous.