Using the Google Custom Search API via jQuery

I've been using the Bing Search API for a while on my peter.hahndorf.eu site. The site is totally static, there are no server-side components. So a search had to be implemented in JavaScript. The code was relatively simple and Bing gives you an unlimited number of search queries which is great.

However recently I noted that the API calls didn't return any results at all, while the bing.com site itself returned the results as expected. I looked around a bit online and found out, that the API is not using the exact same data source as the web site and that several other people had problems with their results as well.

So I turned to the other search guys. I had used an older Google API on the server side a while ago but this time I wanted to replace the client side implementation.

I did not just want to place a Google created HTML fragment search box on my site, I wanted to get just the search results via JSON without any Google goo around it.

You can see the working search at peter.hahndorf.eu/search.html

They have something called the 'JSON/Atom Custom Search API', and I will describe how to use it:

To start with, you need a Google account, quiet likely you already have one. Next you need to define your own custom search engine. This allows you to search just your site not the whole web. Go to http://www.google.com/cse/manage/all and create a new search engine, follow their steps and you receive a 'Search engine unique ID' which you later need in your code.

Next you need an API key. Follow the instructions at http://code.google.com/apis/customsearch/v1/getting_started.html#get_account, the API key is the second parameter that you have to provide to Google every time you use the API. The free API plan only allows you 100 queries per day, which is enough for my small site but for bigger sites you may have to pay them.

You can now test your custom search by just putting the following in your browser's address bar:

https://www.googleapis.com/customsearch/v1?key=INSERT-YOUR-KEY&cx=INSERT-YOUR-ENGINE-ID&q=YOUR-SEARCH-TERM&alt=json

You should get a page with search results formatted in JSON.

Building a search page:

Let's start with the html:

<input type="text" id="txtSearchTerm" size="40" />
<button id="btnSearch" style="display:none;">Start Search</button>


<noscript>JavaScript is required for this page.</noscript>


<div id="searchResult"></div>

<div id="output"></div>


<div>
    <a href="#" id="lnkPrev" title="Display previous result page" style="display:none;">Previous</a> <span id="lblPageNumber" style="display:none;"></span> <a href="#" id="lnkNext" title="Display next result page" style="display:none;">Next</a>
</div>

Let's see, we start with an input field and a button. Because without JavaScript our search doesn't work we hide the button and display a message to the user using the 'noscript' tag.

Next we have three divs, the first 'searchResult' is to display a search result summary or an error message. The second one 'output' is for the actual results and the third one is for the navigation to the next or previous pages. The API returns up to 10 records per call and up to 110 records in total. I did not implement a number of links to jump to the result pages directly. I feel the 'Next' button is good enough.

We also need to include our script file and jQuery, I always put these at the end of the page:

<script type="text/javascript" src="jquery.js"></script>


<script type="text/javascript" src="googlesearch.js"></script>

When developing the script you will do a lot testing and may run into the 100 queries per day limit, to work around this I created a local dummy result page. Use the url from before but instead of the &alt=json parameter at the end, use &callback=SearchCompleted. Also make sure that the search term you are using returns more than 10 results.

https://www.googleapis.com/customsearch/v1?key=INSERT-YOUR-KEY&cx=INSERT-YOUR-ENGINE-ID&q=YOUR-SEARCH-TERM& callback=SearchCompleted

now you get a slightly different result which is JSONP, it wraps the JSON data in a JavaScript function call. Save the page into a text file named dummy.js on your development server. Rather than calling Google for the results, we will now use this file.

Create a 'googlesearch.js' file for our logic, start with the document ready function:

$(function ()
{
    $('#btnSearch').show().click(function () { Search($("#txtSearchTerm").val(),0);});
    $('#lnkPrev').click(function () { Search($("#txtSearchTerm").val(),-1); });
    $('#lnkNext').click(function () { Search($("#txtSearchTerm").val(),1);  });
});

We just set up the event handlers for the links and the button which we also unhide, the second parameter 0,-1 and 1 is for paging.

Next create a search function which initiates the API call:

function Search(term, direction)
{
    url = "http://localhost/dummy.js?callback=?";
    $.getJSON(url, '', SearchCompleted);
}

The url points to our dummy file, notice the callback=? part as a parameter. It surely doesn't make any difference in our static file what parameters we call it with but we still need it. It tells the jQuery Ajax magic to treat the result as JSONP and execute it after receiving it. Naming the dummy file *.js tells the web server to send the file content with a Content-Type header of 'application/x-javascript', which is required for JSONP.

So what actually happens here? Just doing an AJAX call for the JSON data from Google does not work because we can only do AJAX calls to our own domain, not a different one like Google.com. To work around this JSONP wraps the data in JavaScript because the <script> tag in html allows the src attribute to point to a different domain. JQuery executes the JavaScript received from the server. It does nothing else than calling our callback function passing in the JSON data as the only parameter. You can see that in our dummy.js file.

The simplest version of the SearchCompleted function would look like this:

function SearchCompleted(response)
{
    var html = "";
    for (var i = 0; i < response.items.length; i++)
    {
        html += response.items[i]. htmlTitle + "<br />";
    }
    $("#output").html(html);
}

We loop through the JSON data and build up an HTML string which we then display in our output div.

Most likely we want to do a bit more with the results, like linking back to the actual page. Look at our dummy.js file to see all the data we get from Google: ] .htmlTitle, .link and .htmlSnippet are the most useful ones.

You can look at my actual implementation to see an example of how to massage the search results before displaying them. Peter.hahndorf.eu/search.html and peter.hahndorf.eu/css/googlesearch.js

Using the real thing

Now that our results look okay, we can switch over to the Google results, we need to change the url in the Search function:

var url = "https://www.googleapis.com/customsearch/v1?key="+ mGoogleApiKey + "&num=10&cx=" + mGoogleCustomSearchKey + "&start=" + startIndex + "&q=" + escape(term) + "&callback=?";

mGoogleApiKey and mGoogleCustomSearchKey are two variables that I set elsewhere with my real values. The start parameter is needed for paging and we again need the callback=? to tell JQuery to do its JSONP magic.

Paging:

So far we always only get the first 10 results. So lets add some basic paging. Normally this include quite a bit of logic, luckily Google provides some help. If you look at the dummy.js file, you see 'nextPage' and 'request' under 'queries'. Here we can see the total results in (response.queries.request[0].totalResults) and the start index for the next page (response.queries.nextPage[0].startIndex). All we have to do is checking whether there is such a value and then unhide the appropriate link and remember the StartIndex value for the next Ajax call.

Again look at peter.hahndorf.eu/css/googlesearch.js for the code.