Keith Richards’ guitar gallery in 4 lines of code

January 24th, 2011, By eitanb

The first line of “code” would be this query, that returns a list of Keith Richards’s guitars (click to check it out):
http://api.headup.com/v1?raw=true&q=Keith Richards/popularmeaning/`instrument`/render(“videolist.html”)
Let’s break down the query to its parts:

  • Keith Richards/popularmeaning – gives us the URI (a unique ID) for Keith Richards, dbpedia:Keith_Richards.
  • `instrument` – that’s a fuzzy matching of the free-form text “instrument” with a predicate of dbpedia:Keith_Richards. We get a list of the instruments that Keith played.
  • Then we render this list of instruments as using a template (that we’ve prepared in advance) called videolist.html.
This is actually a more “fuzzy” way of querying the graph. The strict way of querying it would have been:
http://api.headup.com/v1?raw=true&q=dbpedia:Keith_Richards/dbpedia-owl:instrument/render(“videolist.html”)

The rest of the code resides inside the videolist.html template. Let’s have a look inside:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
  <title>SemantiNet's Video Portal</title>
  <link rel="stylesheet" href="http://www.blueprintcss.org/blueprint/screen.css">
  <style type="text/css" media="screen">
  body {background-color: #FAFAFF}
  </style>
</head>
<body>
 <div class="container">
 <div class="span-24">
  <table style="font-family: Consolas; font-size: x-small; background-color: white" class="span-24">
   <%foreach ./take(20)%>
  <tr width="100%">
   <td>
    <%select keytermsforquery/youtube:getplayers/first%>
   <iframe title="YouTube video player" class="youtube-player" type="text/html" width="300" height="230" src="http://www.youtube.com/embed/<%= f:v1/split('/')/*/at(4)%/>" frameborder="0">
   </iframe>
    </%select%>
   </td>
   <td>
   <h2><%= label%/></h2>
    <%= abstract/str:unescapeunicode%/>
   </td>
  </tr>
   </%foreach%>
  </table>
 </div>
 </div>
</body>
</html>

Most of the template is made of simple HTML markup. The query lines are:

  • keytermsforquery/youtube:getplayers/first
    • keytermsforquery - query refinements, for making calls to search APIs (YouTube in this case) –  adds semantic “cues” when querying APIs to get more accurate results.
    • youtube:getplayers – gets YouTube players for the entities.
    • Take only the first video, we want 1 for each instrument.
  • label – brings a nice readable name.
  • abstract/str:unescapeunicode – brings the Wikipedia abstract – and removes Unicode escaping.
In the same way, we can display a nice YouTube gallery of any list we’d like:

Introducing SemantiNet’s API

January 16th, 2011, By eitanb

We’re excited to release an alpha version of our new API. This is the first post in a series of blog posts, describing ways you can rapidly build impressive data intensive applications.
SemantiNet’s new API is designed for easy querying of a selected collection of useful Web Services, Wikipedia, Linked-Data, and the unstructured web. The API also provides a flexible templating language, for easy creation of semantic mashups and data intensive applications, directly from your browser.

To see what we mean – let’s start with a very simple querying of DBPedia (click the link to see this query in the playground):
The API returns a node that represents Bar Refaeli in DBPedia, with some of the data that is connected to this node. Quite simple so far. So, let’s devise a template that takes this node, and present information from this node, as an HTML:
<html>
  <body>
    <!-- label provides a nice representation of the node's name -->
    <h1><%= label%/></h1>
    <!-- personage calculates the age, based on information we have from the birth-date and the current time -->
    Age: <%= personage/round(1)%/><br/>
    <!-- We want a nice representation of the birthPlace, so we take dbpedia-owl:birthPlace/label -->
    Born in: <%= dbpedia-owl:birthPlace/label%/><br/>
    <!-- Take the first image returned from YahooBoss's images search -->
    <img src="<%= yahooboss:images/first%/>" width="100px">
  </body>
</html>
Click here to see it live in the playground


What’s returned from a couple of examples:

Nice. We’ve seen how to query both LinkedData (from DBPedia in this case) and the web (through Yahoo Boss) – to get the picture.
Just to get a feeling of what’s possible, let’s play with these models’ place of birth.
This query, will return the 3 birth places:
/multy('dbpedia:Carolyn_Murphy','dbpedia:Marisa_Miller','dbpedia:Bar_Refaeli')/dbpedia-owl:birthPlace
Using this template – we will take the places, and put them on a map:
<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <script src="http://maps.google.com/maps?file=api&amp;v=2&amp;key=ABQIAAAAkMzYJpqzT4X0Hj0W-xMFIhTkBdPb1_Y7shJWGA4g7zFU4DbUwRSRxPPUjb7uuS8U3pAZlGMUGn5Vww" type="text/javascript"></script>
  </head>
  <body>
    <div id="mapdivid" style="float: left; height: 100%; width: 100%"></div>
    <script type="text/javascript">
    if (GBrowserIsCompatible()) {
     var map = new GMap2(document.getElementById("mapdivid"));
     map.setCenter(new GLatLng(0, 0), 1);
     <%foreach .[location]%>
       map.addOverlay(new GMarker(new GLatLng(<%= /location/geo:lat%/>, <%= /location/geo:long%/>),{title:"<%= /label%/>"}));
     </%foreach%>
    }
    </script>
  </body>
</html>
Check it out “live” here. In this example, we iterate over the birthplaces, using a ‘foreach’ directive, and for each place – we take the location’s latitude, longtitude and label.

The following query, will return a list of female models, include only those that we have data about their height, and order them by their height:
category:Female_models/deepinstances(3)[dbpedia-owl:height]/orderdesc(dbpedia-owl:height)
To show a table of the tallest 10 models, we’ll use this template:
<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body>
    <table style="float: left; clear: both">
      <%foreach ./take(10)%>
        <tr>
          <td><img width='100px' src='<%= /yahooboss:images/first/%>'
          onerror='this.onerror = null; this.src="http://static1.headup.com/images/placeholder.jpg"' />
          </td>
          <td><%= /label%/><br/>
          <%= /dbpedia-owl:height%/>
          </td>
        </tr>
      </%foreach%>
    </table>
  </body>
</html>
Check it out in the playground here, or – for a more generic version of this table, check this out:

Scraping pages from accross the web – is quite sweet as well. Let’s do a call to this page:
http://sportsillustrated.cnn.com/2010_swimsuit/models/
And now, let’s scrape the images from this page, using SemantiNet’s API:
/fetch('http://sportsillustrated.cnn.com/2010_swimsuit/models/')/htmlxpath('//div[@class="cnnIndexList"]//img/@src')/*
This gets us a list of photos. And if we put them in this template:
<%foreach .%>
  <img src='<%= .%/>'/>
</%foreach%>
What this query results is a scraping of the site’s images:


In the following blogposts we’ll write about more characteristics of the SemantiNet’s API and language:

  • Extensible – anyone can easily extend this language (we internally call it CSlang), building on top of existing predicates
  • A rich NLP library for Entity extraction, contextual disambiguation, access to WordNet
  • Powerful first-class-citizens of the language – graph querying primitives, lambda calculus
  • Very short development feedback loop
  • Piping
  • Fuzzy ontology support – freeform queries allow quick trials and discovery
  • Inference rules one-liners
  • Enrich the ontholgy based on web-links
  • Data mining primitives – map-reduce, grouping, histograms, order-by, max on lists

Want to get started? Take a spin in the playground, and check out our wiki: http://wiki.headup.com/index.php?title=Knowledge_Graph_API

Free to use C# fix for the Seeqpod.com API

March 26th, 2009, By talk

Seeqpod pulls a fast one

A few hours ago we discovered that SeeqPod.com changed their API without notice. For us this meant that some of our audio playing capabilities were impaired. It seems they started using an authentication method that requires changing the HTTP Get request’s headers.

Seeqpod - The Headup summary

Seeqpod - The Headup summary

Our trusty development team, always up for a challenge, was put on code-red-alert mode to produce a quick fix so our users could resume hearing music ASAP. Half an hour ago team member Yuval Kesten finished writing the code that will enable us to utilize the new Seeqpod API and we’ve deceided to share his  solution with the world for everyone to enjoy.

The code changes for the Seeqpod API

The following C# (Silverlight CLR)  is free for anyone to use and enjoy (no copyrights).
Enjoy & pass it on…
: )

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Security.Cryptography;
using System.Net;
using System.IO;

namespace SeeqPod
{
class Program
{
static void Main(string[] args)
{
const string apiKey = “”;
const string apiUid = “”;

string timeStamp = GenerateTimeStamp();

HMACSHA1 hmac = new HMACSHA1();
hmac.Key = Encoding.UTF8.GetBytes(apiKey);
string data = “/api/v0.2/music/search/nirvana/” + timeStamp;
byte[] call = hmac.ComputeHash(Encoding.UTF8.GetBytes(data));
string sCall = BytesToHex(call);

var wc = HttpWebRequest.Create(“http://www.seeqpod.com/api/v0.2/music/search/nirvana/”);
wc.Headers.Add(“Seeqpod-uid”, apiUid);
wc.Headers.Add(“Seeqpod-call-signature”, sCall);
wc.Headers.Add(“Seeqpod-timestamp”, timeStamp);
var response = wc.GetResponse();

// Get the stream associated with the response.
Stream receiveStream = response.GetResponseStream();

// Pipes the stream to a higher level stream reader with the required encoding format.
StreamReader readStream = new StreamReader(receiveStream, Encoding.UTF8);

Console.WriteLine(“Response stream received.”);
Console.WriteLine(readStream.ReadToEnd());
response.Close();

Console.ReadKey();

}

private static string GenerateTimeStamp()
{
TimeSpan ts = (DateTime.UtcNow – new DateTime(1970, 1, 1, 0, 0, 0, 0));
return ts.TotalSeconds.ToString();
}

private static string BytesToHex(byte[] input)
{
StringBuilder sb = new StringBuilder();

foreach (byte b in input)
{
sb.Append(string.Format(“{0:x2}”, b));
}

return sb.ToString();
}
}
}

A detector for memory leaks on Silverlight applications

February 11th, 2009, By talk

Abstract: Currently there’s no memory profiler for Silverlight

If you’ve ever programmed in Silverlight, you’ve probably wondered whether your objects in memory were collected by the GC (garbage collector) or not. What if the objects you create stay in the memory for the whole life-time of your application – will you know?
Regular .NET memory profilers don’t work with Silverlight applications and therefore, if we have a memory problem and want to examine it, the only way to approach the problem is to convert our project to a WPF application, which is difficult in the best of cases, and in my scenario was nearly impossible.

When should you use a memory leak detector?

The memory leak detector I wrote is meant for use in those cases where you have the feeling one of the objects you wrote isn’t being collected when it should be by the GC. You can use the memory leak detector to monitor this behavior.

How to use the Silverlight application memory leaks detector?

To use my memory leak detector first add the object you want to monitor by calling the detector’s ‘AddReference’ method in the object’s constructor. Next, at a point where the object should have been already collected by the GC call the ‘Check’ method to verify that it was indeed collected and is no longer in memory.

How is the GC duped into collecting objects despite the fact that the memory leak detector has references to them?

The trick behind this memory leak detector is the usage of the ‘WeakReference’ class. ‘WeakReference’ is a class that provides the ability to reference an existing object in memory, without preventing the object from being collected by the GC. For more information about the ‘WeakReference’ class visit: http://msdn.microsoft.com/en-us/library/system.weakreference.aspx

The Code – an overview

The memory leak detector manages a main list named ‘elementsList’ consisting of ‘ObjectStruct’ elements. Each element contains a ‘WeakReference’ to the monitored object and a few other debug values. You can use the ‘StackTrace()’ in order to identify the monitored object’s creator. This field can help finding the source of the memory leak. The code is divided into three main static public methods. The first two must be called while the last one, ‘SignalDisposed’, is for further debugging.

Main Methods:

  • AddReference’ – Adds the object you want to monitor to the memory detector. You can choose whether you want to create the ‘StackTrace’ upon creating the object or not. The object’s constructor is a good place to call this method.
        [Conditional("DEBUG")]
        public static void AddReference(object obj)
        {
            lock (lockObject)
            {
                // avoiding dups - can happen when adding an element
and its base-class to the profiler
                if (elementsList.Any(p => p.Reference.IsAlive &&
p.Reference.Target == obj))
                    return;
                elementsList.Add(new ObjectStruct(new WeakReference(obj),
generationCounter, new StackTrace().ToString()));
                //elementsList.Add(new ObjectStruct(new WeakReference(obj),
generationCounter, null));
            }
        }

    -

  • Check’ – Use this method to check the memory condition. A good place for implementing this call would be an idle state of your application, where most of the objects already created can be disposed and collected. Each call to the ‘Check’ method increases the static generation counter by 1 providing us a method for following when the monitored objects where instantiated. The ‘Debugger.Break()’ command stops the application at a point allowing you to examine all objects alive in memory.
        [Conditional("DEBUG")]
        public static void Check()
        {
            lock (lockObject)
            {
                long memUsage = GC.GetTotalMemory(true);
                for (int i = 0; i < elementsList.Count; i++)
                    if (!elementsList[i].Reference.IsAlive)
                        elementsList.RemoveAt(i--);
            }
            generationCounter++;
            Debugger.Break();
        }

    -

  • SignalDisposed’ – My experience hunting for memory leaks with this detector during my work on our semantic web Firefox add-on, led me to the conclusion that there is value in having a Boolean variable indicating whether or not the dispose method of the monitored objects was called. To debug your leak, call this method within the ‘Dispose()’ method to detect scenarios where the ‘Dispose()’ method was called but the object remains in memory.
        public static void SignalDisposed(object obj)
        {
            lock (lockObject)
            {
                var objectsToSignal = elementsList.Where(p =>
p.Reference.IsAlive && p.Reference.Target.Equals(obj));
                if (objectsToSignal.Any() && objectsToSignal.Count() == 1)
                {
                    var objectToSignal = objectsToSignal.First();
                    objectToSignal.DisposedSignaled = true;
                }
            }
        }

-

Notes:

  1. All three functions have the ‘Conditional(“DEBUG”)’ attribute that allows using the memory leak detector freely in a project without worrying about performance in the release version of the application. For more information about the ‘Conditional’ attribute visit: http://msdn.microsoft.com/en-us/library/system.diagnostics.conditionalattribute.aspx
  2. The class supports multi-threading.

Ideas for future development:

Knowing which object holds a reference to the in memory monitored objects would be great…

This post was written by Amit Kanfer.

Older Posts »