I have created an AIR runtime version of my SPARQL browser.
You can download the executable here.
You will need to install the AIR runtime to use it. The reason for doing this now was that I realised that the sort of people who would use this are not experts in setting up a web-server etc.
There is one feature which is a bit of pain and that is setting up the queries file. In theory it would be loadable in a sort of “file..open” kind of way. I have not had time to implement or think this through so for the moment you will have to edit the config file which is bundled in the install. There is a howto on the google code site here.
I have had a look around at the other kind of graphical tools for SPARQL which came in the wake of this one and I still haven’t found anything that is as general as this.
The online version is currently broken. This is a direct side-effect of me moving this website and all it’s files to a new provider. I do plan to fix this, just not right away. The application uses a proxy-script to issue requests and I am not sure how to set this up on my new webspace quite yet.
More changes to the SPARQLbrowser.
Right click to view/download the source code.
- Datagrid columns are rebuilt on each query. Each column represents one of the query bindings.
- Cancel query button. This is only active when the query is running.
There are bugs:
- Datagrid columns sort isn’t sorting on the column contents. It’s a long story but basically because I am building the grid columns from scratch I seem to have run into a problem with missing behaviour which the mxml version does not exhibit.
- Sometimes (but annoyingly not always) the graph just sits there twitching away and does not expand to fill the available space. The reason for this is something to do with the canvas area it has to draw on shrinking. The solution is to change the size of the browser window. The problem will immediately go away. No doubt there is a Noddy fix for this.
The world most esoteric Flex application just got better! I have fixed the issue with the datagrid wordwrap and cell height. Just to recap – if the wordwrap was switched on then long text in a grid cell would push the datagrid off the page!!! Do you know how I fixed this? I heard a colleague at work tallking about setting the minHeight property to zero. I was busy at the time but I knew he was trying to fix a datagrid formatting issue so I thought to myself tonight ‘ho hum..lets see what happens if I set minHeight to zero’. Well it works. No idea why. I will have a chat to the developer tomorrow.The other grid sorting bug remains. Still I remain hopeful. There is always a way (sometimes).
Some cosmetic changes. I have moved the ‘cancel query’ button over to the left and taken out the ‘execute’ label.
I have been making some changes to the SPARQL browser.
Right click to view/download the source code. This is Flexbuilder 3 project only. Note I have made a slight modification to the BirdEye library so you can’t just use the latest swc file from their site you will have to use the one bundled with the project.
- Uses the latest version of the Birdeye graphing component(as of 19/08/08).
- Displays tuples in result grid
- Directed edges
- Edge labels for binding names
- New node renderer uses colour codes to indicate type. Blue is primary result URI, red is a literal and green a URI which you can click on and view in the Browse tab.
Also, I have fixed an issue where queries were failing if you chose a different endpoint. This should work with most SPARQL endpoints now.
This now has added ‘favicon’ goodness!
After all the graft required to add support for 4,8 and 24bpp bitmap types to the flexlib IconLoader class I wasn’t sure if it was going to be worth all the groundwork but I have to say I reckon this looks pretty good.
You can find a full FlexBuilder2 project here.
You can find a full FlexBuilder3 project here.
These are complete the complete projects. If you have FB2 or Fb3 the instructions to get them to run are:
- Unzip the whole project to a directory on your computer.
- Boot up Flexbuilder (2 or 3)
- Right click on the projects view and select ‘Import’.
- Select the ‘Existing projects into workspace’
- Set the root directory to where you unzipped the project. <dir>/social-graph-viewer. You should see the project name’SocialGraphViewer.
- Press ‘Finish’. This will import the whole project which should be ready to run.
Probably the last update I’ll make. I have added the ability to double cllick on a favicon and it will requery the Google API for the supplied URI. I.e. it will perform a search on that node. I wanted to implement back and forward buttons for the queries but I find that I’ve run out of steam. If you want to go back you must select ‘Edges In’, requery where you came from and it should appear as one of the nodes.
There is a whole bunch of stuff I am supposed to be getting on with as regards the IFrame component but I can’t help myself from coming back to this application and tweaking it.
I have now hooked the grid into the visible nodes of the graph so the grid rows represent all the nodes you can see. I have removed the ‘double-click to navigate’ from the node icons and instead you now double click the row in the grid to indicate where you want to navigate to. Remember your original root node is always in the list but if you navigate away from this you should always be able to go back by selecting ‘Edges In’ and double clicking the node you jumped from. As I have said before this interface is pretty experimental and I am just trying out new things. The best ideas should stick.
Some of the favicons for obvious sites like ‘linkedin’ weren’t getting loaded. This was due to a fault in the regular expression used to parse the source url and turn it into one which can fetch the favicon for the site. This was the reason for some of the blank favicons on the end of nodes. The other that some sites use GIF files and not ICO files. Currently GIF files are not handled.
I have made quite a few fixes to this plus I have generated a Flex3 version of the project.
– Graph builder generates all the nodes with relationships then tries to join the primary nodes (those generated from valid urls in the initial query) with a relationship and then as a plain edge. The reason for this was that urls in the query were not appearing on the graph unless there was a proper relationship.
– I have had to modify the base graphing library to fix an irritating bug where the mouse-up wasn’t getting captured when dragging the graph.
– GIF icons will cause the graph builder to attempt to match an appropriate icon for the relationship.
– If you switch the favicon proxy off it will use default icons.
– I am no longer using the socgraph proxy script since Google have deployed a crossdomain.xml file for the endpoint. The switch is still there though.
I should really stop fannying around with this and do some work on the IFrame but I am starting to get bloody minded about this – I will get this working with no errors if it kills me.
– Definitive bug fix for loading icon (for now at least). There should be _no_ more exceptions to do with these anymore.
– Integrated a new edge renderer. This one now indicates the direction of the label relationship. Can’t decide if this looks better or worse than the old one.
– I _think_ the bug where dragging the graph gets stuck is fixed.
I started to bring in some of the other layouts that are available for flexvizgraphlib but there are issues with the rendering of the labels so I had to leave them out for now.
– The autofit seems flaky as to when it gets applied.
– From a large graph, if you make a query with no nodes the old node connectors sometimes ‘ghost’ i.e. they are still there.
– I am seeing occasional query failures. Strangely this never happened with the proxy.
A query with a large number of nodes will cause your browser to come to a standstill. Try focussing your search to ‘Edges In’ or on a particular uri.
The IconLoader is still occasionally throwing errors for parsing icons. As I see them I will try to track them down. If your site causing an exception mail/comment me and I will try and figure out what’s going on.
I am sometimes seeing an exception from the processing of the graph data. Haven’t tracked that one down yet. It happens after a query but before the graph has been drawn.
Nodes which should get the default favicon don’t have anything. I think this is because the favicon is being returned but the bitmap is corrupt or blank. Strange.
This application FaviconGrabber loads a remote icon from a specified url.
This was developed to test out the technique of pulling a favicon from a remote url for display in a Flex application. The application I had in mind was the Social Graph Browser where the favicon for a results urls would be embedded into the graph nodes (where available). I thought that pulling favicon files from a site would be simple to do but it turned out to be a bit of an adventure.
Firstly you can’t use HTTPService to get the icon directly. If you run the app locally it will fetch the data but since the service only deals with xml and text it will throw a ‘decode’ error. I.e the binary data in the icon contains bytes which are not accepted by the service. What’s more if you deploy the request to the web site it will simply throw a security error since this is an illegal type of operation unless the remote site has a crossdomain.xml file.
My final recourse was to write a Python cgi script to fetch the icon and then encode it into Base64. This is an encoding scheme which lets you embed binary data into xml. You can see the script bundled into the source.
Okay, so I had my Base64 encoded data. I can use HTTPService to execute the script and get the data back. Flex has a built in class Base64Decoder which you can use to decode the result of the proxy query. i.e.
var decoder:Base64Decoder = new Base64Decoder(); decoder.decode(encodedData); var rawData:ByteArray = decoder.flush();
Having got the data I realised that I had no means to display it. The ICO format is not a format that AS3 recognises.
A bit of a rummage around the internet and I discovered that flexlib has an IconLoader class which will display ICO format files. The problem is that it will only do it from an embedded resource or local url. It doesn’t take raw data.
This was already starting to become a pain in the backside for what was supposed to be a ‘nice to have’ feature of an application that was pretty esoteric to begin with. Still I soldiered on.
After modifying the IconLoader class to take raw data I got the Facebook icon working. Ah that sweet feeling of success how fleeting it is! The next icon I tried was from google. Didn’t work. Eh! Tracing through the code in debug mode it turned out that IconLoader didn’t support that particular icon format (8 bits per pixel [bpp] as it turns out). In fact all it supports is the easy peasy 32 bits per pixel which handily enough matches the AS3 bitmap format exactly. Since most of the favicons I subsequently looked at were either 8bpp or 4bpp I was well and truly stuffed.
There was nothing for it but to write the code for loading the other icon types into the control. This was a bit of work and I am now on familiar terms with Microsoft bitmap formats. More so than I wanted to be. I will be submitting this code back into flexlib so that others can use it. The extended component is bundled with the source for FaviconGrabber. It now supports 4bpp, 8bpp, 24bpp and of course 32bpp Windows icon formats.
Source code here.
All that and I still haven’t incorporated it on the graph! That’s my next task.
– Fixed bug in loading multiple icons from the same ICO image.
– Altered FaviconGrabber to accept any url. You don’t have to stick ‘favicon.ico’ onto the end. Just put in the url and it will attempt to get the associated favicon.
You can read up on the Social Graph and what it is here.
In short, Google have applied their web crawlers to collect semantic link information which are links between things. The links not only provide a connection but they also describe relationships between these things. Semantic links underpin the next generation of the internet and are a big deal in social networks as they are a means to track relationships.
The Browser uses the same Flex framework as the the SPARQL browser. The two core components which do all the work though are the parser which extracts data from the JSON data and the algorithm to build the graph.
This application uses flexvizgraphlib for the graph component.
You can find a full FlexBuilder2 project here. This is licensed under the MIT license. The reason for this is that it was just easier given that the 3rd party components used are under the same license.
The Google Social Graph API uses JSON to return data to any request made through the graph endpoint. This format is convenient for bog-standard web pages but a little more work under Flex. I had to use the JSON parser which is bundled with the corelib library. This serialised the returned JSON into AS3 Objects. This AS3 strutures were processed further to generate node information. I could not use the serialised AS3 structures in the manner given in most of the Noddy Flex JSON examples that are kicking around. This is because the structures were heavily nested and used a great deal of name paired behaviour between keys and objects (look at the code).
There are different approaches to building a visual graph to represent the JSON graph data. You can see a couple of them here and here. I liked Victors the best as has provided a good test bed to try out things which is the osocial web site. The osocial site is just the right size to provide data which can be queried with the Google API and represented on the graph when developing.
Basically the approach I have taken is:
– Build all the nodes at the start.
– Interate through all the relationships and draw the edges between nodes and label the edges with the appropriate associations. If a node already has an association when processing edges then move on. I make the assumption that a doubly connected node will have the same association although I don’t believe this will hold true for all cases. This means we may be losing associations.
You should be able to use this to the sort of stuff described here and display the graph for it.
Okay, a few notes on getting the best from this.
- If you deselect all the checkboxes for query conditions your results nodes will not be connected.
- Try moving the separation length slider up and down to expand or contact the number of visible nodes.
- To get an expanded graph deselect the ‘autofit’ checkbox and move the ‘Link Length’ slider up and down.
- The graph can get out of hand if there are a lot of edges. Try selecting the ‘Follow Me’ checkbox only.
- Pointing the browser at myspace is pointless as you will get an unmanageable number of results. A better approach is to pick a particular myspace page and query that instead.
- Blue circles represent result or ‘primary’ nodes (as I call them).
- Grey stars represent ‘Edges Out’
- Blue stars represent ‘Edges In’
- You will only see ‘claimed’ or ‘claims’ nodes when the ‘Follow Me’ checkbox is selected. If you select this you will see that the result node list (on the left) expands to produce follow nodes.
- If no node appears after making a query try flicking the ‘Separation’ slider up and down.
– Fixed error when blank search issued.
– Fixed About box problem where application unreachable.
– Added in ability to switch between graph types Normal/Force Directed