My Final Year Project

I haven’t talked about it here on the blog before but I thought I would take this opportunity provided by the bad Irish weather, and the even worse ability of Irish people to adapt to said weather to discuss my project. The opportunity provided, by the way, is that an exam scheduled for today has been postponed. Next one is on Wednesday, so why not take a few moments to blog, eh?

The project’s title is “Social Search & How it Affects Traditional Search Engines”. This is still a working title until I think of something shorter, but so far the academic nuts I’ve shown the paper to have liked it. Especially since I registered a domain and gave a name to the code part of the project – SCLSRCH (.com) – which is, obviously, the words “social search” but without vowels or spaces.

The sclsrch logo - much like the logo for this blog...

The aim of the dissertation is to research social media, it’s roots, what it does now, and how it fits into the overall view of the web today, including web 2.0 and this ridiculous idea of “web 3.0″. Not only am I looking at twitter and facebook, but how they work and how communities have changed the web to being more of a social meat market rather then just a point of reference for information streams. On top of that, it’s not worth looking at social media trends without looking at what I’ve termed “traditional” media. No, not newspapers and what not – but more traditional forms of communication online, and traditional methods of ranking and storing this information in a searchable manner (i.e. Google and page rank systems). Furthering this area of research I’ve looked into semantic web, and social semantic web – whatever they are! Although my project coordinator reckons semantic web is a dead duck, a lot of talk has come up in the last year regarding it, and a new and quite excellent book was published a month or two ago from a rival college…

A lot of reading, research and conclusions are drawn in this already. Especially around the next section – how all of this blends together in search. The biggest question, of course, being how to rank tweets or updates on facebook or other such services to find relevant information. Of course, on twitter one idea would be to rank based on the number of retweets – but even that isn’t a scientific measure of “important” tweets like the web, where it’s fairly safe to assume a page with 1000 links from other sites to it is more important then another site with no links to it from other sites. I pontificate a bit here about my own idea’s, and pull apart the Google solution – which is to only display results from certain twitter accounts. Which in my view is counter-productive and not in the same vein of thinking as “social media” would like. e.g. Posting results from @CNet or something isn’t as useful as pulling information from anyone tweeting about tech. Why? Because CNet aren’t interacting on twitter. It’s just using twitter as a social outlet to pimp their articles – much like an RSS feed… granted, an RSS feed you can talk back to.

I’ve been spending the best part of a few weeks tying up the dissertation. It’s taken a lot of work and my brain is fixated on it now, as it’s so different to what others are doing in the same course. It’s far more up-to-date then maybe DIT expects and because I didn’t take it from the “suggested projects” lists some lecturers put on their college pages I’ve no idea how it will be judged, and hence, marked.

Social Semantic Web by a few Galway heads - one from boards.ie, no less

The software side is decidedly weaker then the dissertation, in my opinion anyway. What I’ve done here is create a search engine from scratch, but rather then pull web results as per Google, I’m only searching social media outlets. Twitter is fine, Facebook poses a challenge I’ve decided to bow out of (for the moment) as you cannot search status updates publicly. Though I think the API has changed to allow users to have “public” status updates – something I’ll come back to no doubt. WordPress, blipFM (I’ve decided to ignore lastFM as their API doesn’t really allow for a “social media” result to come from users) and digg make up most of the search engine otherwise. Sounds fairly impressive but looking at other peoples software in the course scares the bejesus out of me, and at the same time humbles me. I fear mine won’t stack up in a college campus that probably would prefer I bash out a generic C++ application that doesn’t do remarkable things – but does it’s simply functionality remarkably smoothly but importantly for DIT, contains thousands of lines of code.

Most of my software is written in PHP by hand. I’m afraid to look at any online examples of API usage for fear of being called out for plagiarism. A ridiculous fear because no one is utterly original – especially on the web. Though I know I would benefit from seeing proper code examples being used in commercial products. Especially with my current annoyance – the flickr API.

I thought I’d write this blog post to put it out there that I’m working on something very cool, relevant and interesting. Once it’s all submitted and I hold a degree in my hands I’ll see about publishing the dissertation online. Some of it will be outdated (a lot of my conclusions are dated and timestamped to give references). The week after I submitted my proposal and everyone got excited, Google and Yahoo announced integration of social media to their services. Rather annoying as no longer was I on the cutting edge. Now it looks like I’m just copying other peoples work!

The DIT logo - evil, isn't it?

Despite my reservations about how it will be received on an academic level, I am confident that the work is sound and the project is great. I do think I have a legitimate case for saying I might be submitting it to the wrong college, but I don’t think I’ll suffer too much in terms of my grade.

The plan afterwards is to either secure a good job from this, or to simply move on with education. My preference (and I am applying to it soon) is Amsterdam VU (vrije universtaat – “free University” referencing free from state/religious interference, rather then being actually monetarily free) because it’s in Amsterdam – a city I feel in love with upon visiting last summer (and I visited Boston & New York, but still preferred Amsterdam – and no, I’m not a weed head) and the masters course is 1) cheaper & 2) more relevant to my undergraduate study then the equivalent DIT course. On top of that there’s a potential grant of €5,500 which would be spectacular. To explain VU to people, it’s basically the Amsterdam equivalent of DCU – it’s own secluded campus about 20minutes from the city center. Epic win.

I hope this post was an insight to what I’ve been up to, and in part, why I’ve been so vocal on twitter, boards.ie and other “social” outlets. I’ve been researching the future of all of this with regard to search, but also managed to read every little mistake made by companies – and all of these mistakes are being repeated. When I said Twitter needs to adapt quickly before a rival service kills it off by the end of 2010, I wasn’t making a wild statement. I really think this is going to happen in the same way I believe Facebook will go the same way as myspace and die off – granted, I think Facebook will put up more of a fight.

Here’s to 2010 – the year I plan on graduating with a dissertation my college probably won’t understand!
(Wish my luck in my exams this week!)