The Idea Dude


Tuesday, April 03, 2007

So what does it really all mean?

Red Herring reported that ZoomInfo announced the first-ever semantic search engine. Having dabbled in AI since the late 80's, involved with some reasoning systems in the 90's and worked recently with a semantic search company for almost 2 years, I'm perhaps a little jaded and a little skeptical. Especially when there are folks like Doug Lenat running semantic companies showing the same skepticism about the claim. Doug wrote some pretty influential books in the early 80's that became textbooks on knowledge representation and AI. So semantic search is hardly new and its promise and success like Mark Twain's reported demise is highly exaggerated.

Make no mistake, there is some really cool technology out there and they do solve problems. My issue is that too often the problems they solve are aimed either at very narrow niches or applied to a corpus of data that is scrubbed clean and primed to produce a seemingly remarkable result. Like fuzzy logic, I have no doubt semantic search will find their way into our lives in some way but I would hardly doubt that it will be life-changing and dramatic.

The theory seems simple, create ontologies that represent rich relationships which, when applied to real world day, will reveal hidden relationships and associations that are not immediately obvious. Apply it to general data, it will determine with better accuracy which pieces of data are more relevant than others. Unfortunately, when you throw real world data that is noisy, ambiguous and sparse, it all falls apart. Then there is the matter of creating and maintaining extremely rich ontologies to maintain the quality of the results. We saw how knowledge-based systems aka expert systems in the 90's suffering similar blowouts (by my definition, it means, the results didn't live up to the hype). It isn't the technology that is wrong, it is the knowledge engineering part, i.e. the activity in getting the knowledge represented be it in rules or semantics. While we have made great strides in understanding how the human brain works, it is hard to beat the flexibility, fault tolerance and massive connectivity in our heads that never ceases to learn and absorb from the day we are born.

So is this another emperor with no clothes or is this a disruption that threatens the likes of Google. If they stay in a narrow business niche, they may stand a chance, to be the next search paradigm would be long shot.


Post a Comment

<< Home