February 6, 2018

Building a Realtime Knowledge Engine

James Neufeld by James Neufeld

There was once a time when a nightly report was the best way to get up-to-date weather information. Now, we check a live-updating app on our phone (or make an API call) to plan, prepare, and react as the conditions change around us. We can’t imagine going back to a static weather report. Yet, the rest of news remains this way—static text, photo, and video content, requiring large amounts of cognitive resources to interpret, understand, and contextualize. Despite being a critical source of information to many industries that shape the world’s economies, news has not become an easily decipherable data stream used to inform critical business decisions.

But what if we could turn all news events into a weather app of sorts? What if you could detect and distill global events down to the data points of who, what, when, and where, without needing human resources to determine what information is important and what is not? What sorts of businesses and applications could benefit from knowing exactly what is happening, when it happens, at any given moment? We believe there is a long list of industriesnews, logistics, travel, first responders, finance, and corporate securitywho depend on the delivery of fast, accurate data about world events.

Here at SAM we’ve always known social media data to be an incredible source of real-time information. One that extends far beyond simplistic marketing signals. Social data is a global sensory network of micro-updates that can tip us off to breaking news stories, expose what some try to keep hidden, usher shifts in economic markets, and detect earthquakes faster than seismic waves. Despite the incredible potential, social media is wildly unstructured and requires a tremendous amount of analytical effort before any value or meaning can be extracted from it

That’s why today we are very excited to unveil the beta version of the new SAM platform. We’re aiming to become the world’s first real-time knowledge engine, answering the question of what’s happening in the world right now, with unrivalled speed and accuracy.

The SAM v2 beta aims to inform users by scanning social media with proprietary AI algorithms that can autonomously understand what is being said, where it is being said, and the authenticity of who is saying it. More succinctly, our initial beta is focused on providing three things: fast alerts, accurate information and incident awareness.

Fast Alerts: We’ve built and trained our AI models to understand the difference between “shots on net” and “shots fired”, and the difference between “McDavid is on fire” versus “my school is on fire”. This engine automatically scans and surfaces anomalies and world events by detecting patterns of social media chatter and media reports that would signify a noteworthy occurrence. (Some examples of our AI’s speed can be found here)

Accuracy: By studying the world’s largest dataset of verified social media content (our own CMS) we have a unique understanding of what verified chatter looks like. Our system asses the authenticity of a user by processing each social profile through a verification matrix which scores the user. Based on the validity of each source, SAM’s AI can trigger an alert or look for additional sources to corroborate or debunk a less credible source. Real-time knowledge is only meaningful if the information provided is credible.

Awareness: The SAM v2 beta aims to provide complete awareness of global events by autonomously collecting related media and headlines. As the amount of social media content continues to rapidly increase, it becomes more and more difficult to collect and organise. We aim to ease the processing required to collect and disseminate the related content to world events.

Our initial beta wraps all these components together in a new SAM dashboard that allows you  to follow world events and customise elements of the data feed around key locations and event types.Today we’re opening up access to the beta, and we hope many of you will join us in building the world’s first real-time knowledge engine. You can request early access on our new landing page www.samdesk.io

This new product is exciting not only because we love innovating and building new things, but because of what we believe it represents. While we’ve long been at the forefront of social media newsgathering (the SAM team has collectively trained thousands of journalists/analysts around the globe), our products have always required a knowledgeable operator to get the most value (as a workflow tool we made UGC teams 10x more efficient in processing and verifying social content). However, when we looked at the exponential growth of social media information becoming available, combined with the immense complexities required to find and verify this information, the task ahead became very clear: we needed to build an intelligent machine to make sense of what is happening in the world. And so our real-time knowledge engine concept was set in motion, we beefed up our AI skills, bolstered our team, and charted a course to make news and information as easy to understand as a weather app.

I could not be more excited to show the world what the amazing team here at SAM have been working so hard on!

To learn more about our beta please visit our new website: www.samdesk.io