AI Assistants — Jarvis vs. HAL

Author(s): Dan Lovy

Originally published on Towards AI.

A Battle of AI Assistants U+007C Towards AI

A look at what makes a great AI assistant and a test project

Written by

Dan Lovy, Jonathan B, Briana Brownell, and Roger Sanford

Pundits and futurists have been painting a picture where people have virtual assistants that allow them to off-load the humdrum and allow them to focus on what they were born to do. You will be more creative, more productive and happier once the burden of minutia has been lifted from your shoulders.

What does that really mean and how do we build it? I am part of a team that is involved in a project attempting to do just that in a specific industry. More on that later. We have been thinking about what core issues are key to making a virtual assistant valuable.

Whenever I think about future technology, I often look to science fiction for thought-provoking models. Here are two of the more popular.

Tony Stark’s Jarvis — Wouldn’t it be great to have a virtual assistant (with a British accent) that took all the drudgery out of operating your technology. Need a new server set up? Just say so. It asks the right set of questions and faithfully executes. It’s helping you in the lab and is with you in your superhero suit.

2001’s HAL — Now HAL is beyond the Jarvis model. He has been given the keys to the spaceship. This works pretty well, right up until the homicides. In the movie, he is referred to as a member of the crew. HAL isn’t directed to do anything. He performs his duties and interacts with his fellow travelers as part of the team.

Whether we are building Jarvis or HAL, what are the core principles that we need to define?

We believe progress in virtual assistants will be driven along two dimensions: transactional and reactive

Transactional → Relationship

Today, I can speak into my phone and ask for things such as, “What is the GDP of Bolivia?” I will get back an answer that I absolutely trust. I have a voice-operated assistant that can get me any basic fact across most of the human knowledge. Most electrical power plants are so automated that they can be turned on and off remotely. These are all focused on understanding, optimizing and executing single transactions. This is the core of Jarvis.

Growth in this dimension is a shift from a focus on individual transactions to the relationships in between. It’s these subtle relationships that are the key. For example, IBM Watson touts its success with Subway (the sub-shop chain). Watson looked at weather patterns and used that information to make a menu and price changes at local shops.

We are the masters of transactional systems. We need to focus on the patterns and complexity of relationships. Some will be obvious and some will be subtle and hidden. This relationship dimension extends beyond the data to the relationship with the user. As we shift more away from pure transactions, the subtleties of the relationships with people become more important.

Reactive → Proactive

Jarvis is very reactive. Tony Stark gives a command and Jarvis springs into action. The challenge is how to move from a purely reactive mode to one where your assistant is proactively doing things. Like HAL running your spaceship (once we get past the homicidal tendencies), it is an immense leap. Bring me information or do things for me that I did not ask for. The assistant figures that out. Things happen in the background and information bubbles up as needed. If you have ever had a truly great assistant, that’s what makes them great.

Bellboys and Concierges

A hotel bellboy and a concierge are at the two ends of both these spectra. A bellboy is highly reactive and completely transactional. Take these suitcases to room 857. Bring me some soup. All commands appropriate for bellhops.

A concierge engages in a dialog, “What kind of show are you interested in seeing?” A new restaurant has just opened that I think you will like. A great concierge learns their customer and creates an experience. I saw you were coming in and I went ahead and booked a table at a restaurant I think you would really like. We value one barber over another, one mechanic over another, one dentist over another because we trust them. We only select them for their external features in the beginning. We stay with them because we trust them to do right by us.

Let’s look at specifics that we would need to make these transformations happen.

Predictions — Quoting Yogi Berra, “‘It’s tough to make predictions, especially about the future’”. Moving along the dimensions outlined will require increasingly sophisticated prediction engines. Google Maps drive time estimates are pretty good but not very sophisticated. Watson’s Subway weather system is a much more sophisticated engine. There are systems that make predictions all the time. The new A.I. technology can sift through data, see patterns and generate predictions that can astound. The leap here is not simply to make and report predictions but build predictions into the fabric. This means it is always on, always thinking.

Offer Prescriptive Suggestions. Recently, a friend was describing how challenging it was to set up a Google Adwords campaign if you had no background in online advertising. You are presented with many choices that have consequences that you have no feel for. A truly valuable assistant would make suggestions or, better still, simply carry out actions without direction (to a point, of course, that’s how Hal 9000 made crew members disappear). Without this, systems become bogged down as complexity increases.

Search Out Relevant Opportunities. Beyond efficiency and automation, there is growth. Tell me what I don’t know. Bring me the unexpected. This is the most challenging aspect of all. You can imagine using more traditional machine learning technology to help make predictions is a specific domain. Breaking free from a specified area and finding things for you in other areas will require a new set of techniques and tools yet to be invented. Let’s use A.I. and machine learning techniques to uncover possible opportunities I might never think of.

Let’s Build Something

As I mentioned before, we are building a virtual assistant for a specific industry, advertising. We are focusing on one aspect of advertising; the creation and placement of video advertisements.

Advertising has become technologically driven and increasingly complex. It is ripe for a virtual assistant approach.

Using these ideas as our rubric

• Transactional → Relationship
• Reactive → Proactive
• Predictive
• Prescriptive Suggestions
• Searching Out Relevant Opportunities

We are designing a virtual assistant for this audience. This is a work in progress, stealth mode as it were, so I can only some of the ways we are thinking about designing this assistant.

AI Focus Group

What people are feeling about a message is at the core of advertising. This feedback is usually captured through interviews or focus groups. It is now possible to use modern sentiment analysis techniques to get at this. It is possible to comb through mountains of social media content and extract patterns. We are looking at applying the latest in video response A.I.

Rather than creating a separate tool, we are giving our assistant access to this capability and an ability to gather this information in the background. As part of the media creation process, all the way through the media planning and feedback the robot assistant will have its robotic fingers on the pulse of viewers. The leap here is to build current knowledge and ultimately, predictions, into the fabric. This changes the tool from a command-driven transaction utility to something that can continually inform the entire process. We think this promotes us from the bellboy to concierge.

Media Planning

40 years ago, before most media strategists were born, you had 3 major networks and the Nielsen demographic road-map, media planning was very different. Today audiences now are spread across a vast array of habits, networks, and devices. They are watching whenever, whatever, and wherever they want to watch. We also have much more information about each viewer.

With our partners, we are working to take this data to and find the right audiences to spread across this landscape. Our assistant becomes a better and better matchmaker. Like the Netflix movie recommendation engine, it continually improves based on results and analysis of where to find audiences. In our model, our assistant is not asking, “Tell me what to do.” It is asking, “Tell me what you want.”

Automated Execution

There is a vast network of interlocking systems and interfaces to manage the placement of advertising. Here we want something like Jarvis to handle it all. There is an array of vendors that support the actual placement of advertising and reporting of results. Each one is in an arms race with the other that is creating increasingly complex tools.

‘Jarvis’ should be able to use all of the information behind the scenes. For instance, I really don’t want to interact with all the trading desks, real-time bidding and supply/demand partners out there. I have my plan and my assistant should be able to carry it out. We will do this through partnerships with the major platforms and we will automate those interactions.

Brilliant Feedback

Personally, I’ve grown weary of the word ‘smart’ in front of everything. We expect the feedback from the results to be brilliant. Instead of generating boring, fancy reports with graphs, we want the results to feed information back into the other phases so our virtual assistant gets better and better.

In addition to feeding back into other phases, the interface itself matters. We don’t always want our concierge to tell us how they came up with their recommendations (unless it’s off and we ask them why they selected it), we just want the recommendation to be clear and getting better all the time. I’m thinking of how amazingly powerful the simple interface of the Google search bar was when it first came out. It seemed especially amazing because it hid all the work it took to generate the answer behind the scenes and just presented the answer. How amazing would it be if media plans automatically used the results from previous campaigns to constantly improve its recommendations? What if it offered creative suggestions?

Great assistants always make a complex world easier to manage. Our hope is that we have found some core principles for creating a new breed of automated assistants. We are taking these first principles and applying them to a humanly complex specific industry, advertising.

Stay tuned.

Dan Lovy is Executive Director of SIMC

Jonathan Bissell is Executive Director of Community, Continuing and Corporate Education at San Mateo County Community College District

Briana Brownell is Founder and CEO at Pure Strategy Inc.

Roger Sanford is Project Guru: Insight at Crunch Mediaworks

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI