Interview with JP Abello of Nielsen: A look into the Internet of Things & AI


Interview with JP Abello of Nielsen – A look into the Internet of Things & AI: An opportunity to fix consumer advertising

Jean-Pierre Abello leads global engineering, research and development for the measurement of the Internet of things and digital media as part of the thought leadership team at Nielsen. Nielsen is the leading measurement and data company for consumer goods and consumer behavior in media. He was recently interviewed by AI Trends Editor John P. Desmond

Q. How do you see consumer internet of things, including dishwashers and refrigerators evolving? I think you have some new ideas on that.

A. Jean-Pierre: Well, the consumer side of IoT – primarily the smart home today – is the part of the IoT space that interfaces directly with consumers. This contrasts with the Industrial side of IoT, which is more about direct machine-to-machine interactions, and evolved the old M2M space. Consumer IoT has the potential to grow into an essential part of the IoT ecosystem as consumer devices become increasingly connected.

In a way, this can be compared to the migration from the dumb phone to the smartphone. As consumer devices become more connected, they can offer a lot more services, but they also cost a lot more. And device manufacturers are expecting consumers to replace them frequently as the technology advances rapidly. The problem with this mindset is that it’s really more of a consumer electronics industry mindset. It’s still companies making hardware products and selling them to consumers at a premium, and expecting them to upgrade every few years. That’s in sharp contrast to the refreshment cycles of home appliances such as refrigerators and dishwashers, which tend to last 10 to 20 years easily.

It would be hard to migrate to a model where consumers are asked to replace their fridge every two or three years, like their smartphones. And it’s also very different from the model that we have been used to on the Internet, especially on the web and mobile, where it’s less about selling hardware and more about providing consumer services that can be monetized by data. So, this is an industry that needs to learn about the value of data and how to monetize it in ways that also benefit consumers.

Q. So, these are the users of the dishwashers and refrigerators. So, how would publishers get involved with this branch of consumer IOT?

A. Jean-Pierre: The way publishers do business on the web and on mobile and on TV and radio today, is by making their services available to consumers either on a subscription basis or for free when supported by advertising. It has worked well, and has enabled that consumer industry to become mainstream and accessible to nearly everyone in the world. In fact, a founding principle for the web as defined by its inventor and W3C founder Tim Berners-Lee was to make it into a medium freely accessible to all. And ad-supported content greatly helped enable this.

Unfortunately this is also in sharp contrast to what we see in the smart home where hardware companies try to charge consumers full price for their latest devices, and are not familiar with the service business. These manufacturers need to try to think more like web companies, and more similarly to the Internet publishing industry. Once you have a service, then you can think of different ways to monetize it with consumers and advertisers. So, for publishers, getting into the consumer IoT is really about getting access to more sources of consumer data.

If smart appliances like refrigerators, coffee makers and washing machines become services that generate consumer data, that data can be leveraged to better understand consumers. The way advertising works on the web is by looking at the web sites that you browse, the emails that you read, and analyzing all that data that you as a consumer generate to put you into a consumer segment. And advertisers want to reach audiences by consumer segments. Today, this is only based on what you as a user do when you browse the web or watch TV or listen to radio, and it’s  not a complete representation of the way you live your life, your everyday life.

In fact, most of your everyday life today is still offline. You don’t cook, eat or do your laundry online. When you browse the web or watch TV, publishers don’t know what type of coffee you drink or the way you do laundry or what type of food is in your fridge. So, by getting access to devices that are becoming connected to the Internet, it’s bringing effectively a large part of your offline life online and is generating data that could be used to better understand the type of person you are as a consumer. And that knowledge can be used to put you into the right consumer segment, which is something that publishers need for their advertising business.

Q. Okay, so are any manufacturers moving to what you could call the hardware-as-a-service model that you describe? How far off is it?

A. Jean-Pierre: Well, on the industrial side it’s already happening. GE doesn’t directly sell aircraft engines anymore; they sell aircraft engines as a service. They take care of all of the hardware maintenance and all of the upgrade cycles, and just sell a subscription to airlines for their engines. Home appliance manufacturers are now increasingly considering bringing a similar hardware service model to the consumer space, as the replacement costs tend to be too high for the average consumer.

So for example to provide refrigeration as a service, you would subscribe to a cooling service for a fixed monthly fee, probably under $50/month, instead of buying a new refrigerator for $2,000 or $3,000 every 10 to 20 years. The manufacturer would take care of all of the hardware maintenance, all of the upgrades, and make sure you always have the latest, greatest fridge that generates the most actionable data. And once we get to that point, it’s going to start looking a little bit more like what we have on the web today.

If you look at the web in the early days, web-based email, magazines and news were offered as subscriptions and that’s how they initially got into this business. But as these online services started to generate data, advertisers found opportunities to sponsor the content. In the case of the smart home, smart fridges will likely know what’s on every shelf and what type of food you consume, how often you consume it, what are your food preferences. That kind of data can be used to augment the understanding that advertisers have about what type of consumer you are and put you in the right consumer segment.

And that could lead to much better audience targeting on existing mediums where advertising is already common, such as TV or radio or the web and mobile, and make ads much better and more relevant on those screens. And potentially also, a reduction in the number of ads shown since the value of these ads would be higher with a lot less waste. So, once you go to a hardware as a service model, there is potential for advertisers who see value in the data  these devices generate to subsidize the cost of these services.

Just like on the web, where a majority of websites are completely free to consumers today, smart appliances could also become mostly free for people who sign up to a service plan supported by ads. If we look at the case of webmail, about 80% of Yahoo Mail users don’t pay for their email account in exchange for seeing ads . That’s the majority of the market. And somewhere around 20% of users pay $20 to $50 a year to get an ad-free experience.

This is also similar to the home security space. The companies that offer home security services have hovered at about 20% penetration in the US for some time and are struggling to increase market share. That’s because they charge a rather steep monthly fee for their home security services. The fact is that the vast majority of consumers do want home security but don’t want to pay that much for it. So, just like for webmail, by not offering a free service, these companies are effectively not able to access about 80% of the market.

Once the data produced by these smart home devices gets fed into the ad platforms that advertisers and publishers use, and starts achieving better audience segmentation, there is real potential for subsidizing these IoT services, just like we see on the web today. And this would result effectively in smart homes where a significant portion of the hardware devices are completely free to consumers just like the majority of websites are free to consumers.

Q. Is this a year off, two years off, any estimate like that?

A. Jean-Pierre: Well, it’s something that’s still being discussed. As you know, the consumer IoT industry is still very much in its infancy. It has not been able to cross the chasm into the mainstream, and it’s still very much in the realm of early adopters. And there are many reasons for that. Probably one of the biggest ones is the extreme fragmentation of the ecosystem. There are too many technology standards being developed in parallel. There are also too many proprietary platforms being built outside of these standards. And that creates an ecosystem with many interoperability barriers, where devices don’t work well across brands, except in the case of a few limited partnerships and is not economical to scale.

And for consumers, that is a problem because nobody wants to buy into a brand of devices that might stop being supported a year or two down the line. We need a common platform for the consumer IoT space and IoT in general. Unfortunately we are not anywhere close to that today. When the web was launched, it was built on a single technology standard, HTML maintained by the W3C. Anyone could create a website, and it would work the same on any web browser on any computer anywhere in the world.

With mobile, it got a little bit more difficult. You had iOS and Android, but it was just two platforms and it was manageable. You could effectively still get the same apps across both ecosystems and consumers were not afraid of buying into one or the other. Now, when you’re looking at the consumer IoT space, it’s a complete mess by comparison. Until this situation improves, consumers will be reluctant to acquire a lot of smart home devices. In fact, the migration to hardware as a service will probably help with that a lot. Because you won’t have to worry anymore about your fridge stopping to work two years from now because it became unsupported. Maintenance and upgrades will come as part of the service model, and you will be more likely to adopt that.

So, it’s really a question of how quickly the consumer IoT industry can move to a service model. I can’t tell you what the timelines are. But it looks like manufacturers are tired of waiting for the consumer market to take off, and are starting to think seriously about other monetization models.

The IoT space has been stuck at the peak of inflated expectations in the Gartner Hype Cycle for four years now, and that’s a very long time. I think we’ll see an evolution probably in the next two or three years.

Q. Okay. How about security? What would you describe as the state of IOT security today and how can it be improved?

A. Jean-Pierre: IoT security is also another major problem because this industry historically has been a consumer electronics industry. It cares primarily about making devices that are as easy as possible for consumers to set up and add to their homes. And this means that security has taken a backseat in their priorities list. When you buy a web camera or a home router, or actually any consumer device you connect to the internet, they want you to be up and running as quickly as possible. They typically don’t ask you to change the default username and password. This is a big reason why many smart home devices are so vulnerable today.

And that’s something that was leveraged for the first time on a very large scale last October 21st when the Mirai botnet took advantage of that to launch a major distributed denial of service attack (DDoS) on the Internet last year. It took down many of the largest websites for a good part of a day, disrupting service across most of the US. It’s also a problem that’s growing as the number of unsecured devices deployed is still increasing, but that really needs to be fixed if we want this industry to be healthy. A growing number of consumers are returning their smart home devices after they realize how insecure they can be.

Unfortunately, most consumers don’t understand how to make them secure, how to even change their passwords, because it’s not part of the setup process. So, the way I think this could be improved is first, when a new device is purchased by a consumer, at set up time the consumer has to be required to change the username and password. This is something that we are already used to do everywhere on the web and mobile, when you set up any new account. The first thing they ask you to do is usually to choose a username a password of your own.

Unfortunately we don’t have that step today with IoT devices. So, if we just get to that point, we’ll already be a very large part of the way towards a solution to this problem.

Q. How do media measurement companies stay involved in this game as the IoT market evolves?

A. Jean-Pierre: First of all, to stay in the game you have to be able to measure how these devices are being used, which means that measurement has to be part of the infrastructure being built for this ecosystem. There is some measurement already in place today. It’s called telemetry, and it was designed by device manufacturers for doing remote maintenance and problem detection, to try to reduce the failure rate and return rate of their devices. This enables them to fix some problems remotely when possible. However this is not the right type of data that needs to be collected for consumer measurement purposes.

Measurement companies need to know how these consumer IoT devices are actually being used. And today, unfortunately, we’re still at a point where the software stacks in both open source projects and proprietary platforms have not yet prioritized consumer measurement. This is a key missing component in the IoT infrastructure. And that’s because most IoT companies are still thinking like hardware manufacturers instead of trying to leverage the value of the data that can be collected. Once you can measure how the devices are being used, that opens the door to a service model and the monetization of consumer data.

This measurement will also have to be done in ways that prevent bias or manipulation by the device manufacturer or service provider. The best way to protect against that is to use third-party independent measurement companies that are audited by official industry monitoring groups like the MRC (Media Ratings Council). Independent measurement companies should be responsible for adding the consumer data collected from these IoT devices to existing consumer data sets for TV, radio, the web and mobile, and improve their data models for understanding consumer behaviors.

For that to happen, it will be increasingly important for these companies to get involved in the leading IoT standards being developed, and add consumer measurement to every IoT platform specification. One of the most crucial aspects of this is the development of a global data standard. It’s actually almost okay to have a large number of different IoT application platforms, as long as their APIs generate data that is compatible across all of them. As long as measurement is being implemented across all these standards using the same data taxonomy, the same way of representing data and what it means, then it will be possible to collect data from many different types of devices, many different brands, many different types of platforms and fused together and to generate the value that we’ve been talking about.

Q. What would happen in the IoT market, such as smart homes, if the standards approach does not prevail?

A. Jean-Pierre:: Well, what will happen is we will continue to have what we have today. We’ll have an industry that will continue to appeal mostly to early adopters. If we don’t make progress towards a single platform, it’s likely that consumers will become increasingly weary of the security and privacy breaches, especially if they continue to increase and grow in scale. It will effectively make this into a very small industry; it will remain a niche industry. It will not be what some had hoped would become the “third wave” of the Internet. It will be a collection of smaller, closed ecosystems that will remain very expensive and don’t really appeal to the majority of consumers.

Q. How can artificial intelligence save advertising?

A. Jean-Pierre: Assuming a common IoT standard ends up prevailing, and that it can generate quality consumer data that can be fused together, and that the industry starts to move towards a hardware-as-a-service model, which can be monetized through advertising support, we will unfortunately still be left with one major problem. And that is figuring out how to deal with the massive amounts of data generated, larger than anything we’ve ever seen before. It will be increasingly difficult to make sense of it at the orders of magnitude involved. I think the industry will have to move to probabilistic approaches, where not all the data is being used all the time but it’s being used in a more selective manner, very much in the same way the human brain works.

With advanced AI algorithms like deep learning, it’s possible to detect data patterns more efficiently, and ignore a large amount of noise and focus on what’s really important. This will enable the ability to extract meaning from massive amounts of data in much more efficient ways. Another aspect that will also be important is that with AI, you can create models of user behaviors that get better over time, using self-reinforcing machine learning trained with personal data. Today the advertising industry uses modeling techniques that are fairly crude, integrating data from multiple sources but without enough access to personal data to develop a deep knowledge of each consumer.

But I think there’s potential also to create consumer models that are located much closer to the user. Instead of having advertisers trying to understand who the user is, we could have a user agent effectively sit inside the web browser or inside the private home environment that would analyze data being produced by all your smart home devices. It would be able to create a very intimate model of who you are as a person, almost like a close family member or a celebrity personal assistant. A smart assistant that would know you down to a level that would make you feel very uncomfortable sharing that information with the outside world, especially publishers and advertisers.

And the role of that smart assistant could be to act as a sort of a virtual interface between you and the advertisers trying to deliver a message to you. Advertisers would effectively have to market their content to that smart assistant, which I would like to call the “smart ad assistant”. And that smart ad assistant would decide which ads could be shown to you at the best place and the best time, and understand when would be inappropriate to interrupt you, when that could be annoying to you, when that could be useful to you. That would effectively turn advertising from something that is perceived very much as a disruption, an intrusion into your personal life, into something much more helpful, something much more akin of a suggestion or recommendation from a trusted friend.

Imagine, for example, that you are in the market for a new kitchen appliance, for a new cleaning service, or that you are looking to change brand of detergent because you are not happy with the way your clothes are turning out. That smart ad assistant would know what you are going through, and it could choose to let through more advertising relevant to what you are looking for, without disclosing personal information about you. So, that’s one way AI could help save advertising.

The rise of ad blocking, both on the web and mobile, is a direct threat to the main revenue source of the publishing industry. To survive, advertising will have to find a way to become much more consumer-friendly, more helpful and less intrusive and become part of your life in a form that you’re more likely to accept. And it has to be done also in a way that protects your security and privacy. All this could be achieved by leveraging the smart ad assistant’s ability to create a superior data model of you as a person, and keeping that knowledge private so it’s not scattered across advertising databases on the Internet.

Q. That’s a very interesting vision. Is there anything you’d like to add or emphasize?

A. Jean-Pierre: As a closing thought, I would say that the consumer IoT space probably has the most potential of any of the IoT verticals, even though it is the smallest of these verticals today. It has the potential to be really transformative for our society, because it will impact our everyday lives in a very direct manner, by interacting directly with people instead of just machines. This will profoundly transform the way we live, removing busy work from our daily routines while also reducing out of pocket costs for consumers. And it will also open a green field of opportunities for much more relevant and helpful advertising using much more personal types of AI that we’ve seen before.

In my opinion the smart ad assistant could become almost a direct extension of the real person. Some people consider their smartphones already as an extension of themselves, but this is only the first step. The next step is for AI to take this extension to the next level. And it will be your personal AI which will interface with key aspects of the outside world for you. And that personal AI will be there to protect your privacy, protect your security, and make sure that only the most useful information, the most useful services, get through to you at the right time and right place. The end result should add value to your life, cutting unnecessary clutter saving you both time and money.

I think that’s what the real potential for consumer IoT is in the end.