See more TechXchange Talks videos.
Handling machine-learning chores at the edge is becoming more common as tools and hardware have improved. Editor Bill Wong talks with Dr. Sailesh Chittipeddi, Executive VP & General Manager for IoT & Infrastructure at Renesas, about artificial intelligence in edge-computing nodes. Check out the video or its transcript:
Wong: Well, artificial intelligence and machine learning to edge-computing nodes is becoming more common these days, especially as microcontrollers and SOCs gained support for machine-learning inference models. Today I'm speaking with Dr. Sailesh Chittipeddi, who is Executive Vice President and General Manager of IoT & Infrastructure business at Renesas.
Hopefully, we'll be getting a better understanding of this trend. So to start with, the advantages of centralized cloud resources have made AI in the cloud very popular. Where do you see AI at the edge or even the endpoint? How does it fit into the mix?
Chittipeddi: Good question, Bill, and thank you for having me. Let me begin by kind of talking about the dominance of the edge of AI in the cloud. Primarily that's been the sweet spot for a long period of time.
Obviously, there's always going to be workloads where it makes a lot of sense to have it continue to be in the cloud; for example, things like weather forecasting and so on. It would never make sense to move that to the edge of the network, however, as computing becomes much more ubiquitous and powerful at the endpoint.
As we start getting much more capability to do even AI and tiny machine-learning capabilities at the end point of the network, we actually start to see significant enhancement of capabilities to do some of these inference models at the edge of the network, and that's driving a trend for moving certain workloads, not all, obviously, from the cloud to the endpoint of the network, and the trend that's driving this is the need for very low latency. Obviously, that's a very important factor in the discussion.
The second point also is the security aspect of it, which is equally important as you move towards the endpoint.
And then the third aspect, which is the need to be able to have instantaneous filters for stuff where you don't want to wait, for example, for the transit time, for information to go to the cloud and get it back right.
So even when you look at things like video surveillance, for example, where the initial trend was having all the processing done at the core of the network, if you will, the trend much more now is to have simpler facial-recognition models embedded in the video camera application itself.
So that's kind of broadly driving some of the trends that we're seeing. And power consumption obviously is another important factor. There is less power consumption at the end of the network, and it actually works a lot better relative to what needs to happen.
And the other thing also that I don't want to underestimate is our networks that are significantly improving quality moving from 4G to 5G networks, right? And that certainly provides us better capability in terms of driving some of these trends that we're seeing in the marketplace.
Wong: Well, what's driving the acceleration to moving AI to the endpoint? Is there additional hardware capabilities that are coming about? Are there improved pieces of software out there? What are the pieces?
Chittipeddi: Yeah. So there's both. Right. So there's two elements to it.
As I briefly mentioned earlier, one is the hardware element, which is a massive amount of compute power. A massive amount of computing at ultra-low power certainly helps. That's from a CPU perspective. But also you now have embedded AI solutions, whether it'd be spiking neural networks or whether it be CNN, either one of those, right. You have capabilities now of embedding AI together with our CPU to enhance the capabilities of AI at the end point. Something that you normally did not have before because the traditional model was consuming significant amount of power in order to handle the processing.
But now, actually, it's transitioning much more for the ability to serve the needs of the end point significantly better than was ever done before. Software certainly is an important factor. You know, the capabilities for doing these, the AI scripts, if you will, what I call inference scripts at the end point, and that certainly makes a big deal.
Then you have simplified libraries and compilers that are available for doing more AI at the endpoint of the network. So all those factors are driving the move to where we're seeing,
Wong: OK. So do you anticipate AI at the edge becoming the norm as opposed to the exception as it is right now?
Chittipeddi: I think there will always be a mixed bag, right? It's strong. Depends on the workloads that you're trying to handle, the workloads that are compute-intensive or CPU-intensive always continue to be done in the core of the network.
On the other hand, as the workloads get simplified, you'll find them increasingly moving towards the end point or towards the edge of the network. And I think you'll see that trend accelerated. That is a matter of fact as the number of devices that are connected at the end point increase.
You'll find this capability going up significantly by some estimates and I was looking at a piece of paper over here. There'll be 55 billion connected devices generating 73 zettabytes worth of data by 2025 and it's certainly not all in the cloud. So that kind of gives you an idea of the growth that you have in there.
Wong: Could you give some examples of AI endpoints that are starting to emerge?
Chittipeddi: So, voice being the biggest, voice features are certainly the most elementary example of that, but we have other technologies, even with the dinosaurs that we're working on that handle everything from video processing to preventive maintenance technologies using what we call our dynamically reconfigurable technologies (DRP) that we have, our DRP technologies, which is embedded AI using a feedforward neural network together with the core MPA in our devices.
This device, for example, one of the best examples that I can give, is it allows you to do simple facial-recognition technology at the end point. So it's got so sophisticated right now, Bill, that you have the ability now to be able to track certain faces in a crowd and pick them up in a crowd pretty close to the end part of the network.
And that's, you know, it has both its positives and its negatives. Of course, we prefer to focus on the positive aspects of it. But certainly that'll be a trend that will continue and then being able to look for defects within a line. That's something that you don't want to be waiting around for all the data to go up to the cloud. That's in line where you find out what a defect is. So those are kind of some of the simplest examples.
And then voice, having a subset of certain features being available at the end point is, of course, a trend that's continuing, and we're partnered with a number of other companies in this area to enhance that capability.
Wong: OK, what's the impact of communications bandwidth with respect to AI?
Chittipeddi: I mean, that's a good question. Certainly the endpoint to the edge, especially with 5G, becomes far more important, right? You have much more bandwidth now between the edge of the endpoint than you were able to do before and, certainly, that's good.
That's going to be a major contributing factor to anything that we're seeing in this particular area. But, also increasingly, even in terms of easier access to the core, the bandwidth is improving, whether it's optical networks or whether it's wireless networks.
There are trends that are driving favorability in that regard but, nonetheless, there always will be a need for the low latency factor between the edge and the end point that'll be contributing factors to helping this trend to its end point. I call it Edge II, if you will, because often times it still doesn't make sense to drive everything to the core of the network and then wait for the response, even though the latency times are improving significantly.
Wong: There's also low-speed networks like LoRaWAN, for example, where you really can't shove a lot of data back and forth.
Chittipeddi: Exactly like narrowband, IoT level like narrowband IoT, and so on. Then, of course, you have increasingly the ability to use Zigbee-based technologies for the simplest of sensors to be able to access information.
So, yeah, obviously that's a very important point as well for the data sets that don't that periodically send a burst of data. It is a contributing factor to what we're seeing as well.
Wong: Excellent. Well, thanks for the overview. It was very informative and I appreciate you speaking with us.
Chittipeddi: You're welcome.