How Remote Expert Technology is Helping Preserve and Share Knowledge

In a new episode of SAP’s Digital Supply Chain podcast,Tom Raftery, sat down with Librestream’s VP of Product Management, Jon Newman, to explore how remote expert technology is helping workforces drive efficiencies, preserve and share knowledge, improve worker safety and more.

Listen to the podcast or read the transcript below.

Good morning, good afternoon, or good evening, wherever you are in the world. This is the Digital Supply Chain Podcast, the number one podcast focusing on the digitization of supply chain. And I’m your host, global vice president of SAP, Tom Raftery.

Hey, everyone. Welcome to the digital supply chain podcast. My name is Tom Raftery with SAP and with me on the show today I have my special guest, Jon. Jon, would you like to introduce yourself?

 

Jon Newman: 0:57

Hi, Tom. Thanks for having me. Hi, folks. My name is Jon Newman. I’m the VP of Product Management at Librestream Technologies responsible for product strategy, roadmap and execution. Librestream is a company that is the leader in remote expert solutions globally. And so we sell to the largest Defense, Oil and Gas, Test Inspection companies around the world in order to allow for their remote inspection and remote expert capabilities.

 

Tom Raftery: 1:28

Okay, so for people who might be unaware, Jon, what’s remote expert?

 

Jon Newman: 1:33

So remote expert is the ability for companies that have workforces in locations that may be more challenging and more difficult to, to communicate it. So, sort of non-office environments. So, it could be things like military bases, oil rigs, ports, remote farms, and things like that. And it’s allowing people out in these remote locations to be able to talk to people in other facilities, may be in head office, or other parts of the supply chain, to be able to communicate with them to get guidance, and expert advice in terms of what they may be able to do or what they want to do, or to perform things like inspections and have supervision, or to be able to for maintenance tasks, and have more senior folks kind of watching and talking to them a lot. So, it’s allowing for these more industrial use cases, and allowing them to happen in challenging environments where maybe bandwidth is not as strong or maybe there’s other security or harsher networking needs.

 

Tom Raftery: 2:44

Okay, so my health insurance company this year has rolled out a new facility on their app on the phone where I can get a tele-consultation with a doctor, just press a button and bang, up comes the doctor on screen and I can have a consultation with them. Is that, at its most basic level, the kind of thing you’re talking about?

 

Jon Newman: 3:06

Yes, exactly. So, the medical consultation is definitely one usage of this, you know, remote expert capability. And that’s something that we do. But our sort of particular focus and our particular leadership expertise is in these more challenging environments. So, defense companies talking to, you know, militaries around the world in remote locations, or oil and gas folks communicating with their employers on rigs, and inspection folks like that. So, it’s more the rugged environments that we specialize in and we’re the leaders in. But yes, the medical concept is one example of a remote expert use case.

 

Tom Raftery: 3:42

I can see how technology is enabling this because we get devices which have better connectivity, better processors, better cameras, etc. So, yeah, the technology is improved to allow this to happen. But what about the demand? What’s the demand for it like?

 

Jon Newman: 4:00

So, the demand for the remote expert capability we’re seeing just increasing over time, I mean, Librestream has been doing this and has been the leader for the last 15 plus years and invented the remote expert category. And so we’ve seen the drive for this increase, more and more, especially as you know, customers in all the verticals that we that we sell to go along their digital transformation journey and look to help to digitize and make more efficient how their workforces interact and engage. And so we’ve just seen an acceleration of this and especially actually in the last year or so, with the global pandemic and people not being able to get on a plane to do a you know, an inspection of you know, of a salmon farm in Norway, where their supply chain is inspecting cargo in a port in Southeast Asia, or find getting a plan to fix military or aircraft installations in the Middle East. So, the restriction on travel has only accelerated this digital transformation, and also the focus on sort of the carbon footprints and how travel impacts that. That’s also something that folks are way more sensitive too, as well. So all these things, in addition to safety, and other aspects that have always been true, are really helping drive this, this digital transformation within these workforces. And driving the remote expert use case as well.

 

Tom Raftery: 5:27

Okay. And I can imagine as well, for example, things like the scaling of knowledge being something as well, that would drive it because now experts sitting at their desk at HQ, for example, can suddenly be interacting with people in multiple locations at the same time, rather than having to, to your point to get on a plane, go there, spend a few days there, come back, get on another plane, go to the next place, you know, it seems a far more efficient use of knowledge and resources.

 

Jon Newman: 6:01

Absolutely. So I mean, the example you talk about where it’s more efficient for the human expert that may be interacting with people remotely in these foreign locales, that’s definitely one aspect of this. And then also, as we look forward to where this is going and how does this converge with advanced technologies, such as artificial intelligence, and augmented reality. And then also with the global need of the older workforce aging out, and you know, not having access to these human experts, given the sort of global worker shortage that’s projected to be 85 million by 2030. So that broader trend is driving a need to, okay, we’re connecting, you know, human experts with human workers today. But what we try to do now, because of this expert shortage that’s happening is trying to start to capture this expertise that’s being shared within the Onsight platform and then analyze it and using advanced technologies, like artificial intelligence, be able to then surface that in a timely fashion automatically to these workers. So, they require less and less interaction with a human expert. And we can more and more rely on providing them with this expertise that we’ve captured through the Onsight platform and then being able to surface it to the worker in ways that makes sense using augmented reality, to allow them to put this expertise that that’s being captured, to be able to put it in front of their field of view, and be able to allow them to have access to what they need and all the data and insights they need as they’re going about the task. Be it like a standard maintenance task or repair, or some sort of inspection and be able to do this in a safe way to allow you know, the use of AR wearable so that they can be hands free. Or they can be interacting via voice so that they can still use the tools they need still have access to that timely information, not necessarily using a human expert.

 

Tom Raftery: 8:00

Okay. So just to clarify there, because you gave me a very big answer to a very small question, I just want to try and break it out a little bit. So the shortage of expertise I gather is happening, because a lot of experts are hitting retirement age, is that correct? And they’re retiring out of the system? Or is there some other reason for the lack of expertise? Or is there a greater requirement for expertise that just isn’t bubbling up enough. How does that break out?

 

Jon Newman: 8:28

Yeah, absolutely. So I mean, I think I mentioned that the global worker shortage is projected to be about 85 million by 2030. And the statistics we’re seeing as the baby boomers are retiring from all sectors at a rate of about 10,000 a day, in the US alone, for example. And so what you’re seeing is that generation that typically tended to stay at a job for 2030 years, which developed a vast amount of expertise in something that dramatically aging out of the workforce and the people that are coming in that need to do the same jobs and fill their shoes, they don’t have that same tenure and years of experience under their belts. So that is huge skills shortfall, that’s not going to be met by you know, experienced workers coming in, and that’s where we see a huge opportunity for technology to pick up some of this knowledge gap.

 

Tom Raftery: 9:19

Okay, and you refer to them, the two technologies, to AI and AR and if I understood it correctly, what you’re doing with the AI is you’re taking a lot of the knowledge that has been gleaned over the years from the experts, and you have that in some kind of back-end system. And then when people come in with queries, the queries go through the AI. The AI can test or can take from that kind of knowledge base at the back-end and then suggest answers. Is that more or less what it is?

 

Jon Newman: 9:49

Yeah, absolutely. So at a high level what we’re doing is because Onsight today is the conduit for these remote expert sessions and bringing in as well, additional information from IoT sources, for example, or other sensory information, or even pulling in your training material and manuals from sources that all these companies have, because we have that central point today, and we’re bringing in, then we have the ability to sort of analyze and collate this using artificial intelligence to extract the kind of the meaningful aspects from all these different disparate sources.

And then, you know, using augmented reality devices or other devices to your phone or tablet and the like, we are able to understand the context of where the worker is, and understand the context of what they’re in front of and what they’re doing. And we can then again, use artificial intelligence to figure out what we should be surfacing for that worker at that given time. So being able to recognize what equipment they are standing in front of or being able to pull in all historical information about that piece of equipment, be able to see what task they’re doing, and potentially offer up steps to work through or just in time training information that may require. And using the augmented reality side of things, to do this in a way that’s intuitive and safe for the worker, because they’ll be typically in difficult or dangerous situations where they’ll have like, in that field of view, there’ll be a lot of machinery or have one or two hands that they they’ll need to use for tooling or holding onto something and the like. So augmented reality is the way we serve this information in a way that’s intuitive for the worker, and AI is how we extract and figure out what to serve that worker.

 

Tom Raftery: 11:32

Okay, very good. Now, my daily interaction with AR happens when I get into my car, because my car has a heads up display in front of it, which I absolutely love, it is so awesome. It’s one of my favorite things about a car, about my car, but it’s a very limited set of data, you know, gives me navigation gives me speed, road signs, all that kind of thing. And that’s by design, because you don’t want to be cluttering up to your point, you know, with information that doesn’t need to be there. In terms of the kind of AR that you’re feeding to people who were using your system, you mentioned a number of devices, and different information sets that are being served up. First of all, I guess in terms of the devices, is it a customized device? Or it cannot be any device? Is it up to the customer to provide the device? Is it kind of Microsoft HoloLens devices? Is it iPads? Is it, you know, what’s the hardware component? And then how do you decide what’s the appropriate information to display at any point?

 

Jon Newman: 12:43

You know, that’s a great question. So, what we find with our customers is they’re not looking for one particular device, they’re looking for a portfolio of devices to support these experiences on because they will provide different workers different tools in order to do their job. And so, we work on the widest range of devices. So, as we’re seeing everything from you know, iPhone, Android phone, all the way up to the more advanced augmented reality, wearables and the high end of the moment being the Microsoft HoloLens application. So what we find is a customer will choose to have a portfolio of devices that they’ll want to use Onsight on from that selection. And it’s, you know, it’s imperative for our customers that we provide the same consistent Onsight experience across a broad range of these devices. So, they can pick and choose also have access to that same shared information. So, no matter what device they log into, they will have access to all that answered information. And then what we do is we obviously tailor the Onsight experience. So that’s familiar across everything, but to maximize the peak of platform it’s on. So, what you’ll see on say, a phone will be more limited compared with what you’d see on an advanced AR platform like the HoloLens 2, which would be a much more advanced, much more immersive AR platform. And we see the HoloLens 2 is a great example of showcasing where we see this going. And the example we always, always talk about you mentioned about your car,  the Iron Man heads up display in Iron Man. And we see, I mean, that’s kind of the, you know, we can talk about all the technologies, but just to visualize in people’s minds. That’s kind of essentially where we see this all going. That’s our vision around the AI connected expert, where, you know, the worker will have some form of advanced AR device that they’ll be wearing when they’re working. And, you know, just like with Iron Man, where Jarvis is kind of like the mentor or the expert that’s coming along for the ride. That’s sort of where we see workforce experience ultimately going. So it’s always helpful we find to give that image in people’s heads of what we think the worker will ultimately have. And then it’s a journey to want to collect all that knowledge on the back end and be able to do that in a way that’s more automatic. And it’s not a huge human endeavor. And then to serve it in a way that’s contextual, and timely for the workers trying to do.

 

Tom Raftery: 15:17

Interesting, I get this kind of picture in my mind’s eye, not just of Jarvis. But then also if I think back a number of centuries, to the idea of a master and an apprentice, and an apprentice working away with a master looking over his or her shoulder, typically his because I’m talking a couple of centuries ago, but you know, what I mean. Is that kind of another way of putting it? Is that a good analogy as well?

 

Jon Newman: 15:40

Yeah, that’s actually an excellent one. I mean, we always use the analogy of the old-time expert, like Old Bill, you have a 30/40 year veteran. And when we talk to our customers, every customer has an Old Bill where they can go in and basically just listen to the sound of an engine or machine or just put their hand on it, I’m feeling that the vibration. Their expertise has gone to such a level that they can just do that. And all of our customers have someone like that. And so, it’s exactly that it’s trying to capture your Old Bill’s, wisdom and expertise, digitize it so it lives forever. And then the analogy you use of the master with the apprentice, having the virtual master that’s basically, you know, an Old Bill that, without wisdom lives forever, and can then continue to mentor, you know, younger and less experienced workers indefinitely. So that’s kind of the end game, and that how we see this evolving. And obviously, we’re taking steps on the knowledge capture side. And that’s what the Onsight Knowledge Network vision is around. And then on the sharing with the worker side, the AI connected expert vision.

 

Tom Raftery: 16:47

Okay, and how far away from that vision are we do you think?

 

Jon Newman: 16:52

So, about a year and a half ago is when we put out our AI connected expert vision to kind of showcase, you know, the Iron Man scenario, if you will. And so, we, in terms of executing that vision, then we have taken steps to add the additional third initial foundation for that. So, some of the key foundational tenants that we’ve added to Onsight to do that, you know, one is computer vision, so we can see what the worker is seeing and identify that. The other part is around natural language processing. We can hear and communicate with the worker and start to capture that expertise. The other part is around tying in with IoT or digital twin systems that our customers have. So instead of bringing in that raw data and visualize that. And then the last key pieces around the advanced augmented reality, so providing support for the HoloLens 2 and the other ones that come out, so you can have that true heads-up display AR experience. And so those initial building blocks, we actually have in the Onsight platform at the moment, and we put them in as foundational pillars. And then what we are doing is as we progress along this multi-year journey around, you know, the Knowledge Network and the AI connected experts, then we are building capabilities that solve specific customer business needs and challenges now, but on a journey to getting to this ultimate state. So we think, you know, over the coming years, we will start to get closer and closer to the vision that I that I outlined. And you know, probably in 5-10 years, we’ll be getting close to that Iron Man experience that has all the information automatically collected, analyzed, and can be surfaced to the workforce.

 

Tom Raftery: 18:39

Superb, I mean, you’ve mentioned customers a few times. Do you have good customer examples you can talk to?

 

Jon Newman: 18:45

The customers that we find are really pushing the envelope in terms of some of the advanced augmented reality and some of the advanced artificial intelligence usage, we find, it’s a lot of the aerospace and defense customers that are really pushing the envelope here, which totally makes sense, because even some of these new devices are quite expensive. And some of these technologies are expensive to implement as well. But for the defense and aerospace folks, obviously, their products that they’re working on are like aircraft missiles, you know, spaceships where the cost of an error can be millions or tens of millions of dollars.

 

Tom Raftery: 19:27

And lives.

 

Jon Newman: 19:28

And lives as well. Exactly. So, we find those customers tend to be on the leading-edge of these things just because the products are working on are a lot more valuable. And therefore, they want to invest as much as they can to ensure mistakes aren’t made, that their workforce, which needs to be highly skilled to work on these things, has access to everything that they need. And so, we find these folks tend to be at the at the cutting-edge of the adoption of this advanced technology and investing in these advanced use cases.

 

Tom Raftery: 19:59

Nice. Jon, we’re coming to the end of the podcast. Now, is there any thing that I’ve not asked that you wish I had? Or is there any topic we’ve not touched on that you think it’s important for people to be aware of?

 

Jon Newman: 20:12

Ah no, I think I think that covers about everything, Tom. I mean, it’s as you can tell, it’s an exciting time, the transformation that the workforce is going through and the challenges they have, and it’s a perfect intersection of their tremendous opportunity from the business standpoint, and the maturity of some of these key technologies in augmented reality and artificial intelligence to really solve some of these large challenges that are some of the big problems that we’re facing today with the workforce.

 

Tom Raftery: 20:43

Okay, great. If people want to know more about Librestream or about any of the topics we discussed today, where should I direct them?

 

Jon Newman: 20:53

So, you can find us on LinkedIn, Twitter, Facebook, and YouTube. And also, we have access to a WBR report that we have around AR and the field service industry that also touches on more of these themes as well. And I believe your listeners can access the full report via the in this podcast.

 

Tom Raftery: 21:14

Cool. I’ll include that link in the show notes. So, people have access to it. Great, Jon. That’s been fantastic. Thanks a million for coming on the podcast today.

 

Jon Newman: 21:21

Great. Thanks for having me, Tom.

 

Tom Raftery: 21:23

Okay, we’ve come to the end of the show. Thanks, everyone for listening. If you’d like to know more about digital supply chains, head on over to sap.com/digitalsupplychain or, or simply drop me an email at tomraftery@sap.com. If you liked the show, please don’t forget to subscribe to it on your podcast application of choice to get new episodes as soon as they’re published. Also, please don’t forget to rate and review the podcast. It really does help new people find the show. Thanks. Catch you all next time.