Dr. Linda Ng Boyle, NYU Tandon School of Engineering: The Role Of Cars In The Future Of Work

Van Ton-Quinlivan

Hello, I’m Van Ton-Quinlivan, CEO of Futuro Health, welcoming you to WorkforceRx, an ongoing conversation with leaders and innovators offering insights into creating a future-ready workforce.

What if the daily commute in your car became one of the most productive parts of your day? Our guest today, Dr. Linda Ng Boyle of the NYU Tandon School of Engineering, is going to help us understand how close we may be to that level of human-machine interaction and what it means for the future of work and mobility.

Dr. Boyle focuses on these and related issues as director of the HumanFuel Lab at NYU. She’s also co-author of the textbook Designing for People: An Introduction to Human Factors Engineering, and is a member of the prestigious National Academies Board on Human Systems Integration.

Thank you very much for joining us today, Dr. Boyle.

Linda Boyle

Thank you for inviting me. I’m very happy to be here.

Van Ton-Quinlivan

So, Linda, let’s start today by talking about the current state of affairs. What type of workers work in their car today and what are some of the issues that human drivers face while working in their cars?

Linda Boyle

Yeah, there’s actually a lot of different types of workers that work in their car, probably more than people realize. In addition to those who just drive for a living, you know, there are those who actually travel for their jobs like sales reps, real estate agents, home health aides. They all are already working in their cars. But increasingly, knowledge workers, those who think for a living, are also using their cars for work. For example, lawyers, consultants, counselors, they may be taking client calls between meetings, executives reviewing briefs during their commute, data analysts looking at data summaries on the road. But in general, the everyday commute actually uses their drive for catching up on calls, listening to briefings, or even mentally prepping for the day. So many people are really using their cars right now.

Van Ton-Quinlivan

So what’s your assessment of whether current technology actually supports them in terms of using drive time or time in their car for productive work?

Linda Boyle

So in terms of the technologies that are supporting the drive time, there’s actually quite a few right now. Voice interfaces are actually very important. They’ve come a long way. We can do hands-free calls, dictations. There’s also some of them come with smart assistants that can actually set reminders or send messages. We have other technologies that are more driving-based, like adaptive cruise control, lane keeping systems that can offload some of the physical driving burdens, freeing up mental resources for lighter cognitive tasks. We have navigation systems that reroute around traffic to reduce the intentional demands of wayfinding. But that said, we are still largely in the era where technology supports simple linear tasks rather than complex interactive work, where most professionals actually need to do those types of more complicated work issues.

Van Ton-Quinlivan

So when we talk about the issues that human drivers face when trying to work in their car, you sort of reference this that the technology only supports simple linear work. So what are the human factors we should be thinking about?

Linda Boyle

Yeah, good question. So some of the issues associated with the human, basically the most obvious are the physical constraints. Like for example, it’s really hard to take notes or read a screen or work even on your laptops while you’re driving. So working in a car can still be difficult. And then people are also still prone to motion sickness, especially depending on what the vibration is and what’s going on in the car. So the temperature, the air circulation, and lighting all need to be just right for some people. The main issue though is that driving is really demanding and can actually impact what you’re doing cognitively. So more specifically, work tasks actually compete for the same mental resources as driving. So the driver is not necessarily good at multitasking, but they are probably doing more what’s called task switching, which is they’re switching between tasks from driving as well as other activities. And so because of this, it’s difficult for them to dedicate fully to each of those tasks.

Van Ton-Quinlivan

All right. That sounds like it invites, if we want to be safer, it invites us to have autonomous vehicles where we’re not driving, right?

Linda Boyle

Yeah, that’s a good point. I believe that when we do move over to autonomous vehicles, it’s going to be a really dramatic shift for a lot of workers. As more of the driving responsibilities will be handed over to the autonomous vehicles, there will be more people using their car as a mobile extension of the office. Now that being said, when we envision what that autonomous vehicle looks like, it’s not just about the systems, but also about the interior. So the physical constraints I mentioned, many people are thinking about how we actually redesign for productivity. We would probably need to make sure that there’s better lighting, the acoustics, having swiveling seats, larger displays, and even leveraging the windshields and the side windows as like a digital whiteboard can be very productive. Now that being said, we have to make sure that when we’re moving forward to autonomous vehicles, while we can be more productive, we also want to make sure that we can safely get to our destination, on time and comfortably, but also safe. To me that’s the most important thing.

Van Ton-Quinlivan

Well, so to make this future happen, basically the drivers completely decouple from the driving aspect of the car in order for our cognitive load not to overheat.

Linda Boyle

That’s right. That’s right. And that’s actually a challenge right now because no car on the road is really fully autonomous where the human still needs to kind of be able to be attentive to what’s going on around us. I think some of the functionalities or features that are most promising in autonomous vehicles is understanding how to seamlessly have the car basically be able to take over when they should or hand off control, and being able to make sure that the humans know when they’re supposed to be taking over. So the systems basically need to be able to kind of work alongside the humans to be able to reengage and basically work together on these issues.

Van Ton-Quinlivan

Do you think that there’s a future where the automobile is being intentionally used as that workspace? I suppose the decoupling of our workplace and the whole telecommuting, telework transition leads in that direction. I know that there’s a house next to us where now they’ve just converted the whole living room kitchen space to a co-working space where all the renters are coming down to the first floor in order to telework. So things are getting very creative these days.

Linda Boyle

That’s right. And I would say that’s the same thing for the car. I think the car is being really considered about what does that mean to redesign the interior so that we can increase productivity and comfort. And that includes many different things. I already mentioned about lighting, noise cancellation systems, swiveling chairs. People are also thinking about, when you’re working, it’s more than just the work itself. You want a place to maybe have coffee, maybe review work material and even take a break or basically be able to relax. You can kind of think of this as like maybe a private mobile office in like a first class train compartment. What would that look like? And what is it that we should think about to design for productivity and comfort? Now this all being said, I do think it’s important that from a safety perspective, how do we actually make sure that we have context-aware interfaces? So what I mean by that is if you’re working on complex tasks while you’re driving, you want the system to be able to say, this is not a good time to be working on this because you’re in heavy traffic. You want to be able to have these good voice interactions so that people can actually pay attention to the road. And this is actually particularly true for partial automation, because as we had discussed earlier, the human still needs to be absolutely attentive.

Van Ton-Quinlivan

So part of me is grounded in today, and as you’re speaking, I’m thinking, okay, wait a second, am I moving my cubicle from the office right into the house and then creating that kind of experience within the vehicle? But in a prior project that Futuro Health had with Nissan Japan R&D along with Stanford, we were looking at what’s the use of the car, what’s the future of mobility, especially when we’re not driving the car and the intersection with health. And some of the concepts were about, for example, having the privacy to do AR/VR as part of mental health treatment, which is a much more immersive use of the car, not just moving your cubicle over. So as you’re thinking about the future of mobility and how it intersects with potentially the future of healthcare, for example, are there concepts that come up or occur to you that we may think differently about how these vehicles could be used?

Linda Boyle

Yeah, I think, because you mentioned healthcare, I think that’s one of the biggest areas that working in the cars can actually benefit. So for example, the home healthcare person can actually get everything ready and prepped inside their vehicle before they actually meet with the patients. They can actually set up maybe virtual check-ins during their commute. The clinician could review case notes, for example, between visits. So there are many ways that we can actually use the cars to do additional work before they actually interact and meet with the patients, and then the healthcare provider would actually be more prepared. Now this being said, exactly what you pointed out, how do you actually design that workspace inside the car so it can actually have the right equipment for them to be able to look at x-rays, for example, maybe do some sampling, have kind of the environment so that they can actually take care of many of these things.

So that would be one area that they can look at. Another one is, right now, they could — we could set it up so they can actually maybe even meet with patients in the car. And that would be something really far out, but maybe something that could be very helpful for those who actually don’t have the mobility to move outside or even go to a medical facility. So there are many ways that we can look at this, and I think healthcare is one area that could actually greatly benefit from working in the car.

Van Ton-Quinlivan

That’s true. That would allow care to move all the way to the home, and especially in communities that are far out. Is there a world where you can mix and match component parts of cars? So like if I’m one type of healthcare worker, I have a different — I guess the equivalent of a briefcase of equipment. Whereas if I’m a different type of healthcare worker, maybe I’m a dietitian, then I’m bringing a different set of equipment or types of luggage, right?

Linda Boyle

Yeah, absolutely. So it’s the same thing with all types of workers. Like even if you’re a lawyer, you could be different types of lawyers as an example, and you may need different things inside the vehicle. So those are all things that from a research perspective we think about — how do we actually design the interior of the car so that it is adaptable, but that there are certain things that while you’re in motion you need to make sure is set up. I had mentioned earlier about just the ambient lights, or just lighting in general, the noise, even the vibration — when you’re driving, there are things that can actually greatly impact how well you can actually interact with different people while you’re in the car. And how do we actually design the car so that there are certain things that can be portable or replaceable depending on the profession that you’re actually working in.

Van Ton-Quinlivan

What innovation would you be most excited for your students to work on once they leave their schooling?

Linda Boyle

Yeah, so one thing that we’ve been kind of looking at is how we actually integrate a lot of — my students, they do a lot of work on data, and every profession that we’ve been kind of looking at, they have to use data in some way, even plumbers. They basically have to look at the different types of data and be able to kind of digest it and use it and think about it before they actually meet with customers or patients or other people. So how do we actually design the vehicle so that they can actually look at this type of information inside the car? And understanding — you had mentioned VR/AR — how do we actually use things like the doors or the windows to basically project this type of information in a way that they can actually quickly decipher it, make decisions based on it, and then immediately go back to the driving task if they have to take over? And what does that challenge require? Because oftentimes when you’re looking at data, you have to really think about it and you can’t just look at it, come back, and then think in five minutes you can do something with it. So you have to really think about how do we actually put in stopping points, how do we set it up so that people can highlight information or be able to go back and digest it in some way, synthesize it in a way that would be usable for the people that they want to give this information to.

Van Ton-Quinlivan

The future that you’re laying out sounds so fascinating to imagine, but yet if I pull it back to today, I’d be petrified because we’re already highly distracted by our texts. Between the texting and the GPS and trying to fuss with the radio, there’s already a lot of moving parts in the car. So you’ve done a lot of research on distracted driving and I want to get your thoughts on the difference between what we can do in a car versus what we should do in a car.

Linda Boyle

Yeah, that’s a really good point. So technologically, if we think about it, we can do a lot of things. We can do almost anything in a moving car. But of course that does not mean we should. So for example, just because I can answer a complex email while eating breakfast at sixty miles per hour, that does not mean that I should, because it will impact my driving performance as well as my work quality. So what we look at is that the work team basically tries to understand how do we actually design systems that can provide good feedback to the human. And because a lot of humans are not really good at assessing their own driving ability as well as whether or not they’re impaired — so when they’re distracted, they think that they can still do all these different things — we look at basically using the technology to help mitigate the impacts of the technology and how to provide feedback at the right time, what kind of alerting systems that we should be designing for the human so that they can actually respond in the right moment in time. And also because technology, as we all know, is not perfect, we also want to make sure that the humans accept and use it and trust it at the appropriate time, and not to overtrust or use it in a way that uses that system to compensate for other things. So these are all research questions that are really important, trying to understand how we design the system so that it takes into account individuals’ confidence level, but also recognizing what their limitations are.

Van Ton-Quinlivan

Off Linda, designing some infrastructure where maybe the car gets to park and then we set up our mobile office and then we can drive to our next destination and park — because if humans fundamentally can’t multitask, then we’re creating our own problems by driving and thinking hard, right?

Linda Boyle

Right. Yeah. So we actually look — there are different ways that — right now in the car of today, there are systems that people are looking at to say, this is not a good time for you to be doing certain types of tasks. And how do we actually monitor that information and capture it? So that we can give the human information that says, it doesn’t look like you’re paying attention, you may need to pull over. And if you’re not going to pull over, is there a way that we can get the system to pull over? And I want to be very careful when I say this, because if I’m on a freeway going at like sixty-five miles per hour, you can’t just stop the vehicle. You have to find a safe place to be able to move it over to the shoulder, or maybe get off the freeway somehow. So there has to be this interaction or this collaboration between the human and the system to be able to work together to make sure that you can be safe, whether it’s to pull over, to slow down, or just to tell them to stop working on whatever secondary tasks they are working on at that moment in time.

Van Ton-Quinlivan

I’ve noticed that the chairs in rental cars buzz if you start to not drive in a straight line. So I assume those are some of the human factors engineering product results. So you have a philosophy in terms of the driver and the technology being co-controllers and constantly adapting to each other. Is there anything more you’d like to add on that?

Linda Boyle

I think right now — I mentioned this earlier — the systems that we have designed are not really tailored toward individual differences. So it’s basically going to work one way. There’s like even an adaptive cruise control — there are certain settings that you can set so that the warning provides you information that’s really close or really far away, but it’s not tailored to the individual person. So I think moving forward, one of the things that people are looking at is how do we actually have more of a better relationship with the system, and how do we have the system kind of understand what your specific limitations are, which may not be everybody else’s limitation? And then how do we actually provide feedback to the driver in a way that they will actually respond? Like if you have a passenger in your car and they kind of gauge the traffic along with you and they even gauge the conversation such that if there’s a lot of traffic they will actually kind of change the way they’re speaking to you — and it’s the same thing with the systems. How do we actually change the way they’re providing information to you such that you would be more willing to kind of work with it.

Van Ton-Quinlivan

So, Linda, of course, I have to ask you the AI question. You talk about human AI teaming. What is that about?

Linda Boyle

Yeah, so earlier on I was talking about the humans and the systems collaborating together. But in reality, the systems that we’re using in the car, they’re all going to be powered by some model that is based on AI, artificial intelligence. And so what I mean by that is that the system is actually learning about you as the driver, as the human operator in the car, while the human operator is actually also learning about your imperfections and how to work with you.

And so similar — and I mentioned this earlier — similar to having a passenger in the car or your spouse or a friend, you’re kind of learning over time what that interaction is. And you gauge your conversations, you gauge your interactions based on how well you know each other. And it’s the same thing with you being with your car or working with your car or interacting with it. You’re learning about what the imperfections are or even the good parts of your car.

And the car is actually learning about your behavior as it’s learning about how you’re driving over time. And so that is a relationship that you’re trying to kind of work together, so that when you get to some point you’re working collaboratively — not one telling the other person to do something versus the other. You’re working collaboratively to learn about what your imperfections are, but at the same time that becomes a perfect relationship. So that you can basically make sure that you’re on the road safely and that you know how the system can help you and not help you.

Van Ton-Quinlivan

The cars in the last few years — we were in the market for a used car for our college-age son, and the last few years have fully integrated the Apple CarPlay function. And the car’s capabilities are now tied to the Apple CarPlay. So do you foresee a day, for example, where your biomarkers or anything like that being read by your phone would be one of those indicators that affects your driving?

Linda Boyle

I do. I do. And I’m going to go a little bit further. I actually think you may not even need your phone. I think just being in the car, there are going to be certain types of biomarkers. So I’ll give you an example. Just as a driver, we happen to always sit in the same place. After a while, you can put sensors on the cabin — and there already are in some vehicles, because that’s how they tell you, like it could shake if you go too far off to one side, like provide a rumble strip. But anyway, so because you’re in the car for so long and it’s usually the same person going in, you can get a biomarker of that person such that if you see that person’s not sitting the same way that they normally do, it may be an indication that they may have some sort of a health issue that they may not have had in the past few weeks that they’ve been kind of observing you. Or it could be that all of a sudden your temperature may seem a little bit higher than what it normally is, and that you might be having some sort of a medical condition — and so the car can actually be able to detect that over time. And I think that could be something that would be very useful for humans, to be able to understand your physiological kind of state and being able to help you make sure that if you’re driving or if you’re somehow in motion, that we can give you some advice about whether or not you should pull over, go to the hospital, maybe get a refreshment right now. And I think that would be very far out in the future, but I think it is a possibility.

Van Ton-Quinlivan

And so to be able to measure or sense the physiology — how would you get to that place where the car can read that? Is it a set of new hardware? New software?

Linda Boyle

Yeah. So for example, in some vehicles right now, if your hand is not on the steering wheel for a long period of time, it’ll say, I noticed that you haven’t put your hand on the steering wheel, could you just touch it so that I know that you’re still with me and that you’re paying attention. So already there are sensors in different places in your vehicle. There are sensors already in the steering wheel, there are sensors in your seat pad. And then sometime in the future, the ambient temperature, the way it gauges it, it could actually provide some information about that as well. And they’re not all in cars right now, but I can see that that could be a possibility moving forward, because the technology exists for a lot of these different things. Like if you talk about CarPlay, your Apple phone has a health monitor, so it knows how many steps that you’ve taken, it can actually look at your heart rate. And so it’s the same thing. You would have these different types of information that you can actually gather, and if you’re measuring it over time you can kind of see a pattern of what’s called normal for that particular individual. And if you see different spikes all of a sudden, that could be an indicator of something that may be off. And the system can say, it seems you’re a little bit off from the past week. Is everything okay? kind of a thing, and then maybe help you kind of redirect you a little bit.

Van Ton-Quinlivan

My mind is going with all the possibilities. Like if my mood is not good, maybe it’ll turn on my favorite music and get the temperature right, maybe pop out a little soda as a treat. How fun.

Linda Boyle

Yep, exactly. And that’s — because honestly many people spend a lot of time in their vehicles now. And so if we can make that environment more — not just productive, so even though we’re talking about the future of work, it’s actually more comfortable and gives you more inner peace, you can say, to help increase overall health and well-being. One thing that we were talking about with the future of work earlier, and the fact that we want people to be more productive — but there are also negatives associated with being productive, right? Which is that while you can maybe get more billable hours, where does the difference between home and work actually get divided? And if you’re constantly on call twenty-four/seven, that can have great impact on your overall health and well-being. So what are some things that we can do inside the car to help with that? And it’s not just about the productivity, but it’s also about the comfort and having more inner peace — and what are some ways that we can provide that? And the same technology that can help you be more productive can also help you have more relaxation as well.

Van Ton-Quinlivan

It’s true. In the days when we used to commute, there was at least a time when you were on the road to decompress from coming home from work, and now there’s no decompression between your office and moving to other spaces in your room. But I could see the car perhaps transitioning the ambience and the mood and whatever it needs to do to help you kind of make some mental transitions. By the way, for those of us who sit all day long because we’re on calls, it sounds like we sit in the office and then we’ll sit in the car. As you do human factors engineering, do you think about our posture and how we’re bent in the neck and all of that? Any advice on that front?

Linda Boyle

Yes, absolutely. There are a lot of people from the design side that look at, going back to comfort, what can that include? And so when we think about that seat inside our vehicle, we want it to be as comfortable as possible. I mentioned for productivity and work, we want to be able to swivel around, have meetings inside the car. But in terms of comfort, we want to be able to recline. It could be a massage chair. We could probably create an environment where we can actually convert it into a bed so that we can lie down. So there are many ways that we can look at creating an environment where we can not be so physically taxing on our bodies.

Van Ton-Quinlivan

And what about human factors engineering and thinking about the rapidly aging and growing population of older workers and older adults who rely on driving or the car for independence? Is there a lot of activity around pre-thinking some of the solutions?

Linda Boyle

There is. And I’m going to add to it not just older individuals but people with disabilities or people with mobility impairments. I think this is one of the most promising applications for fully autonomous vehicles, and I don’t think we have been doing enough research on it. I don’t think we’ve been giving it as much attention as we should. I think one of the things about older individuals is that the independence, their ability to be independent, to have social connections, and also even being able to do some work activities outside — not a full-time job, but just being able to be cognitively aware of their surroundings — is very important. And I think autonomous vehicles help extend that mobility, particularly for these populations. It gives them an opportunity to go places that they may not normally go, be able to create different types of interactions with people in places, different geographical areas that they may not have normally been able to go to. So I think that’s really important. I think also going back to what we were talking about with the design of the car, we can actually set it up so that we can maybe help enhance some of the vision and hearing that goes on in the car for different things. Basically being able to understand how we design that user interface in the car so that it’s tailored for specific populations so that they can feel like they’re part of the community and part of being active — I think that is one of the biggest things that we could do with autonomous vehicles.

Van Ton-Quinlivan

I wonder if there could be a future where the car comes to you as someone who is older and with mobility issues, and then you can go into the car and experience a more three-sixty immersive experience of engaging with friends who are not nearby. That would be so helpful in our society because we have such growing issues with regards to isolation and loneliness.

Linda Boyle

Yep, exactly. And I used to work at the University of Iowa and that was one of the biggest concerns. We have a lot of rural communities with people with limited mobility, and one of the things was how do we actually make sure that we can enhance their mobility, have them continue to be part of the community, and have a good group of people that they can basically interact with on a regular basis. To me, I think there’s a lot of research about how that actually not only helps improve the quality of life, but also how long they can live too. So it’s really important. And I do think that if we design autonomous vehicles correctly so that we can provide these opportunities — for again, not just the elderly but also for those who have limited mobilities in other ways — I think that is a very important contribution that we can provide.

Van Ton-Quinlivan

And Linda, this is an introduction for me to human factors engineering and it’s so fascinating. What kind of person or student or learner should consider a career in this area?

Linda Boyle

Wow, that’s a great question. My students actually come from many different areas, but primarily for me it’s mostly in psychology and engineering. Those are the two areas — the psychology people are very interested in human behavior, and in engineering they’re very interested in design. And human factors engineering takes into account the combination of both the behavioral side and the understanding of design, and also analytics — understanding how to actually look at the data and be able to design systems based on what we understand about what’s going on in the world and how we want the world to be. And so understanding how to bridge that gap is a really important contribution. So primarily those are the two areas, engineering and psychology, but I’ve also had students who are statisticians, who are sociologists, who work in public health. So all of these are areas — anyone who’s interested in understanding how to make systems better or technology better for humans would be, I think, very interested in human factors engineering.

Van Ton-Quinlivan

I had so much fun learning from you and thinking about possibilities as we talked. All right, so let me close right now by asking what makes you optimistic about the future of work?

Linda Boyle

I think, kind of related to your last question, this is a real interdisciplinary group of people that are working on this and I really love that. We have engineers, human factors, urban planners, public health — there are a lot of different people in addition to the people that I mentioned. And we’re not looking at just what the technology can do, but what do people actually need. And how do we design the system so that the humans can actually have the appropriate trust in the systems, that they know how to transition between the human and the system and work along together, so that we ensure that everybody that is in our transportation network has equal access to facilities, to transportation, to education, to health, and that the car could be one way that we can make sure that people maintain that productivity but be safe as well.

Van Ton-Quinlivan

Thank you very much, Dr. Boyle, for being with us today.

Linda Boyle

Yeah, thank you. Thank you so much again for inviting me.

Van Ton-Quinlivan

You got us thinking hard. I’m Van Ton-Quinlivan with Futuro Health. Thanks for checking out this episode of WorkforceRx. I hope you will join us again as we continue to explore how to create a future-focused workforce in America.