by Ben Levine

February 12, 2018

In this installment of the Innovation of the Month series, we explore the use of smart technologies to help blind and visually impaired people better navigate the world around them. A team at Ohio State University has been working on a "smart paint" application to do just that.

MetroLab's Executive Director Ben Levine sat down with John Lannutti, professor of materials science engineering at Ohio State University; Mary Ball-Swartwout, orientation and mobility specialist at the Ohio State School for the Blind; and Josh Collins, chief technology officer at Intelligent Material to learn more.

Ben Levine: Could you please describe what "smart paint for networked smart cities" is? Who is involved in this project?

John Lannutti (OSU): The goal of "smart paint for networked smart cities" is to assist people who are blind and visually impaired by implementing a "smartpaint" technology that provides accurate location services. You might think, "Can't GPS do that?" But, surprisingly, current GPS-based solutions actually cannot tell whether somebody is walking on the sidewalk or down the middle of the street. Meanwhile, modern urban intersections are becoming increasingly complex. That means that finding a crosswalk, aligning to cross and maintaining a consistent crossing direction while in motion can be challenging for people who are visually impaired.

And of course, crosswalks aren't the only challenge. For example, our current mapping technologies are unable to provide the exact location of a building's entrance. We have a technology solution to those challenges. Smart paint is created by adding exotic light-converting oxides to standard road paints. The paint is detected using a "smart cane," a modified white cane that detects the smart paint and enables portal-to-portal guidance. The smart cane can also be used to notify vehicles, including autonomous vehicles, of a user's presence in a crosswalk.

As part of this project, we have a whole team of educational, city and industrial partners, including:

Educational partners: Ohio State School for the Blind-- testing and implementation of smart paint technology in Columbus involving both students and adults

Western Michigan University-- implementation of smart paint technology with travelers who are blind and visually impaired to maximize orientation and mobility

Mississippi State University--the impacts of smart paint technology on mobility and employment for people who are blind and visually impaired

City partners: Columbus Smart Cities Initiative--roll out of smart paint within Columbus and the paint's interaction with the Integrated Data Exchange (IDE), a cloud-based platform that dynamically collects user data to show technological impact

The city of Tampa, Florida--roll out of smart paint at the Lighthouse for the Blind

The Hillsborough Area Transit Regional Authority, Hillsborough County, Florida--integration of smart paint with existing bus lines to enable precise location determination

The American Council of the Blind--implementation of smart paint with the annual American Council of the Blind convention

MetroLab Network--smart paint implementation in city-university partnerships

Industrial collaborators: Intelligent Material--manufactures and supplies the unique light-converting oxides that make the paint "smart"

Crown Technology--paint manufacturing, product evaluation and technical support

SRI International--design and manufacturing of the "smart" white cane hardware

Levine: Can you describe what this project focused on and what motivated you to address this particular challenge?

Lannutti: We have been working with Intelligent Material in integrating light-converting oxides into polymeric matrices for specific applications for several years. Intelligent Material supplies these oxides for highly specialized applications across a variety of industries, and has deep experience in filtering and processing the resulting optical outputs. They were already looking at using this technology for automotive applications when the idea to develop applications for people who are blind was introduced. We were extremely fortunate to have the Ohio State School for the Blind (OSSB) right here in Columbus and even more fortunate to have interested collaborators there who have helped us at every step of the way. They even have a room filled with previous white cane technologies; we used those to better understand what works and what doesn't, helping refine our own product. At about this same time, the National Science Foundation released a call for Smart and Connected Communities proposals, which gave us both a goal and a "home" for this idea.

Levine: What need motivated this collaboration? Why did you decide to partner with Ohio State University to meet this need?

Mary Ball-Swartwout: At Ohio State School for the Blind, we work with students of all ages and abilities. OSSB's orientation and mobility team, which includes me, Phil Northrop and Rachel Smith, is very familiar with the challenges our students face in navigating cities. Orientation and mobility training is essential for students' navigation within the outside world, but we are always searching for additional technologies that could make our students' travel more efficient. When Ohio State University and Intelligent Material approached us, our team was very interested because their technology better integrated into the modern world than past products. None of the prior products have created a lasting benefit for many of our students, and we enthusiastically support any new tools that help our students to enhance travel skills.

Levine: How will the tools developed in this project impact planning and the built environment?

Ball-Swartwout: One of the great things about smart paint is that it can be added to the built environment easily at little extra cost. We expect that once smart paint is widely adopted, most sighted users will not notice much difference as smart paint is not visually different from regular road paint. Some intersections might need to have more paint features that enable smart white cane-guided entry from the sidewalk into the crosswalk. Paint that tells users that they have reached their destination may become visible as horizontal stripes along modern sidewalks. These paints could be either gray or black or even invisible to sighted pedestrians, but would still be detectable by "smart" white canes to tell users that they have arrived at their destination.

Levine: What business need motivated this collaboration?

Josh Collins (Intelligent Materials): We specialize in the design and manufacturing of rare-earth doped nanocrystals capable of converting light energy up and down the visible spectrum. Our core Intelligent Material technology enables up to parts-per-billion sensitivity of detection. Nothing found in nature converts light in the same way and the prospects for widespread application of these patented oxides into road and sidewalk paint worldwide are very attractive to us from the business perspective. Discovering a range of additional business and academic partners dedicated to helping people who are blind and visually impaired has been very encouraging.

Levine: Can you tell us about the new technologies that are associated with this project? Can you talk about the status quo versus your vision for the future?

Collins: Beyond converting ceramics in paint, placing a highly sensitive excitation source and detector package at the tip of a moving white cane is truly novel. Also challenging is powering this package using minimal battery weight to decrease the likelihood of wrist and upper neck fatigue. The paint behaved normally during the installation. The status quo is that the travel of citizens who are blind and visually impaired can be unpredictable. They need better technologies for routine travel and especially for travel to any new destinations. In addition, we anticipate that this technology could assist in the travel of people who have a variety of physical and cognitive impairments. Our vision for the future of this technology is that it will be widespread and utilized constantly. Outside the U.S., Japan and Europe have integrated relatively expensive technologies into streets and sidewalks, and we see smart paint replacing that very quickly. Because the "pain" of installing smartpaint is very small, we believe that grass-roots pressure will enable rapid introduction of this technology.

Levine: What are some of the other applications for this technology?

Collins: It turns out that the same technology that informs users who are blind which way they should go can be used to tell airport ground vehicles which way they should not go. Runway incursions of ground vehicles in an active airport could lead to terrible accidents if a rapidly moving jet were to collide with one of these vehicles. These incursions can involve vehicles, general aviation traffic and other large air carriers. To address this, we are also looking at deploying smart paint at airports. We have worked with OSU to successfully add smart paint to "hold bars" at the Ohio State University Airport in Northwest Columbus. This enables unique optical signatures to provide warnings to both ground vehicle operators and pilots to eliminate incursions. The corresponding visual, audible or tactile warning signals triggered by passage over the smart paint will prevent unauthorized ground vehicle or plane passage. This will satisfy the need for updated safety enhancements that benefit airport operational performance.

Levine: What was the most surprising thing you learned during this process?

Lannutti: In my mind, the most surprising thing was discovering that sound was not necessarily the best means of guiding users who are blind. This is a bias on the part of sighted individuals as they are used to beeping and buzzing noises that guide or inform them throughout our day. Pedestrians who are blind,on the other hand, need to constantly listen to aspects of their environment to fully navigate it. For example, listening to traffic noise is extremely important to them as a means of avoiding danger. People who are blind or visually impaired cannot see but need to hear their environment. So we had to dial back our expectations regarding the utility of sound. Instead, we now focus on vibration along the white cane as a means of alerting the user.

Ball-Swartwout: The OSSB orientation and mobility team has been surprised to learn how rapidly technology is advancing to give people with visual impairments greater potential independence. Also, the high level of interest and excitement on the part of the staff and students at the OSSB regarding this project has been amazing.

Levine: Where will the project go from here?

Lannutti: We are preparing for a proposal submission to the National Science Foundation at the end of February. We are looking for considerable support from industry, particularly in the area of software development. We believe that with the right partner this idea can easily create a "Google Maps for People who are Blind," thereby increasing the travel abilities of a wide range of individuals who have disabilities.

Ball-Swartwout: Thanks to the city of Columbus, we now have several acres of smart paint applied at crosswalks and sidewalks across our campus. We are looking forward to Institutional Review Board-approved testing of this technology, utilizing our students and adults who are blind and visually impaired in the community. Professors Dae Kim and Robert Wall Emerson from Western Michigan University will visit us to work with our students in quantifying crossing efficiency and other safety factors associated with smart white cane-assisted travel.