Perceptual and Implementation Strategies for Knowledge Acquisition of Digital Tactile Graphics for Blind and Visually Impaired Students (Collaborative Research: Smith)

This project lays the foundation and framework for enabling digital, multimodal tactile graphics on touchscreens for individuals with visual impairments (VI). Given the low-cost, portability, and wide availability of touchscreens, this work promotes the use of vibrations and sounds on these readily available platforms for addressing the graphical access challenge for individuals with VI. An open-source vibration library has been created and fundamental perceptual building blocks (e.g.\ shapes, lines, critical points, line width and gaps, etc.) guiding how basic graphical components should be rendered on these platforms is being disseminated.

Project URL: 
https://quorumlanguage.com/
Full Description: 

Students with disabilities often have fewer opportunities for experiential learning, an important component of quality STEM education. With continued shifts toward the use of digital media to supplement instruction in STEM classrooms, much of the content remains inaccessible, particular for students with visual impairments. The promise of technology and use of tactile graphics is an effective, emerging innovation for providing more complete access to important information and materials. Tactile graphics are images that use raised surfaces to convey non-textual information such as maps, paintings, graphs and diagrams. Touchscreen-based smart devices allow visual information to be digitally and dynamically represented via tactile, auditory, visual, and kinesthetic feedback. Tactile graphic technology embedded in touchscreen devices can be leveraged to make STEM content more accessible to blind and visually impaired students.

This project will develop a learner-centered, perceptually-motivated framework addressing the requirements for students with blindness and visual impairments to access graphical content in STEM. Using TouchSense technology, the investigators will create instructional materials using tactile graphics and test them in a pilot classroom of both sighted and BVI students. The investigators will work with approximately 150 students with visual impairments to understand the kind of feedback that is most appropriate for specific content in algebra (coordinate plane), cell biology, and geography. Qualitative research methods will be used to analyze the video-based data set.

Recent Publications:

Tennison J., Greenberg J., E. Moore, and Gorlewicz, J. L. Haptic paradigms for multimodal interactive simulations. Journal on Technology and Persons with Disabilities. CSUN Assistive Technology Conference, March 2021.

Gorlewicz, J. L., Tennison, J. L., Uesbeck, P. M., Richard, M. E., Palani, H. P., Stefik, A., Smith, D. W., &   Giudice, N. A. (2020). Design Guidelines and Recommendations for Multimodal, Touchscreen-based Graphics. ACM Transactions on Accessible Computing (TACCESS), 13(3), 1-30.

Tennison, J. L., Uesbeck, P. M., Giudice, N. A., Stefik, A., Smith, D. W., & Gorlewicz, J. L. (2020). Establishing Vibration-Based Tactile Line Profiles for Use in Multimodal Graphics. ACM Transactions on Applied Perception (TAP), 17(2), 1-14.

Stefik, A. (2020a). Android Quorum; vibro-tactile application source. Retrieved Sept 20, 2020, from https://bitbucket.org/stefika/androidquorum/src/master/.

Stefik,   A. (2020b).  Vibration library in Quorum.  Retrieved Sept 20,  2020, from https://bitbucket.org/stefika/quorum-language/src/master/Quorum/Library/Standard/Libraries/Interface/Vibration/.

Palani, H. P., Fink, P. D., & Giudice, N. A. (2020). Design Guidelines for Schematizing and Rendering Haptically Perceivable Graphical Elements on Touchscreen Devices. International Journal of Human-Computer Interaction, 1-22. DOI: 10.1080/10447318.2020.1752464

Giudice, N.A., Guenther, B.A., Jensen, N.A., & Haase, K.N. (2020). Cognitive mapping without vision: Comparing wayfinding performance after learning from digital touchscreen-based multimodal maps vs. embossed tactile overlays. Frontiers in Human Neuroscience. 14:87. Doi: 10.3389/fnhum.2020.00087

P. Merlin Drews, Cole Peterson, Bonita Sharif, and Andreas Stefik. 2020. A Randomized Controlled Trial on the Effects of Embedded Computer Language Switching. In Proceedings of the 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE '20), November 8-13, 2020, Virtual Event, USA. ACM, New York, NY, USA, 11 pages. https://doi.org/10.1145/3368089.3409701

On The Human Factors Impact of Polyglot Programming on Programmer Productivity. Phillip Merlin Uesbeck. Ph.D. Dissertation. December 2019. University of Nevada, Las Vegas.

J. Tennison and J. L. Gorlewicz. Non-visual perception of lines on a multimodal touchscreen tablet. ACM Transactions on Applied Perception, 16(1):6, 2019.

 J. L. Tennison, Z. S. Carril, N. A. Giudice, and J. L. Gorlewicz. Comparing graphical pattern matching on tablets and phones: Large screens are not necessarily better. Optometry and Vision Science, 95(9):720{726, 2018.

Tennison, J. L., Carril, Z. S., Giudice, N. A., & Gorlewicz, J. L. (2018, August). Comparing haptic pattern matching on tablets and phones: Large screens are not necessarily better. Optometry and Vision Science, 95(9), 720–726.

Hahn, M., Mueller, C., & Gorlewicz, J. L. (in press). The comprehension of stem graphics via a multi-sensory tablet in students with visual impairment. Journal of Visual Impairment and Blindness.

Gorlewicz, J. L.,  Tennison, J. L., Palani, H. P., & Giudice, N. A. (in press). The graphical access challenge for people with visual impairments: Positions and pathways forward [Interactive multimedia]. London, UK: IntechOpen Limited.

Gorlewicz, J. L., Mueller, C. M., & Dahmm, H. (2018, October). Bringing digital graphics to life: The promise of multimodal touchscreens – Progress and future work. Presented at the National Federation of the Blind Tactile Graphics in Education and Careers Symposium, Baltimore, MD.

Palani, H. P., Tennison, J. L., Giudice, G. B., & Giudice, N. A. (2018). Touchscreen-based haptic information access for assisting blind and visually-impaired users: Perceptual parameters and design guidelines. In T. Ahram & C. Falcão (Eds.), Advances in Usability, User Experience and Assistive Technology: Part of the International Conference on Applied Human Factors and Ergonomics (AHFE’18). Advances in Intelligent Systems and Computing (798, 837-847). Cham, Switzerland: Springer International Publishing.

Palani, H. P., Giudice, G.B., & Giudice, N.A. (2018, July). Haptic information access on touchscreen devices: Guidelines for accurate perception and judgment of line orientation. Presented at the 20th annual conference on Human-Computer Interaction (HCI International ’18). Las Vegas, NV.

Palani, H. P. & Giudice, N. A. (2017, October). Principles for designing large-format refreshable haptic graphics using touchscreen devices: An evaluation of nonvisual panning methods. Presented at the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS’17), Baltimore, MD.

Project Materials

There is no content in this group.