We run courses focused on virtual reality, augmented reality, interaction design, and 3D user interfaces, offered to graduate students at Cornell Tech.
INFO 5358
3D Interaction Design
NEW COURSE
Spring 2025
Designing products and experiences for head-mounted displays, often involving novel inputs such as hands and eyes, introduces new challenges for designers when compared to traditional devices and user interfaces. This course presents the theoretical and practical foundations for designing 3D User Interfaces (3DUIs) for virtual and augmented reality applications. The course will highlight the differences between 2D and 3D interaction design, introduce design principles, best practices for prototyping, and case studies.
Students will be tasked with designing, implementing, and evaluating interaction prototypes of different fidelities for 3DUI tasks such as selection and manipulation, travel, and system control. Product design case studies incorporating head-mounted displays, wearables, and novel input devices will be introduced in the context of 3DUIs.
This course is open to students from all Cornell Tech programs; from design programs to computer science. The theory and low-fidelity prototyping skills acquired in this course can be applied to design, engineering, and product management in the field of AR/VR.
Instructor: Harald Haraldsson
Term: Spring (2025)
This is a new course, to be offered in Spring 2025. The lecture content in this course partially overlaps with CS 5678: 3D User Interfaces (not offered in Spring 2025), with more emphasis on the design process and contemporary case studies, and less focus on programming and high-fidelity implementations. Cornell Tech students who are interested in high-fidelity 3DUI prototyping and software engineering are encouraged to consider independent study at the XR Collaboratory, see FAQ for more information.
-
Required textbooks:
Joseph LaViola Jr. et al. 3D user interfaces: theory and practice. Second edition. Addison-Wesley Professional, 2017.
Other readings, such as research papers, will be assigned throughout the semester.
-
This course is open to all Cornell Tech students. No coding or prior tool experience is required.
Enrollment priority is given to students who have completed INFO 5340 / CS 5650: Virtual and Augmented Reality
-
The lecture content in this course partially overlaps with CS 5678: 3D User Interfaces (not offered in Spring 2025). However, this course focuses more on design and low-fidelity prototyping, and less on advanced technical implementation, especially for homework assignments and the course project. Students who have already completed CS 5678: 3D User Interfaces are not permitted to enroll in this course.
-
Topics include:
Foundations of 3D User Interfaces
Perception, Cognition, and Ergonomics for 3D User Interfaces
Human-Computer Interaction and 3D User Interfaces
3D Input & Output Devices
3D Interaction Techniques: Selection and Manipulation
3D Interaction Techniques: Travel
3D Interaction Techniques: System Control
Design Strategies for 3D User Interfaces
Evaluation of 3D User Interfaces
The Future of 3D User Interfaces
Case Studies
-
Homework & Project: Designing, implementing, and evaluating 3DUI prototypes
Quizzes: In-class quizzes from lectures and readings
Participation: In-class activities, attendance etc.
-
Demonstrate understanding of concepts from human-computer interaction in the context of 3D user interfaces
Demonstrate understanding of classic 3D interaction techniques and current trends, both in academic research and commercial product design
Demonstrate understanding of the unique challenges and opportunities when designing products and experiences with 3D user interfaces
Demonstrate the ability to design 3D user interface prototypes of different fidelities
INFO 5340 / CS 5650
Virtual and Augmented Reality
This course presents an introduction to virtual and augmented reality technologies, with an emphasis on designing and developing interactive virtual and augmented reality experiences. The course covers the history of the area, fundamental theory, interaction techniques, and specific application areas.
Concepts from the contributing fields of computer vision, computer graphics and human computer interaction are introduced in the context of virtual and augmented reality. Students are tasked with creating their own virtual reality application as a course project.
Instructor: Harald Haraldsson
Term: Fall
Course website
Assignments
-
![]()
Geometry
-
![]()
Meshes
-
![]()
Rendering
-
![]()
Locomotion
-
![]()
Interaction
Course Project
The course project focuses on the design and development of a custom virtual reality design tool, inspired by design applications such as Gravity Sketch and Arkio. Students are tasked with implementing a set features with the purpose of enabling their users to design a car in VR, using their tool.
Featured student project - Fall 2023
Connor Pugh, Gural Nuriyev, Sidharth Vasudev, Ruslana Yurtyn
Color picker feature
Mesh manipulation feature
Scaling feature
Undo/redo feature
-
Move object
Rotate object
Scale object (per axis)
Select single object
Select multiple objects
Mesh manipulation
-
Create object
Set shape for object
Change color of object
Delete object
Duplicate object
Undo / redo
-
Grab move locomotion
-
Visualize controller state
Help and documentation
CS 5678
3D User Interfaces
This course explores the field of 3D User Interfaces (3DUIs) and interaction design for virtual and augmented reality applications, including theoretical foundations, design principles, and best practices for high-fidelity prototyping.
Students are tasked with designing, implementing, and evaluating their own interaction techniques for 3DUI tasks such as selection and manipulation, travel, and system control for head-mounted displays.
Instructor: Harald Haraldsson
Term: Spring
Course website
Assignments
-
![]()
Balloon Selection
-
![]()
Go-Go
-
![]()
Pengufly
-
![]()
System Control
