Researchdirections

Abstract We propose GANav, a novel group-wise attention mechanism to identify safe and navigable regions in off-road terrains and unstructured environments from RGB images. Our approach classifies terrains based on their navigability levels using coarse-grained semantic segmentation. Our novel group-wise attention loss enables any backbone network to explicitly focus on the different groups’ features with low spatial resolution. Our design leads to efficient inference while maintaining a high level of accuracy compared to existing SOTA methods.

Abstract Navigating large-scale outdoor environments requires complex reasoning in terms of geometric structures, environmental semantics, and terrain characteristics, which are typically captured by onboard sensors such as LiDAR and cameras. While current mobile robots can navigate such environments using pre-defined, high-precision maps based on hand-crafted rules catered for the specific environment, they lack commonsense reasoning capabilities that most humans possess when navigating unknown outdoor spaces. To address this gap, we introduce the Global Navigation Dataset (GND), a large-scale dataset that integrates multi-modal sensory data, including 3D LiDAR point clouds and RGB and 360° images, as well as multi-category traversability maps (pedestrian walkways, vehicle roadways, stairs, off-road terrain, and obstacles) from ten university campuses.

Abstract We present the Geometric-Wave Acoustic (GWA) dataset, a large-scale audio dataset of over 2 million synthetic room impulse responses (IRs) and their corresponding detailed geometric and simulation configurations. Our dataset samples acoustic environments from over 6.8K high-quality diverse and professionally designed houses represented as semantically labeled 3D meshes. We also present a novel real-world acoustic materials assignment scheme based on semantic matching that uses a sentence transformer model. We compute high-quality impulse responses corresponding to accurate low-frequency and high-frequency wave effects by automatically calibrating geometric acoustic ray-tracing with a finite-difference time-domain wave solver.

Abstract We propose GameOpt: a novel hybrid approach to cooperative intersection control for dynamic, multi-lane, unsignalized intersections. Unsignalized intersections are one of the more complex, prone-to-accident scenarios in modern transportation networks. Cooperation among Connected Autonomous Vehicles (CAVs) is a promising approach to unsignalized intersection control providing increased safety, efficiency and fairness. GameOpt is a hybrid formulation that first uses an auction mechanism to generate a priority entrance sequence for all the agents, followed by an optimization-based trajectory planner that computes the optimal velocity commands obeying the priority sequence.

Abstract We present a novel autoregression network to generate virtual agents that convey various emotions through their walking styles or gaits. Given the 3D pose sequences of a gait, our network extracts pertinent movement features and affective features from the gait. We use these features to synthesize subsequent gaits such that the virtual agents can express and transition between emotions represented as combinations of happy, sad, angry, and neutral. We incorporate multiple regularizations in the training our network to simultaneously enforce plausible movements and noticeable emotions on the virtual agents.

Abstract We present a novel algorithm for automatically synthesizing personalized walking gaits for a human user from noisy motion caputre data. The overall approach is robust and can generate personalized gaits with little or no artistic intervention using commodity sensors. Video Paper Generating Virtual Avatars with Personalized Walking Gaits using commodity hardware, ACM Multimedia 2017. Sahil Narang, Andrew Best, Ari Shapiro, and Dinesh Manocha @inproceedings{Narang:2017:GVA:3126686.3126766, author = {Narang, Sahil and Best, Andrew and Shapiro, Ari and Manocha, Dinesh}, title = {Generating Virtual Avatars with Personalized Walking Gaits Using Commodity Hardware}, booktitle = {Proceedings of the on Thematic Workshops of ACM Multimedia 2017}, series = {Thematic Workshops ‘17}, year = {2017}, isbn = {978-1-4503-5416-5}, location = {Mountain View, California, USA}, pages = {219–227}, numpages = {9}, url = {http://doi.

Abstract We present a novel algorithm for automatically synthesizing personalized walking gaits for a human user from noisy motion caputre data. The overall approach is robust and can generate personalized gaits with little or no artistic intervention using commodity sensors. Video Paper Generating Virtual Avatars with Personalized Walking Gaits using commodity hardware, ACM Multimedia 2017. Sahil Narang, Andrew Best, Ari Shapiro, and Dinesh Manocha @inproceedings{Narang:2017:GVA:3126686.3126766, author = {Narang, Sahil and Best, Andrew and Shapiro, Ari and Manocha, Dinesh}, title = {Generating Virtual Avatars with Personalized Walking Gaits Using Commodity Hardware}, booktitle = {Proceedings of the on Thematic Workshops of ACM Multimedia 2017}, series = {Thematic Workshops ‘17}, year = {2017}, isbn = {978-1-4503-5416-5}, location = {Mountain View, California, USA}, pages = {219–227}, numpages = {9}, url = {http://doi.

Figure 1: With differentiable GICP algorithm, we can train our neural network to infer importance, or weight, of each point in the point cloud that could be used in later GICP step. In the figure, points with lower weights are rendered in green, and those with higher weights are rendered in red. Note that points on grounds, which often exhibit non-uniform distribution, have generally lower weights than the others. Abstract We present a novel algorithm for learning-based loop-closure for SLAM (simultaneous localization and mapping) applications.

Abstract We present a novel trajectory traversability estimation and planning algorithm for robot navigation in complex outdoor environments. We incorporate multimodal sensory inputs from an RGB camera, 3D LiDAR, and the robot’s odometry sensor to train a prediction model to estimate candidate trajectories’ success probabilities based on partially reliable multi-modal sensor observations. We encode high-dimensional multi-modal sensory inputs to low-dimensional feature vectors using encoder networks and represent them as a connected graph.

Abstract We introduce a simple yet effective framework for improving the robustness of learning algorithm against (input) image corruptions for autonomous driving, due to both internal (e.g., sensor noises and hardware abnormalities) and external factors (e.g., lighting, weather, visibility, and other environmental effects). Using sensitivity analysis with FID-based parameterization, we propose a novel algorithm exploiting basis perturbations to improve the overall performance of autonomous steering and other image processing tasks, such as classification and detection, for self-driving cars.