NavWareSet: A Dataset of Socially Compliant and Non-Compliant Robot Navigation

Top-left: Lidar point cloud of the robot's environment;
Top-right: Robot's own sensory data;
Bottom-left: Synced point cloud with pedestrian's positions and robot's pose;
Bottom-right: Extracted pedestrian and robot trajectories with environment map.

Overview

Wide range of social navigation scenarios with both individual and group interactions.
Over 192 minutes of interaction data and over 172 minutes of annotated trajectories.
Over 1000 individually annotated human tracks and over 600 robot tracks.
Sensory data from the perspective of 2 different robots (Toyota HSR and Clearpath Jackal).
Stationary Ground-Truth Recording Station (GRS) with video camera and 3D LiDAR scans.
Occupancy grid of the obstacles in the environment.
Recording of both social and non-social behavior of the robots.

Experimental Setup

Environment

The dataset was recorded in a broad garage space measuring 3.9 m × 10.7 m with a lateral opening making it possible to elicit tracks in a straight line or around corners. The environment is equipped with a Ground-Truth Recording Station (GRS) that includes a video recorder and a 3D LiDAR scanner. For each scene goal positions were placed around the map according to different social navigation scenarios.

Ground-Truth Recording Station (GRS)

The Ground-Truth Recording Station (GRS) is equipped with an Intel® RealSense™ Depth Camera D455 and a Robosense RS-LiDAR-16 3D LiDAR. Both sensors are mounted on a custom 3D-printed frame, which is positioned on a tripod for stable data collection. You can interact with the frame in the 3D viewport above (click and drag to rotate) or access the 3D model here.

To minimize occlusions, the GRS was positioned at a height of 2.17 m above the ground. To enhance the resolution of the LiDAR data, the GRS was tilted 15° downward toward the area of interest.

Robots

The robots used in the NavWareSet dataset are the Clearpath Jackal and the Toyota HSR. The Jackal is a small, rugged robot designed for outdoor and indoor environments, equipped with a 3D LiDAR sensor and an RGB-D camera. The HSR is a versatile human-support robot capable of performing household tasks, featuring a manipulator arm, a mobile base, and various sensors for safe and intelligent interaction with people and objects. In most scenarios both robots were teleoperated by a human operator, first in a socially aware manner, and then in a non-aware manner.

Experiment description

The experiment was designed to capture common social navigation scenarios. These scenarios were proposed by Francis et al. in their paper, Principles and Guidelines for Evaluating Social Robot Navigation Algorithms. They serve not only to facilitate data collection but also to provide a standardized basis for evaluating the performance of social navigation algorithms. A detailed description of the scenarios used in NavWareSet is provided in the table below.

Show Scenarios Table

Scenario Name	Scenario Description	Geom. Layout	Scientific Purpose	Robot Task	Human Behavior	Ideal Outcome
FRONTAL APPROACH	A pedestrian and robot approach head-on.	Passable Space	Pedestrian Interaction	Navigate A to B	Navigate B to A	Robot / humans pass
PEDESTRIAN OBSTRUCTION	A pedestrian blocks the robot's path.	Passable Space	Pedestrian Interaction	Navigate A to B	Block path	Robot stops or reroutes
BLIND CORNER	A robot and human meet at a blind corner.	Corner	Pedestrian Interaction	Navigate A to B	Navigate B to A	No collision / obstruction
PERPENDICULAR CROSSING	Person moves perpendicular to robot.	Intersection	Pedestrian Interaction	Navigate A to B	Navigate B to A	No collision / obstruction
FOLLOWING HUMAN	A robot follows a person.	Walking Space	Joint Navigation	Follow human	Lead robot	Robot follows person
CIRCULAR CROSSING	People and robot move in paths that intersect in the center of a circle	Passable Space	Crowd Navigation	Navigate A to B	Navigate B to A	No collision / smooth flow
OBJECT HANDOVER	A robot hands an object to a human.	Passable Space	Interactive Navigation	Deliver/Receive object	Deliver/Receive object	Human/Robot takes object

In total 17 participants were assigned numbers and divided in 2 groups of 5 and 5 pairs. Each group performed all scenarios but the object handover scenario. All scenarios were recorded for 4 minutes. Most scenarios were performed with both robots, except for the object handover scenario which was only performed with the Toyota HSR. Most scenarios were recorded twice, once with the robot navigating in a socially compliant manner and once in a non-compliant manner. Social behavior adhered to predefined rules (maintaining social distances, avoiding abrupt movements), while non-social behavior involved direct waypoint-to-waypoint navigation, disregarding human presence. A detailed description of all recorded scenes used in NavWareSet is provided in the table below.

Show Scenes Table

N°	Name	Scenario	Robot	Robot Behavior	Group	Participants
1	Scene 1	Frontal Approach	HSR	Social	Group 1	1, 2, 3, 4, 5
2	Scene 2	Pedestrian Obstruction	HSR	Social	Group 1	1, 2, 3, 4, 5
3	Scene 3	Blind Corner	HSR	Social	Group 1	1, 2, 3, 4, 5
4	Scene 4	Following Human	HSR	Social	Group 1	1, 2, 3, 4, 5
5	Scene 5	Perpendicular Crossing	HSR	Social	Group 1	1, 2, 3, 4, 5
6	Scene 6	Circular Crossing	HSR	Social	Group 1	1, 2, 3, 4, 5
7	Scene 8	Frontal Approach	HSR	Non-social	Group 1	1, 2, 3, 4, 5
8	Scene 9	Pedestrian Obstruction	HSR	Non-social	Group 1	1, 2, 3, 4, 5
9	Scene 10	Blind Corner	HSR	Non-social	Group 1	1, 2, 3, 4, 5
10	Scene 12	Perpendicular Crossing	HSR	Non-social	Group 1	1, 2, 3, 4, 5
11	Scene 13	Circular Crossing	HSR	Non-social	Group 1	1, 2, 3, 4, 5
12	Scene 15	Frontal Approach	Jackal	Social	Group 1	1, 2, 3, 5, 6
13	Scene 16	Pedestrian Obstruction	Jackal	Social	Group 1	1, 2, 3, 5, 6
14	Scene 17	Blind Corner	Jackal	Social	Group 1	1, 2, 3, 5, 6
15	Scene 18	Following Human	Jackal	Social	Group 1	1, 2, 3, 5, 6
16	Scene 19	Perpendicular Crossing	Jackal	Social	Group 1	1, 2, 3, 5, 6
17	Scene 20	Circular Crossing	Jackal	Social	Group 1	1, 2, 3, 5, 6
18	Scene 21	Frontal Approach	Jackal	Non-social	Group 1	1, 2, 3, 5, 6
19	Scene 22	Pedestrian Obstruction	Jackal	Non-social	Group 1	1, 2, 3, 5, 6
20	Scene 23	Blind Corner	Jackal	Non-social	Group 1	1, 2, 3, 5, 6
21	Scene 25	Perpendicular Crossing	Jackal	Non-social	Group 1	1, 2, 3, 5, 6
22	Scene 26	Circular Crossing	Jackal	Non-social	Group 1	1, 2, 3, 5, 6
23	Scene 27	Frontal Approach	HSR	Social	Group 2	7, 8, 9, 10, 11
24	Scene 28	Pedestrian Obstruction	HSR	Social	Group 2	7, 8, 9, 10, 11
25	Scene 29	Blind Corner	HSR	Social	Group 2	7, 8, 9, 10, 11
26	Scene 30	Following Human	HSR	Social	Group 2	7, 8, 9, 10, 11
27	Scene 31	Perpendicular Crossing	HSR	Social	Group 2	7, 8, 9, 10, 11
28	Scene 32	Circular Crossing	HSR	Social	Group 2	7, 8, 9, 10, 11
29	Scene 34	Frontal Approach	HSR	Non-social	Group 2	7, 8, 9, 10, 11
30	Scene 35	Pedestrian Obstruction	HSR	Non-social	Group 2	7, 8, 9, 10, 11
31	Scene 36	Blind Corner	HSR	Non-social	Group 2	7, 8, 9, 10, 11
32	Scene 38	Perpendicular Crossing	HSR	Non-social	Group 2	7, 8, 9, 10, 11
33	Scene 39	Circular Crossing	HSR	Non-social	Group 2	7, 8, 9, 10, 11
34	Scene 41	Frontal Approach	Jackal	Social	Group 2	7, 8, 9, 10, 11
35	Scene 42	Pedestrian Obstruction	Jackal	Social	Group 2	7, 8, 9, 10, 11
36	Scene 43	Blind Corner	Jackal	Social	Group 2	7, 8, 9, 10, 11
37	Scene 44	Following Human	Jackal	Social	Group 2	7, 8, 9, 10, 11
38	Scene 45	Perpendicular Crossing	Jackal	Social	Group 2	7, 8, 9, 10, 11
39	Scene 46	Circular Crossing	Jackal	Social	Group 2	7, 8, 9, 10, 12
40	Scene 47	Frontal Approach	Jackal	Non-social	Group 2	7, 8, 9, 10, 11
41	Scene 49	Blind Corner	Jackal	Non-social	Group 2	7, 8, 9, 10, 11
42	Scene 51	Perpendicular Crossing	Jackal	Non-social	Group 2	7, 8, 9, 10, 11
43	Scene 52	Circular Crossing	Jackal	Non-social	Group 2	7, 8, 9, 10, 12
44	Scene 53	Object handover	HSR	Social	Pair 1	2, 8
45	Scene 54	Object handover	HSR	Social	Pair 2	1, 9
46	Scene 55	Object handover	HSR	Social	Pair 3	13, 14
47	Scene 56	Object handover	HSR	Social	Pair 4	4, 15
48	Scene 57	Object handover	HSR	Social	Pair 5	16, 17

Data Pipeline

Experiment Design: The pipeline begins with careful planning of social navigation scenarios in a controlled indoor environment. At this step it is defined how robots and human participants will interact in both socially compliant and non-compliant modes. This phase ensures that a diverse set of realistic interactions is captured.

Data Collection: The robots (HSR and Jackal) and a Ground-Truth Recording Station (GRS) simultaneously collect raw sensory data. The robots log LiDAR, RGB-D, stereo vision, IMU, odometry, and velocity commands. Meanwhile, the stationary GRS records the entire scene from an external vantage point, providing an independent reference for later annotation and validation.

Raw Data Organization: All recorded sensor data is saved in ROS bag files. Each scene’s raw data is packaged into separate files for the robot and the GRS. This organized structure allows researchers to easily replay the scenarios and verify the consistency of recorded sensor streams.

Data Processing: The raw data undergoes processing. Robot poses are extracted using SLAM. Human participant positions are manually annotated frame-by-frame using the CVAT tool applied to the GRS point cloud. This ensures accurate human trajectory labeling, crucial for studying robot social navigation behavior. All CVAT annotations were saved using the Supervisely format.

Data Product Generation: After processing, clean and usable data products are created for each scene:
- Annotated point clouds synchronized with robot poses.
- Robot and participant positions in easily accessible CSV files.
- Occupancy grids detailing static obstacles.
- All metadata and calibration offsets are packaged to ensure reproducibility.
These structured files make the dataset valuable for training, benchmarking, and testing navigation algorithms.

Distribution: The complete, organized dataset — including raw bags, annotations, CSVs, and occupancy grids — is shared publicly. A series of tutorials and helper scripts are provided to assist researchers in using the dataset effectively. The dataset is designed to be user-friendly, with clear documentation on how to access and utilize the data for various research purposes. Researchers can access everything through the NavWareSet website and the NavWare GitHub repository. All data can be downloaded in bulk or individually using the dat@UBFC website.

Data Overview

Each scene is divided into four files, each containing its own data. All files start with the number of its respective scene name.

For generic scene "x" we have the following files:

x_robot.zip

The zipped ROS bag with the raw data from the robot.

The available ROS topics for scenes with the HSR are:

/hsrb/head_rgbd_sensor/rgb/camera_info
/hsrb/head_rgbd_sensor/rgb/image_rect_color
/hsrb/command_velocity
/hsrb/head_l_stereo_camera/image_raw
/hsrb/head_rgbd_sensor/depth_registered/camera_info
/static_obstacle_ros_map
/hsrb/head_l_stereo_camera/camera_info
/hsrb/head_r_stereo_camera/camera_info
/hsrb/head_rgbd_sensor/depth_registered/rectified_points
/tf
/hsrb/head_r_stereo_camera/image_raw
/hsrb/base_scan
/hsrb/head_rgbd_sensor/depth_registered/image_rect_raw
/tf_static
/dynamic_obstacle_map

The available ROS topics for scenes with the Jackal are:

/amcl_pose
/camera/color/image_raw
/map
/tf
/rslidar_points
/camera/color/camera_info
/cmd_vel

x_grs.zip

The zipped ROS bag with the raw data from the Ground-Truth Recording Station (GRS).

The available ROS topics are:

/camera/color/image_raw
/rslidar_points
/camera/color/camera_info

x_annotated.zip

The zipped folder containing annotated filtered point cloud, synchronized with the robot’s poses for scene x, organized following the structure below:


                x_annotated/

                ├── scene_x/

                │   ├── pointcloud/

                │   │   ├── 1717330243.123456789.pcd

                │   │   ├── 1717330244.123456789.pcd

                │   │   └── ...

                │   ├── ann/

                │   │   ├── 1717330243.123456789.json

                │   │   ├── 1717330244.123456789.json

                │   │   └── ...

                ├── x_robot_pose.csv

                ├── x_grs_to_bot_offset.json

                ├── x_occupancy_xy_points.json

                ├── meta.json

                └── key_id_map.json

x_poses.zip

The zipped folder containing participant and robot positions with the occupancy grid of the obstacles in scene x, organized following the structure below:


                x_poses/

                ├── x_occupancy_xy_points.json

                └── meta.json

The CSV file with the robot poses and the position of the participants follows the pattern below:

timestamp	robot_x	robot_y	robot_yaw_rad	x1	y1	x2	y2	x3	y3	x4	y4	x5	y5
1730220611292458240	2.7051	-0.8938	-0.0046	1.4400	0.5500	0.6800	-1.1600	10.6500	0.1800	11.0900	-1.5600	10.9200	-2.6500
1730220611392471296	2.7051	-0.8938	-0.0046	1.4300	0.5500	0.6800	-1.1600	10.6500	0.1800	11.0900	-1.5600	10.9200	-2.6500
1730220611492328448	2.7051	-0.8938	-0.0046	1.4300	0.5500	0.6800	-1.1600	10.6500	0.1800	11.0900	-1.5600	10.9200	-2.6500

Scenes 53 to 57 don't have person position annotation because the participants don't walk around the environment; only the robot moves between the participants.

Authors

Johnata Brayan^1,2, Sihao Deng², Armando Alves Neto¹, Iaroslav Okunevich², Tomas Krajnik³, Francois Bremond⁴, Zhi Yan^2,5

¹Universidade Federal de Minas Gerais
²Université de technologie de Belfort Montbéliard
³Czech Technical University in Prague
⁴Inria
⁵ENSTA - Institut Polytechnique de Paris

Funding

This project has received funding from the French National Research Agency (ANR) under grant agreement No ANR-23-CE10-0016.

NavWareSet: A Dataset of Socially Compliant and Non-Compliant Robot Navigation

Top-left: Lidar point cloud of the robot's environment;
Top-right: Robot's own sensory data;
Bottom-left: Synced point cloud with pedestrian's positions and robot's pose;
Bottom-right: Extracted pedestrian and robot trajectories with environment map.

Abstract

Overview

Experimental Setup

Data Pipeline

Data Overview

BibTeX

Authors

Funding

NavWareSet: A Dataset of Socially Compliant and Non-Compliant Robot Navigation

Top-left: Lidar point cloud of the robot's environment; Top-right: Robot's own sensory data; Bottom-left: Synced point cloud with pedestrian's positions and robot's pose; Bottom-right: Extracted pedestrian and robot trajectories with environment map.

Abstract

Overview

Experimental Setup

Data Pipeline

Data Overview

BibTeX

Authors

Funding

Top-left: Lidar point cloud of the robot's environment;
Top-right: Robot's own sensory data;
Bottom-left: Synced point cloud with pedestrian's positions and robot's pose;
Bottom-right: Extracted pedestrian and robot trajectories with environment map.