|
Comparison of Self-Localization Methods Continued
=================================================
This archive contains data logs and evaluation tools, Dieter Fox and me used
for performing the self-locaization experiments reported in our paper at
IROS 2002.
Data Log Files
--------------
The archive contains one directory 'dlogs' with the following data log files:
o normal/dlog.dat (base log file),
o sparse.*/dlog.dat (log files under different level of sparseness),
o noise.*/dlog.dat (log files under different level of outliers), and
o displace/dlog.dat (log file with parts removed to simulate kid-napping).
These log files were generated from the same base log file. See our paper
[Gutmann and Fox, IROS 2002] for understanding what has been done to the
indiviual logs.
The format of each dlog.dat file is as follows. Each line contains a position
estimate from odometry and observations
obs: frame x y theta num ( id1:id2 range bearing ) ...
where:
- frame is a timestamp (1 sec = 125 frames),
- x, y, theta is the robot pose estimated by odometry and measured
in mm and deg,
- num is the number of landmarks seen,
- id1:id2 gives the id of the landmark seen,
- range and bearing are from the current robot pose (kinematic chain of head
is already included) and are measured in mm and deg.
The reference point on the robot is at the neck joint connecting robot body
and head.
You also need the following information about the landmark positions:
cyan:magenta -1500 -1000
magenta:cyan -1500 1000
magenta:green 0 -1000
green:magenta 0 1000
yellow:magenta 1500 -1000
magenta:yellow 1500 1000
and
0 -> green
1 -> magenta
2 -> yellow
3 -> blue
The robot started in the center of the field facing in positive x direction.
It then cycled in an 8-shaped path through the following positions:
(500 -500), (500, 0), (0, 0), (-1000, 0), (-1000, -500). At each of these
positions the operator pressed a button on which a 'mark' was written to the
data log. Potentially there are errors in the ground truth (joysticking the
robot exactly on a spot is difficult, observing it is exactly on a spot
contains errors, and there can be a short time delay until the mark is written
to the log).
See fieldSetup.gif for a visualization of landmark and marker positions.
Evaluation
----------
Basically, you can completely decide by your own how to evaluate your results.
Here is how we did it. If your localization program outputs the pose of the
robot at each mark in the following format:
< method-name > x y th
where x, y are in mm and the in deg, then you can use the accuracy.sh script
for computing the distance to the ground truth locations and the
mean_confidence program for obtaining mean and confidence of your estimates.
E.g. for MLEKF we used:
accuracy.sh MLEKF < pose.log | mean_confidence
Evaluation for kid-napping is a bit different. We used the output of one
localization method as a reference path when processing the base log file
(e.g. we used the SRL output but you are welcome to provide your own if you
feel your results on the base log file are better). Your localization
program should then output the robot pose at each time step in the
following format:
x[< frame >]=< x > y[< frame >]=< y > th[< frame >]=< th >
for example:
x[30048]=-102.502917; y[30048]=-4.598352; th[30048]=-105.912977;
Your localization program should als copy the 'mark' lines to this output.
You can then use the 'recoverTime' utility for computing the number of seconds
your method needs for recovering from kid-napping:
recoverTime ../reference-log/pose.log.SRL < allpose.log | mean_confidence
You find our reference pose log, scripts and utils in the evaluation folder.
Papers
------
J.-S. Gutmann, W. Burgard, D. Fox, and K. Konolige. An Experimental Comparison of Localization Methods, International Conference on Intelligent Robots and Systems (IROS'98), Victoria, Canada, October 1998.
J.-S. Gutmann and D. Fox, An Experimental Comparison of Localization Methods Continued, in: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'02), Lausanne, Switzerland, October 2002.
S. Kristensen and P. Jensfelt. An Experimental Comparison of Localisation Methods, the MHL Sessions, in: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'03), 2003.
Good luck!
Steffen Gutmann, 6.5.2004
|
