Testing Benchmarks

Testing Benchmarks


Our lab has identified benchmark problems that are helpful to compare motion planning algorithms. Each contains at least one of the challenges discussed in prior chapters (narrow passage, high dimensionality,...). Motion planning benchmarks can be downloaded from the parasollab website at this link.

In this assignment, you will select a benchmark from the list on the website and solve its query using PPL.

  1. Experimental Set up

    Go to the motion planning benchmarks page and choose a problem to test. When you click on the benchmark image, you will see details about the problem and a visualization of its motion planning path. Be sure to choose a problem that you find interesting and feel free to consult your mentor about this.

    Open up a terminal and change directory into the Examples directory of PPL: cd ~/PPL/pmpl/src/Examples

    Clone the Benchmarks repository from GitLab: git clone git@gitlab.engr.illinois.edu:parasol-group/parasol/open-ppl-env.git

    This repository contains the environment and query files for each of the motion planning benchmarks. Find the one you've chosen and use vizmo to view the vizmo .env and .query file. Use the edit query tool in vizmo to check the flexibility of the robot.

  2. Evaluation

    Like you did in lesson 1, you are going to run pmpl using input from the CfgExamples XML file. To do that, you will need to edit the file to update the testing environment. You can do so by opening CfgExamples.xml and replacing the input in the following nodes to match the properties of the chosen benchmark:

    • Problem: Rename the file name to example_benchmark
    • Environment: Update the file name (e.g. ./open-ppl-env/BenchmarkName/benchmarkName.env)
    • Robot: Update the robot file name (e.g. ./open-ppl-env/BenchmarkName/benchmarkName.robot)
    • CSpaceConstraint: Update the point with the right number of degrees of freedom. Hint: Feel free to use values in the .query file for your start and goal configurations.The first digit (usually a 0) is a robot ID and should not be copied.

  3. Modifying XML File for Different Types of Benchmarks

    As you can see in the CfgExamples.xml, it contains various options and settings that will be applied to the algorithms you would like to run. For example, you can set the types of the environment by changing the Environment tag and change the validity checker of the sampler by changing the vcLabel label inside the sampler tag (e.g., UniformRandomSampler, etc. Some of the tags are commented out. You can uncomment them if you want to apply those tags.).
    Which settings (i.e., validity checker or distance metric) you are using affects the performance of the algorithm. Thus, it is important to think about what settings you should use before running the algorithm.
    Now we will explore other benchmarks which need special settings. You can find these variants in here.

    • Collision Detection for Linkages
      Think about the Periscope benchmark. Each side of the Periscope is connected to each other with revolute joints. The angle of each revolute joint defines the state of the Periscope. However, even if each angle of the revolute joints is in the valid angle range (i.e., [-2pi, 2pi]), the resulting state of the Periscope could not be a valid state. This is because each side cannot penetrate the other (See the example in Fig.1.).
      For this reason, you may want to add additional features to the validity checker and collision detector to consider the collision between one link and the adjacent links. This can be done by adding flags to the validity checker and collision detector tags.
      (Hint: You may find "ignoreAdjacentLinks" flag useful. Add this flag to the validity checker and the collision detector you are using.)
      example
      Fig.1 - An example of valid and invalid configuration for the linkages.
    • Multi-Robot Motion Planning
      In multi-robot motion planning, you can let multiple robots do their own planning simultaneously. This requires larger configuration space as the number of robot gets larger. PRM can solve this planning task, and it is defined as "GroupPRM" under the "MPStrategies" tags. The GroupPRM requires "CompositeQuery" as the map evaluation method which is defnied as "GroupQuery" under the "MapEvalulators" tag. Also, GroupPRM uses "GroupTask" under the "GroupTasks" tag, and it defines the robot group with group query points as follows.

      <RobotGroup label="2-robotGroup" robotLabels="boxy1 boxy2">
      <GroupTask label="2-groupQuery" group="2-robotGroup">
      <Task label="query1" robot="boxy1">
      ...
      </Task>
      <Task label="query2" robot="boxy1">
      ...
      </Task>
      </GroupTask>


      To use this multi-robot task, make sure that you comment out all the tasks other than the multi-robot task in order to specify the task you want. If you want to run this strategy, set the solver in the "Solver" tag as the label defined in the "GroupPRM" tag (i.e., "CompositePRM").
    • Multi-Query Motion Planning
      In multi-query motion planning, you can let the robot visit sequential points. You can let the robot know those points and assign the strategies to make the robot get each query point. This can be done by first building the roadmap and then querying multiple query points. You can find the "build", "query1", "query2", and "query" in the xml file as follows.

      <Task label="build" ...> </Task>
      <Task label="query1" ... > ... </Task>
      <Task label="query2" ... > ... </Task>
      < Task label="query3" ... > ... </Task>


      To use this sequential task, make sure you comment out all the tasks other than the sequential query task to specify the task you want. The "StrategySequence" tag under the "MPStrategy" tag refer to these tags to set the query points. If you want to run this strategy, set the solver in the "Solver" tag as the label defined in the "StrategySequence" tag (i.e., "Sequence").