Experiment: Varying the sampling technique for PRMs

Experiment: Varying the sampling technique for PRMs

This experiment is intended to give you further experience with running the pmpl code. In particular, it will allow you to learn how to adjust the input parameters by modifying XML files. It will also allow you to learn about pmpl's .stat files, how to run experiments, and how to collect data for analysis. This assignment will also introduce you to gnuplot, a Linux tool for creating plots and graphs.

You will be using a 3D environment for their experiments. Specifically Examples/3D. Make sure that pmpl is compiled and ready to run. Use the following steps to collect data on various sampling methods within the code base. The objective of the experiment, beyond learning the tools and running pmpl, is to critically analyze the effectiveness/cost trade-off of the following sampling methods: uniform random sampling, Gaussian sampling, bridge-test sampling, obstacles-based sampling, and medial-axis based sampling.

  1. Running Experiments
    1. We need a good test file to alter and change. So firstly, we should copy Examples/CfgExamples.xml to Examples/MyCfgExamples.xml, thus we can change it as much as we want. Type cd Examples and then cp CfgExamples.xml MyCfgExamples.xml.
    2. Open MyCfgExamples.xml in vim.
    3. Find the section for MPStrategies. Find BasicPRM with the label PRM-build.
    4. This is the strategy resembling the probabilistic roadmap (PRM) method. You can see how we select which samplers to use, such as Gauss or OBPRM (which are all commented out for now), connection strategies, etc.
    5. Comment out all Connector nodes.
    6. Make sure that the Evaluator is set to 'NodesEval'.
    7. Find the ConditionalEvaluator for NodesEval by searching: hit esc to be sure you are not in insert mode, then type /ConditionalEvaluator label="NodesEval". Vim should take you to a line that looks like this:
      <ConditionalEvaluator label="NodesEval" metric_method="NumNodes" value="1000" operator=">="/>
      Change value from 1000 to 1. This tells PPL to make sure at least one node exists before terminating. You can type :noh to turn off the search highlighting afterward.
    8. Now return to the section for PRM-build. Change the number on UniformRandomFree from 1 to 100. This implies that when we attempt sampling we try to generate 100 samples with 1 attempt per sample; so we could generate anywhere from 1 to 100 samples. (The lower bound is 1 because of the code not our experiment. Bonus: Look into the code to find out where this is. Add this to your report if you can!)
    9. We are ready to generate a map and stat combination file. Quit your editor. Type ../pmpl -f MyCfgExamples.xml. This generated PRM-build.12345678.map and PRM-build.12345678.stat. Because we actually just did uniform sampling, lets change the names of those files. Type mv PRM-build.12345678.query0.map Uniform.map and then type mv PRM-build.12345678.query0.stat Uniform.stat.
    10. Now lets create the other stat and map files accordingly. Open the XML file, change the Sampler label to Gaussian sampling (Quiz: Do you know what the proper text to type here is?). Repeat running and saving the map and stat files. Repeat for OBPRM, MAPRM, and Bridge test sampling (change the d value from 0.2 to 0.4, this causes a significant difference in results).

      11. Note: Do not simply uncomment the other samplers. Change the label from UniformRandomFree to Gauss (or whatever the method might be) otherwise the number and attempts will be messed up, or more than one sampler will be generating samples at a time.

    You have now run the experiments. You should have five sets of map and stat files. One each for uniform random sampling, Gaussian sampling, Obstacle-based sampling, Medial-axis sampling, and bridge-test sampling. Now let us collect and analyze the data.

  2. Collecting Data

    We would like you to collect the following data and generate the following plots. In order to answer the following questions:

    • How does the efficiency of the samplers differ?
    • How do the samplers differ in the samples they generate?

    The first is a quantitative question asking us to compare the number of successful nodes generated (number of nodes in the map files) and how costly they were to attempt (number of collision detection (CD) calls made) and the second is a qualitative question.

    1. Open an example stat file.
    2. Find Number of Nodes. This will be the number of successful nodes added to the roadmap.
    3. Find All Collision Detectors = [some number]. This number is the amount of CD calls needed for 100 attempts.
    4. Collect these various data elements within two different files nodes.dat and cd.dat with the following format.

      Sampler [metric]
      Uniform [number]
      Gaussian [number]
      OBPRM [number]
      MAPRM [number]
      Bridge-test [number]

    5. Download this GNUPlot script. Copy it into the folder with your data files. (cp [from destination] [to destination])
    6. Alter the following lines to make your images.
      • yrange - Change the yrange for the number of nodes range to [0:100]. Change the yrange for the collision detection (cd) image (second yrange setting) to an appropriate range to fit your data.
      • ytics - Change the ytics to 10 for the number of nodes image. Change the second one accordingly based on the range of your data. This is for setting how often a tic on the y-axis shows up, i.e. if the number is 10 the numbers 10, 20, 30, etc. show up.
    7. Make the images by typing gnuplot sampler.gnu. View the images typing gio open [image]. When the images look good and you are happy with the ranges selected you can move on.
    8. View and compare the roadmaps made in Vizmo as well.