Eurobench Software documentation

This document gathers documentation on software aspects related to the Eurobench project.

The Eurobench website containing all protocols is accessible here.

How to

Define a new protocol

A protocol definition should contain:

  • general information of the protocol

    • used to fill the Eurobench database and to configure the automation tools

    • template provided and commented in Eurobench protocol templates.

  • extended documentation to prepare the testbed and / or to conduct the experimentation

    • that information would be uploaded on the Eurobench website, and made available at the protocol webpage

    • a simple manual template is proposed here

  • Algorithms implementing the performance indicators, following the Eurobench policy

For now, there is no methodology to upload automatically a protocol. Needed information is to be transmitted to the Eurobench team.

Search a protocol and conduct an experiment

1. Search for an adequate protocol

The Eurobench software website protocol section enables accessing to all registered protocols. The filtering mechanism provide a key-words based search of protocol.

2. Read the protocol documentation

Once selected, the protocol description should provide a general overview of the purpose, objectives, metrics and execution methodology of the protocol. When required, additional documentation is provided on the left area.

From this information, the reader should be able to prepare and execute experimentations.

3. Do the experimentation and prepare experimental data for upload

The documentation should provide indications, if needed, on how to adjust the data collected during an experimentation. We are encouraging the use of standard data format. It is described in the Eurobench Data Format. If not directly provided by the testbed, the protocol documentation should explain you how to convert the Raw data into such format.

Also, as the metric computation is handled automatically through the Eurobench website, we also provide some guidelines on how to name the collected datafiles. See Organisation of experimental dataset. But, once more, this is the protocol description that should guide you in that process.

4. Upload the experimental data and benchmark

Through the website, a user can log an experiment conducted. Given the protocol used, and after datafile upload is able to compute the related metrics automatically, and make them available to the experimenter. Comparison with other similar experiments will be provided soon.

Handle incomplete or unclear protocol

The integration status of the different protocols and related algorithms can be checked in the following GitHub repository. We also created a service desk on the following address. You can add there issues, or even send remarks or complains to the following email.

The Eurobench website is a tool under development, and actively updated. If any description is unclear or missing, please let us know so that we can work on improving it. Thx!

Modification Instructions

All the documentation is subject to evolution according to Eurobench needs. If you identify errors, or want to provide additional material, please use traditional GitHub tools:

eurobench logo

Supported by Eurobench - the European robotic platform for bipedal locomotion benchmarking. More information:

eu flag

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement no. No 779963.