Eurobench Software documentation This document gathers documentation on software aspects related to the Eurobench project. The Eurobench website containing all protocols is accessible here. How to Define a new protocol Search a protocol and conduct an experiment Handle incomplete or unclear protocol Define a new protocol A protocol definition should contain: general information of the protocol used to fill the Eurobench database and to configure the automation tools template provided and commented in Eurobench protocol templates. extended documentation to prepare the testbed and / or to conduct the experimentation that information would be uploaded on the Eurobench website, and made available at the protocol webpage a simple manual template is proposed here Algorithms implementing the performance indicators, following the Eurobench policy more information can be found in Performance Indicator Algorithm specification. example of algorithms implemented can be found on the Eurobench GitHub repository. For now, there is no methodology to upload automatically a protocol. Needed information is to be transmitted to the Eurobench team. Search a protocol and conduct an experiment 1. Search for an adequate protocol The Eurobench software website protocol section enables accessing to all registered protocols. The filtering mechanism provide a key-words based search of protocol. 2. Read the protocol documentation Once selected, the protocol description should provide a general overview of the purpose, objectives, metrics and execution methodology of the protocol. When required, additional documentation is provided on the left area. From this information, the reader should be able to prepare and execute experimentations. 3. Do the experimentation and prepare experimental data for upload The documentation should provide indications, if needed, on how to adjust the data collected during an experimentation. We are encouraging the use of standard data format. It is described in the Eurobench Data Format. If not directly provided by the testbed, the protocol documentation should explain you how to convert the Raw data into such format. Also, as the metric computation is handled automatically through the Eurobench website, we also provide some guidelines on how to name the collected datafiles. See Organisation of experimental dataset. But, once more, this is the protocol description that should guide you in that process. 4. Upload the experimental data and benchmark Through the website, a user can log an experiment conducted. Given the protocol used, and after datafile upload is able to compute the related metrics automatically, and make them available to the experimenter. Comparison with other similar experiments will be provided soon. Handle incomplete or unclear protocol The integration status of the different protocols and related algorithms can be checked in the following GitHub repository. We also created a service desk on the following address. You can add there issues, or even send remarks or complains to the following email. The Eurobench website is a tool under development, and actively updated. If any description is unclear or missing, please let us know so that we can work on improving it. Thx! Site content Human model Eurobench Data Format Eurobench protocol templates Performance Indicator Algorithm specification Organisation of experimental dataset Modification Instructions All the documentation is subject to evolution according to Eurobench needs. If you identify errors, or want to provide additional material, please use traditional GitHub tools: The source code of this website is located under GitHub, in the repository eurobench_documentation Create an issue through the Git Hub Interface please indicate the specific section you are referring to. Create an issue from the code file through the Git Hub interface Send an email to us (anthony.remazeilles@tecnalia.com Leader of WP6: Software Development). Supported by Eurobench - the European robotic platform for bipedal locomotion benchmarking. More information: eurobench2020.eu. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement no. No 779963.