Codalab Submission Instruction



Registration


We use CodaLab to host our competition, please register in our CodaLab evaluation portal (updated Aug 23, 2021). we suggest you to register early in case any delay in approval.

Submission


Our CodaLab evaluation portal (updated Aug 23, 2021) accepts submissions for any subset of the tasks. However, to participate our VALUE leaderboard, you will need to submit predictions for all the tasks. Alternatively, if you want to participate the leaderboard for a particular task type, e.g., captioning, you will need to submit predictions for all the captioning tasks. Submissions are required to following specific formats as we showcase in VALUE submission samples. Before submission, you can also use our Evaluation code to check whether you have the correct format.



Contact



Have any questions or suggestions? Feel free to reach us at value-benchmark@googlegroups.com!
For faster processing time, please add [VALUE Challenge 2021] to the email title.