Registration overview

All evaluation activities will be conducted using a NIST maintained web platform shared with OpenSAT 2020. To register, each participant will need to create an account on this web platform, which will allow them to perform various activities such as registering for the evaluation, signing the data license agreement, and uploading submissions.

After registering, agreeing to the terms and conditions, and signing (and having approved) the LDC data license agreement, participants will be able to participate in the DIHARD III evaluation. This page contains step-by-step instructions for creating the evaluation account, joining a site and team, selecting tasks, and signing the relevant agreements.

Setting up an evaluation account (Step 1)

To sign up for an evaluation account, navigate to the DIHARD III page on the OpenSAT site and follow the steps below:

  • Click the “Sign Up” button at the top of the page.
  • This will bring up a form titled “OpenSAT Series Sign up”. At this stage, provide a valid email address and create a password, then click “Sign up”. A confirmation email will then be sent to the provided email address.
  • Click the “Confirm my account” link in the confirmation email. This will take you to a login page, which will have the email address and password fields prepopulated using the values you entered.
  • Click “Log in”. This will take you to the evaluation dashboard, which will display the registration steps.
  • Please complete the steps in this dashboard to complete account creation and registration. Instructions for individual steps follow.

Creating/joining a site (Steps 2 and 3)

After the evaluation account is confirmed, the participant will be asked to join a site (or create one if it does not exist). The participant is also asked to associate their site to a team or to create a team if one does not exist. This allows multiple members to perform activities on behalf of their site and/or team (e.g., make a submission). Clarifying the distinction between participants, sites, and teams:

  • site – a single organization (e.g., NIST)
  • team – a group of organizations collaborating on a task (e.g., Team1 consisting of NIST and LDC)
  • participant – a member or representative of a site who takes part in the evaluation (e.g., John Doe)

Creating/joining a site (Step 2)

To create or join a site, from the “Registration Workflow” panel click “Create/Join a site”. This will bring up a list of all sites that you are a member of. To join an existing site, click “Join a Site”, then:

  • select a site from the “Choose a Site” dropdown menu
  • enter the site password (you will need to obtain this from the site owner)
  • click “Join this Site”

To create a new site, click “Create New Site”, then:

  • enter a site name
  • enter a password for the site
  • click “Create and join Site”

Creating/joining a team (Step 3)

To create or join a team, from the “Registration Workflow” panel click “Create/Join a team”. This will bring up a list of all teams that you are a member of. To join an existing team, click “Join a Team”, then:

  • select the site that is joining the team from the “Choose one of the site(s) you own” dropdown menu
  • choose the team you wish to join from the “Choose a Team” dropdown menu
  • enter the team password (you will need to obtain this from the team owner)
  • click “Join this team”

To create a new team, click “Create New Team”, then

  • enter a team name
  • choose one of the teams that you own to be registered with to this team from the “Choose one of the site(s) you own” dropdown menu
  • enter a password for the team
  • select whether or not you want the team name to be anonymized on the leaderboard during the challenge (you can change this later)
  • click “Create Team”



Selecting tasks (Step 4)



To register for DIHARD III tracks, in the “Registration Workflow” panel,
click “Select tasks”. This will bring up a list of available tasks in the
system, including tasks from both DIHARD III and OpenSAT. These tasks are
displayed as a list of TEAM/TASK pairs, each with a checkbox to it’s left;
e.g., if your team name is TEAM:



    
  
  • TEAM - OpenASR20 (development, evaluation)
    • 
  
  • TEAM - DIHARD3_Track1 (evaluation)
    • 
  
  • TEAM - DIHARD3_Track2 (evaluation)
    • 




To register for Track 1 (mandatory) check the box to the left of
“DIHARD3_Track1”. Similarly, to register for Track 2 (optional), check the
box to left of “DIHARD3_Track2”. When you are finished, click the 
“Register” button.



Terms and Conditions and LDC Data License Agreement (Step 5)



In order to complete your registration and gain access to the data, you will
need to accept the DIHARD III terms and conditions and sign and upload the
LDC data license agreement. To complete these tasks, from the
“Registration Workflow” panel, click “Sign and upload license”, which
will take you to a listing of all available agreements and their current status
(i.e., signed/not signed, approved/not approved).



To agree to the DIHARD III terms and conditions locate the entry named
“DIHARD III terms and conditions” and click “Show”. This will bring up
the text of the terms and conditions. To agree to these terms, click the
“Review” button located in the bottom left hand corner, check the box to
the left of “I acknowledge that I have read and accepted…“, then click
“Update License Agreement.”



The creator of each SITE must sign and upload the LDC data license
agreement. Participants will be unable to download data until LDC has approved
the signed LDC data license agreement. To do so, locate the entry named
“LDC Data License Agreement for…“ and click “Show.” On the next screen,
click on “Download License Agreement” to download a PDF copy. After you
have reviewed the license agreement, sign it and upload the signed copy as a
PDF by clicking on the “Upload Signed License Agreement” button. Someone
from LDC will review your upload and notify you by email when the signed
license agreement has been approved.