F1 79.96 on ontonotes 5 with your pretrained spanbert_large

[houliangxue]

Is F1 79.6 the highest score you've ever got on Ontonotes? Why I got F1 79.95 without anything changes for your experiments.conf. Did you set a seed in your experiment?

[preethiseshadri518]

@houliangxue I have a question for you. I am interested in evaluating performance on OntoNotes using the trained BERT-base coreference model. Did you use evaluate.py? If so, what should data_dir vs. <ontonotes/path/ontonotes-release-5.0> path be, and how are they related? My assumption is data_dir can be anything (I have set it to data_dir='.') and ontonotes path is the full path to where I have ontonotes-release-5.0.

Currently, I am getting that evaluate is evaluating on 0 examples, which is not what we want, and I assume it's because I haven't specified the paths correctly. Any help you can provide would be greatly appreciated. Thank you!