TempQuestions
TempQuestions[1] is a new benchmark containing 1,271 questions, that are all temporal in nature, paired with their answers. Most questions require decomposing
them into sub-questions, and the questions are of a kind that they would be best evaluated on a combination of structured data and unstructured text sources. This collection is derived by
judiciously selecting time-related questions from the Free917, WebQuestions and ComplexQuestions sets, with additional curation and tagging of temporal cues.
This dataset can be downloaded via the link.
Leaderboard: Explicit
| Model / System |
Year |
Precision |
Recall |
F-Score |
Reported by |
| AQQU |
2018 |
27.6 |
60.7 |
31.1 |
Jia et. al. |
| QUINT |
2018 |
29.3 |
60.9 |
32.6 |
Jia et. al. |
Leaderboard: Implicit
| Model / System |
Year |
Precision |
Recall |
F-Score |
Reported by |
| AQQU |
2018 |
12.9 |
34.9 |
14.5 |
Jia et. al. |
| QUINT |
2018 |
25.6 |
54.4 |
27.0 |
Jia et. al. |
Leaderboard: Temporal Answer
| Model / System |
Year |
Precision |
Recall |
F-Score |
Reported by |
| AQQU |
2018 |
26.1 |
33.5 |
27.4 |
Jia et. al. |
| QUINT |
2018 |
25.2 |
38.2 |
27.3 |
Jia et. al. |
Leaderboard: Ordinal
| Model / System |
Year |
Precision@1 |
MRR |
Hits@5 |
Reported by |
| AQQU |
2018 |
28.4 |
57.4 |
32.7 |
Jia et. al. |
| QUINT |
2018 |
21.3 |
54.9 |
26.1 |
Jia et. al. |
References
[1] Jia, Zhen, Abdalghani Abujabal, Rishiraj Saha Roy, Jannik Strötgen, and Gerhard Weikum. Tempquestions: A benchmark for temporal question answering. In Companion Proceedings of the The Web Conference 2018, pp. 1057-1062. 2018.
Go back to the README