IndicXTREME

To evaluate language models on Indic languages, we need a robust human-annotated NLU benchmark consisting of 9 tasks across 18 Indic languages.

Coming Soon

Dataset Descriptions and Examples

Contributors

• Sumanth Doddapaneni (AI4Bharat, IITM)
• Rahul Aralikatte (McGill, MILA)
• Gowtham Ramesh, (AI4Bharat, IITM)
• Shreya Goyal, (AI4Bharat)
• Mitesh Khapra,  (AI4Bharat, IITM)
• Anoop Kunchukuttan, (Microsoft, AI4Bharat, IITM)
• Pratyush Kumar,  (Microsoft, AI4Bharat, IITM)

Corresponding authors: Sumanth Doddapaneni

Citing

If you are using any of the resources, please cite the following article:

@misc{https://doi.org/10.48550/arxiv.2212.05409,
  doi = {10.48550/ARXIV.2212.05409},
  url = {https://arxiv.org/abs/2212.05409},
  author = {Doddapaneni, Sumanth and Aralikatte, Rahul and Ramesh, Gowtham and Goyal, Shreya and Khapra, Mitesh M. and Kunchukuttan, Anoop and Kumar, Pratyush},
  keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
  title = {IndicXTREME: A Multi-Task Benchmark For Evaluating Indic Languages},
  publisher = {arXiv},
  year = {2022}, 
  copyright = {arXiv.org perpetual, non-exclusive license}
}

License

IndicXTREME is released under this licensing scheme:

• We do not own any of the text from which this data has been extracted.
• We license the actual packaging of this data under the Creative Commons CC0 license (“no rights reserved”).
• To the extent possible under law, AI4Bharat has waived all copyright and related or neighboring rights to IndicXTREME.
• This work is published from: India.