Call for Participation: 10th Competition on Legal Information Extraction/Entailment (COLIEE) 2023

in association with the International Conference on Artificial Intelligence and Law (ICAIL) 2023
June 19, 2023, Braga, Portugal

As an associated event of ICAIL 2023, we are happy to announce the 10th Competition on Legal Information Extraction and Entailment (COLIEE-2023), which will include tasks on both statute law and case law.

Four tasks are included in the 2023 edition: Tasks 1 and 2 are about case law, and tasks 3 and 4 are on statute law. Task 1 is a legal case retrieval task, and it involves reading a new case Q, and extracting supporting cases S1, S2…, Sn from the provided case law corpus, to support the decision for Q. Task 2 is the legal case entailment task, which involves the identification of one or more paragraphs from an existing case that entail the decision of a new case. As in previous COLIEE competitions, Task 3 is to consider a yes/no legal question Q and retrieve relevant statutes from a database of Japanese civil code statutes; Task 4 is to confirm entailment of a yes/no answer from the retrieved civil code statutes.

Further details can be found in the following URL:

The datasets for Task 1 and Task 2 are in English, and the datasets for Task 3 and Task 4 are available both in Japanese and English (translation from Japanese bar exams and civil code). You can choose any (or both) language datasets and any task for participation.

The intention is to build a community of practice regarding legal information processing and textual entailment, so that the adoption and adaptation of general methods from a variety of fields is considered, and that participants share their approaches, problems, and results.

Participants should clearly mention what dataset was used (for example, pretrained by Wikipedia dump data as of 2022xxxx, fine-tuned by …) for reproducibility purposes. Participants can use any external data, but it is assumed that they do not use the test dataset and/or something which could directly contain the correct answers of the test dataset, because this task is intended to be a pure textual entailment task.

Please submit a paper at:
Papers should not exceed 10 pages (inclusive of references) and should conform to the standards set out at the ICAIL 2023 webpage, section "Paper Submission" except copyright description (you just delete copyright description or you claim copyright of your paper by yourself).

Participants are required to submit a paper on their method and experimental results. At least one of the authors of an accepted paper has to present the paper at the COLIEE workshop of ICAIL 2023.

As post-proceedings, we plan to publish selected papers by another round reviews at a special issue at "The Review of Socionetwork Strategies"
as the same treatment as COLIEE 2021 selected papers:
Important dates:
15 Mar, 2023 Submission deadline of competition test runs for task 3
22 Mar, 2023 Return of competition test run rankings/assessments for task 3 and announcements of answers (relevant article(s) for each question) for task 4
29 Mar, 2023 Submission deadline of competition test runs for task 1 and 2
29 Mar, 2023 Submission deadline of competition test runs for task 4 using the above answers for the same test of bar exam questions
03 Apr, 2023 Announcements of rankings/ assessments for tasks 1, 2 and 4
12 Apr, 2023 Paper submission deadline for the COLIEE workshop
26 Apr, 2023 Notification for the COLIEE workshop paper
10 May, 2023 Camera-ready copy deadline
19 Jun, 2023 ICAIL 2023
Application Details:
Potential participants to COLLIE-2023 should respond to this call for participation by submitting an application. To apply, submit the application and memorandums of the following URL to rabelo(at) .


Memorandum for Tasks 1 and/or 2 (Case law competition)

Memorandum for Tasks 3 and/or 4 (Statute law competition, English Data)

Memorandum for Tasks 3 and/or 4 (Statute law competition, Japanese Data)

An acknowledgement note will be sent to the email address supplied in the form once the form has been processed.

Task coordinators:
Mi-Young Kim, University of Alberta, Canada
Randy Goebel, University of Alberta, Canada
Juliano Rabelo, University of Alberta, Canada
Ken Satoh, National Institute of Informatics, Japan
Yoshinobu Kano, Shizuoka University, Japan
Masaharu Yoshioka, Hokkaido University, Japan

Questions and further information:

Get in touch with us on Linkedin

Join the IAAIL group on Linkedin! Your posts will be shared on the IAAIL website

Contact us

International Association for AI and Law

Send us an email