Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 22 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
# DocILE: Document Information Localization and Extraction Benchmark
[![Tests](https://github.com/rossumai/docile/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/rossumai/docile/actions/workflows/tests.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

<p align="center">
<a href="https://github.com/rossumai/docile/actions/workflows/tests.yml"><img src="https://github.com/rossumai/docile/actions/workflows/tests.yml/badge.svg?branch=main" alt="Tests"></a>
<a href="https://pypi.org/project/docile-benchmark/"><img src="https://img.shields.io/pypi/v/docile-benchmark" alt="PyPI version"></a>
<a href="https://pypi.org/project/docile-benchmark/"><img src="https://img.shields.io/pypi/pyversions/docile-benchmark" alt="Supported Python versions"></a>
<a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License: MIT"></a>
</p>

Repository to work with the [DocILE dataset and benchmark](https://docile.rossum.ai/), used in the DocILE'23 CLEF Lab and ICDAR Competition.
The competition deadline is on ~~May 10~~ **May 24, 2023** and comes with a **$9000 prize pool**.
Expand Down Expand Up @@ -38,10 +43,14 @@ You can also work with the zipped datasets when you turn off image caching (chec

## Installation

### Option 1: Install as a library
Choose one of the two options below: install `docile` as a library, or use docker.

<details>
<summary><b>Option 1: Install as a library</b></summary>

Install the library with:
```shell
pip install docile-benchmark
uv pip install docile-benchmark
```

To convert pdfs into images, the library uses https://github.com/Belval/pdf2image. On linux you might need to install:
Expand All @@ -56,13 +65,17 @@ brew install poppler

Now you have all dependencies to work with the dataset annotations, pdfs, pre-comptued OCR and to run the evaluation. You can install extra dependencies by running the following (although using one of the provided dockerfiles, as explained below, might be easier in this case):
```shell
pip install "docile-benchmark[interactive]"
pip install "docile-benchmark[ocr]"
uv pip install "docile-benchmark[interactive]"
uv pip install "docile-benchmark[ocr]"
```

The first line installs additional dependencies allowing you to use the interactive dataset browser in [docile/tools/dataset_browser.py](docile/tools/dataset_browser.py) and the [tutorials](tutorials/). The second line let's you rerun the OCR predictions from scratch (e.g., if you'd like to run it with different parameters) but to make it work, you might need additional dependencies on your system. Check https://github.com/mindee/doctr for the installation instructions (for pytorch).

### Option 2: Use docker
</details>

<details>
<summary><b>Option 2: Use docker</b></summary>

There are two Dockerfiles available with the dependencies preinstalled:

* `Dockerfile` is a lighter, CPU-only, version with all necessary dependencies to use the dataset with the pre-computed OCR and interactive browser.
Expand All @@ -83,6 +96,8 @@ docker compose exec jupyter bash
```
After that the `docile` library and its dependencies are already installed in the environment; prefix commands with `uv run ...` (e.g. `uv run jupyter lab`) to run them inside it.

</details>

## Predictions format and running evaluation

To evaluate predictions for tasks KILE or LIR on the test set, you have to make a submission to the benchmark on the [Robust Reading Competition](https://rrc.cvc.uab.es/?ch=26) website. To evaluate on the validation set, use the following command:
Expand Down
Loading