Repository navigation

Adding GPT2 Tokenizer for WordTokenizers' Pretrained tokenizers - #61

Open

shikhargoswami wants to merge 7 commits into

JuliaText:masterfrom

shikhargoswami:GPT2tokenizer

Commits on Mar 17, 2021

Adding GPT2 tokenizer to WordEmbeddings
shikhargoswami
committed
8646640
Added tests
shikhargoswami
committed
179c517

Commits on Mar 18, 2021

Fixedloading issue
shikhargoswami
committed
de91b91

Commits on Mar 19, 2021

Added more tests and did required changes
shikhargoswami
committed
1d90ed2

Commits on Mar 22, 2021

Standardised API to match with existing one
shikhargoswami
committed
6f2f448
Corrected and Modified README
shikhargoswami
committed
6d2e8a9
Deleted Manifest.toml
shikhargoswami
committed
c03685c