Skip to content

Vocab file error #25

Open
Open

Description

@Tinarights

Hi,

I got an error when reading the vocab file: spm9998_3.model
RuntimeError: Internal: /sentencepiece/python/bundled/sentencepiece/src/sentencepiece_processor.cc(823) [model_proto->ParseFromArray(serialized.data(), serialized.size())]

When I google it, they said the file is not in the correct format.
Could you please help ASAP

here is the train instruction:
python train_abstractive.py -data_path data/ranked_wiki_b40/WIKI -mode train -batch_size 10500 -seed 666 -train_steps 1000000 -save_checkpoint_steps 5000 -report_every 100 -trunc_tgt_ntoken 400 -trunc_src_nblock 24 -visible_gpus 0 -gpu_ranks 0 -world_size 1 -accum_count 1 -dec_dropout 0.1 -enc_dropout 0.1 -label_smoothing 0.1 -vocab_path data/spm9998_3.model -model_path model/ -log_file log/log -inter_layers 6,7 -inter_heads 8 -hier

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions