Skip to content

Release v3.0b2

Latest

Choose a tag to compare

@hiroshi-matsuda-rit hiroshi-matsuda-rit released this 07 Aug 04:20
· 28 commits to main since this release
c693f9d
  • Update for Hugging Face Tokenizer
    • Following Llama tokenizer's manner, add BOS instead of EOD
  • No changes for SentencePiece model