AIが動く。未来が更新される。DanTagGen-delta-rev2
DanTagGen(Danbooru Tag Generator) is inspired from p1atdev's dart project. But with different arch, dataset, format and different training strategy. - alpha: pretrain on 2M dataset, smaller batch size. Limited ability - beta: pretrain on 5.3M dataset, larger batch size. More stable, better ability with only a few information provided. - delta: pretrain on 7.2M dataset, larger batch size. Slightly underfit but better diversity. quality tag introduced. - rev2: resumed from delta, same dataset, 2 more epoch. This version of DTG is trained from scratch with 400M param LLaMA arch.(In my personal preference I will call it NanoLLaMA) Since it is llama arch. Theoritically it should be able to be used in any LLaMA inference interface.
翻訳準備中のため原文を表示しています
DanTagGen(Danbooru Tag Generator) is inspired from p1atdev's dart project. But with different arch, dataset, format and different training strategy. - alpha: pretrain on 2M dataset, smaller batch size. Limited ability - beta: pretrain on 5.3M dataset, larger batch size. More stable, better ability with only a few information provided. - delta: pretrain on 7.2M dataset, larger batch size. Slightly underfit but better diversity. quality tag introduced. - rev2: resumed from delta, same dataset, 2 more epoch. This version of DTG is trained from scratch with 400M param LLaMA arch.(In my personal preference I will call it NanoLLaMA) Since it is llama arch. Theoritically it should be able to be used in any LLaMA inference interface.