AI가 움직이고 미래가 업데이트됩니다.DanTagGen-delta-rev2
DanTagGen(Danbooru Tag Generator) is inspired from p1atdev's dart project. But with different arch, dataset, format and different training strategy. - alpha: pretrain on 2M dataset, smaller batch size. Limited ability - beta: pretrain on 5.3M dataset, larger batch size. More stable, better ability with only a few information provided. - delta: pretrain on 7.2M dataset, larger batch size. Slightly underfit but better diversity. quality tag introduced. - rev2: resumed from delta, same dataset, 2 more epoch. This version of DTG is trained from scratch with 400M param LLaMA arch.(In my personal preference I will call it NanoLLaMA) Since it is llama arch. Theoritically it should be able to be used in any LLaMA inference interface.
번역 준비 중입니다. 원문을 표시합니다.
DanTagGen(Danbooru Tag Generator) is inspired from p1atdev's dart project. But with different arch, dataset, format and different training strategy. - alpha: pretrain on 2M dataset, smaller batch size. Limited ability - beta: pretrain on 5.3M dataset, larger batch size. More stable, better ability with only a few information provided. - delta: pretrain on 7.2M dataset, larger batch size. Slightly underfit but better diversity. quality tag introduced. - rev2: resumed from delta, same dataset, 2 more epoch. This version of DTG is trained from scratch with 400M param LLaMA arch.(In my personal preference I will call it NanoLLaMA) Since it is llama arch. Theoritically it should be able to be used in any LLaMA inference interface.