UltraBERT: Beyond the Limit of BERT Pre-Training
Paper • 2610.15028 • Published • 71
UltraBERT
UltraBERT: Beyond the Limit of BERT Pre-Training. Pre-trained 150M/380M encoders and a text embedding model, English, 32K context.
UltraBERT: Beyond the Limit of BERT Pre-Training. Pre-trained 150M/380M encoders and a text embedding model, English, 32K context.