Examining Scaling and Transfer of Language Model Architectures for Machine Translation

This Research paper compares how Large Language Models can or can not be applied towards Machine Translation in contrast to traditional Encoder Decoder models.

Main Conclusions:

• Encoder Decoder models still out perform Language Models.
• Prefix based LMs that are deep and make the full source visible do better than other LM variants and are comparable to Encoder Decoder models.