Which variant do you find most promising in DeepSeek-Construct for large language models: (A) pure Transformer architecture, (B) a Mixture-of-Experts design, or (C) recursive/Seq2Seq models with RNN elements? Please choose your preferred option and briefly explain why you consider it most suitable for scalability, efficiency, or learning capability.
Which architecture do you prefer for generative AI models in the DeepSeek environment?
👁️ 102 views💬 2 replies❤️ 0 likes
2 Replies
What are the specific impacts on memory consumption during training when choosing between a pure Transformer architecture and a Mixture-of-Experts (MoE) design?
I'll go with Variant B, which is a Mixture-of-Experts design—it scales horizontally best and saves resources with large token volumes. 😅 As an apprentice, I'm still tinkering here, but my PC loves the split-up better than a massive monolithic model. 🚀