"Large Language Model: Principles and Engineering Practice" - Evaluation + Application + Engineering Practice

rtyu789

"Large Language Model: Principles and Engineering Practice" - Evaluation + Application + Engineering Practice [Copy link]

<div class='showpostmsg'># 1. Evaluation of large language models Main evaluation dimensions: 1. Language model capability 2. Comprehensive knowledge capability 1. Common sense reasoning, such as PIAQ, SIQA, ARC, etc. 2. Reading comprehension, such as Natural Questions, TriviaQA, etc. 3. World knowledge, such as MMLU, AGIEval, etc. 3. Mathematical calculation ability 4. Coding ability 5. Vertical field Main evaluation tasks: 1. Life chat 2. Methodology guidance 3. Language comprehension 4. Common sense encyclopedia 5. Mathematical calculation 6. Logical reasoning 7. Abstract generation 8. Copywriting 9. Translation 10. Coding 11. Multi-round dialogue! [1_FinanceIQ Chinese evaluation data set] (/data/attachment/forum/202410/10/230947jzkncrnvyv40vvrq.jpg.thumb.jpg?rand=992.6585322099935) # 2. Application of large language model - prompt word engineering Prompt word engineers make up for the difference between the thinking methods of humans and large language models, so that the output process of large language models can imitate the thinking process of humans! [2_Prompt word engineering] (/data/attachment/forum/202410/10/230948ipe1p5fec1d9cgm7.jpg.thumb.jpg?rand=6410.109318548003) Prompt word technology: 1. Zero sample prompt 2. Few sample prompt 3. Thinking chain prompt 4. Self-consistency prompt 5. Thinking tree prompt retrieval enhanced interaction technology, RAG technology# 3. Engineering practice## 3.1 Data parallelism During the training process, the data set is split and loaded into each computing node. The content of each computing node is completely consistent, and the data in each GPU is gradient differentiated! [3_Data Parallel PS Architecture](/data/attachment/forum/202410/10/230948zedkdei3diizd8rl.jpg.thumb.jpg?rand=5610.468624230159) ## 3.2 Model parallelism is mainly to solve the problem of excessive scale of model weight parameters. The weight parameters of a model are divided into multiple small parts, and the weight parameters of each part are large enough to be accommodated by a single computing node. Can be divided into tensor parallelism and pipeline parallelism  ## 3.3 ZeRO parallelism reduces storage redundancy in data parallel training and splits the data among multiple computing nodes, effectively reducing the amount of data that needs to be stored on each device  ## 3.4 Training optimization 1) IO optimization Optimization solution 1: Highly available large-capacity storage service + local cache. Suitable for training TB-level data! [8_Local cache acceleration](/data/attachment/forum/202410/10/230952h3rgwjgng7fd8dn7.jpg.thumb.jpg?rand=6308.319707358532) Optimization solution 2: High-availability large-capacity storage service + high-performance distributed cache. Suitable for training data volumes of tens to hundreds of TB 2) Communication optimization In multi-machine and multi-node communication, reasonable communication optimization is performed. Nodes within a single machine use NVIDIA's proprietary high-speed link NVLikn, and multiple machines communicate through Ethernet or RDMA networks 3) Stability optimization 1. 1. Troubleshooting and prevention, capacity monitoring, traffic monitoring, demo testing of small-scale data volumes 2. Timely detection of faults: training interruptions or training abnormalities 3. Quick recovery: when a fault occurs, take effective measures to quickly recover # References [Parallel Technology for Distributed Training of Large Models (II) - Data Parallelism](https://zhuanlan.zhihu.com/p/650002268) [Advanced Road to Distributed Training of Large Models: Data Parallelism, Pipeline Parallelism, Tensor Parallelism, 3D Parallelism](https://blog.csdn.net/python123456_/article/details/141557678)</div><script> var loginstr = '<div class="locked">To view all the contents of the post, please <a href="/bbs/member.php?mod=logging&action=login" style="color:#e60000" class="loginf">log in</a> or <a href="/bbs/member.php?mod=register_eeworld.php&action=wechat" style="color:#e60000" target="_blank">register</a></div>'; if(parseInt(discuz_uid)==0){ } </script><script type="text/javascript">(function(d,c){var a=d.createElement("script"),m=d.getElementsByTagName("script")[0],eewurl="//counter.eeworld.com.cn/pv/count/";a.src=eewurl+c;m.parentNode.insertBefore(a,m)})(document,523)</script>