1. Base

• One-hot, TF-IDF, Bi-gram

• N-gram Model: n元语法模型。(n=1)uni-gram 一元模型，(n=2)bi-gram 二元模型，(n=3)tri-gram 三元模型

• Neural Network Language Model(NNLM)：神经网络语言模型

• RNNLM

• Word Embedding：

  • word2vec：1) Continuous Bag-of-Words, CBOW 连续词袋模型 2) Skip-gram 跳字模型
  • Glove (global vectors)

• Seq model

  • RNN (Recurrent Neural Networks)

  • LSTM (Long Short-Term Memory)

  • GRU (Gated Recurrent Unit)

  • Encoder-Decoder

  • Attention

• Large Model

  • ELMo
  • Transformer
  • BERT
  • GPT
  • DETR
  • CLIP
  • Stable Diffusion
  • Llama
  • Deepseek

2. Focus

• NLP: RNN, LSTM, Attention, Transformer, Bert, ChatGPT, Deepseek 原理

• CV:

  • Classify: ResNet
  • Detect: RCNN series, YOLO series
  • Segment: SAM
  • Generate: GAN, AE/VAE, Stable Diffusion, Sora
  • LM: ViT, DETR

• MLLM：BLIP-2, Clip, LLaVA, Qwen-VL