---

[Github] https://github.com/createmomo (You can also find the Wechat Public Account here)

Notes

• [2023-, ing…] Open-Source Language Model Pocket
• [2023] Using ColossalAI SFT in Kaggle or Colab (in Chinese)
• [2023] General Understanding of Decoding Strategies Commonly Used in Text Generation
• [2022] Understand Gradient Checkpoint in Pytorch
• [2022] Baidu World Conference 2022: AI Applications (in Chinese, Notes)
• [2021] Super Git Revision Notes
• [2021] Baidu World Conference 2021: AI Applications (in Chinese, Notes)
• [2019] Probabilistic Graphical Models Revision Notes
• [2018] Super Machine Learning Revision Notes

Collections

• [2023-, ing…] Past of Goal-Guided Conversational AI Models
  • 1 Introduction
  • 2 Target-Guided Open-Domain Conversation (ACL2019)
  • 3 Proactive Human Machine Conversation with Explicit Conversation Goal (ACL2019)
  • 4 Towards Conversational Recommendation Over Multi-Type Dialogs (ACL2020)
  • 5 Knowledge Graph Grounded Goal Planning for Open-Domain Conversation Generation (AAAI2020)
  • 6 Towards Topic Guided Conversational Recommender System (Arxiv 2020)
  • 7 Towards Effective Automatic Debt Collection with Persona Awareness (EMNLP2023)
  • 8 Reinforcement Learning of Cooperative Persuasive Dialogue Policies Using Framing (COLING 2014)
  • 9 Dialogue Scenario Collection of Persuasive Dialogue with Emotional Expressions via Crowdsourcing (LREC 2018)
  • 10 Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good (ACL2019)
• [2023-, ing…] Using Language Models in Specific Domains
  • 1 Introduction
  • 2 Domain-specific Training Data
    • [Any Domain] Use Unlabelled Text to Improve Instruction Following Language Models (Notes 1)
    • [Medical/Health] ChatDoctor (Notes 1, 2, 3; Slides 1, 2, 3)
    • [Medical/Health] MedicalGPT-zh (Notes, Slides)
    • [Medical/Health] MING (Notes, Slides)
    • [Medical/Health] SoulChat (Notes, Slides)
  • 3 Automatic Model Evaluation
    • [Any Domain] Evaluating Language Models with Language Models (1 Introduction)
    • [Any Domain] Evaluating Language Models with Language Models (2 PandaLM)
    • [Any Domain] Evaluating Language Models with Language Models (3 Shepherd, 1,2,3,4)
    • [Medical/Health] Comparing ChatDoctor and ChatGPT3.5 using BERT-Score
• [2023-, ing…] Use Unlabelled Text to Improve Instruction Following Language Models
  • 1 Introduction
• [2023] Evaluating Language Models with Language Models
  • 1 Introduction
  • 2 PandaLM
  • 3 Shepherd (1, 2, 3, 4)
• [2023] Chinese Natural Language Understanding, NLU, in Dialogue Systems
  • 1&2 Task Introduction&Chinese vs English in NLP
  • 3.1 Academic Methods （Joint Training）
  • 3.2 Academic Methods （Separate Training）
  • 4 Industry Methods
  • 5&6 Chinese Dialogue System Challenge Track & Resources
• [2022-2023] Conference “Interesting”s
  • Interesting · ACL2023
  • Interesting · EACL2023
  • Interesting · EMNLP2023
  • Interesting · ACL2022 (Findings)
  • Interesting · ACL2022 (Short Papers)
  • Interesting · ACL2022 (Long Papers)
  • Interesting · NeurIPS2022
  • Interesting · EMNLP2022 (Findings&CL Papers)
  • Interesting · EMNLP2022 (Main Conference)
• [2021] Fantastic Trees (Decision Trees, Random Forest, Adaboost, Gradient Boosting DT, XGBoost)
• [2020] Improving Your English Communication Skills (Writing Emails, Speaking English and Building ePortfolio)
• [2017] CRF Layer on the Top of BiLSTM
  • CRF Layer on the Top of BiLSTM - 1 Outline and Introduction
  • CRF Layer on the Top of BiLSTM - 2 CRF Layer (Emission and Transition Score)
  • CRF Layer on the Top of BiLSTM - 3 CRF Loss Function
  • CRF Layer on the Top of BiLSTM - 4 Real Path Score
  • CRF Layer on the Top of BiLSTM - 5 The Total Score of All the Paths
  • CRF Layer on the Top of BiLSTM - 6 Infer the Labels for a New Sentence
  • CRF Layer on the Top of BiLSTM - 7 Chainer Implementation Warm Up
  • CRF Layer on the Top of BiLSTM - 8 Demo Code
    

Paper Explained

• [2021] GPT Understands, Too
• [2021] Few-Shot Text Classification with Distributional Signatures (ICLR 2020) Part1
• [2021] Few-Shot Text Classification with Distributional Signatures (ICLR 2020) Part2
• [2021] Few-Shot Text Classification with Distributional Signatures (ICLR 2020) Part3

---

Detailed Links:
* Fantastic Trees (Decision Trees, Random Forest, Adaboost, Gradient Boosting DT, XGBoost)

* Probabilistic Graphical Models Revision Notes

• Representations
  • Bayesian Network (directed graph)
    • Defination
    • Reasoning Patterns in Bayesian Network
    • Flow of Probabilistic Influence (active trial)
    • Independencies
    • d-seperation
    • I-Maps (Indenpendency Map)
    • Factorisation and I-Maps
    • Naive Bayes
    • Template Models
      • Temporal Models (involve over time)
      • 2 Time-Slice Bayesian Network (2TBN)
      • Plate Models
    • Conditional Probability Distribution (CPD)
      • General CPD
      • Table-based CPD
      • Context-specific Independence
      • Tree-Structured CPD
      • Multiplexer CPD
      • Noise OR CPD
      • Sigmoid CPD
      • Continuous Variables
  • Markov Network (undirected graph)
    • Markov Network Fundamentals
      • Pairwise Markov Networks
      • General Gibbs Distribution (a more general expression)
      • Induced Markov Network (connects every pair of nodes that are in the same factor)
      • Factorization
      • Conditional Random Fields
      • Independencies in Markov Networks
    • Local Structure in Markov Networks
      • Log-linear Models (CRF, Ising Model, Metric MRFs)
    • Decision Making
      • Maximum Expected Utility
      • Utility Functions
      • Value of Perfect Information
    • Knowledge Engineering
      • Generative vs. Discriminative
      • Designing a graphical model (variable types)
      • Structure
      • Parameters: Local Structure
      • Iterative Refinement
• Inference
  • Conditional Probability Queries (Overview)
    • Sum-Product in Bayes Network and Markov Network
    • Evidence: Reduced Factors
    • Summary of Sum-Product Algorithm
  • MAP (Maximum A Posterior) Inference (Overview)
    • Max-Product
  • Variable Elimination
    • Variable Elimination Algorithm
      • Elimination in Chains
      • Elimination in a more complicated BN
      • Variable Elimination with Evidence
      • Variable Elimination in MNs
      • Summary Variable Elimination Algorithm
    • Complexity of Variable Elimination
    • Graph-based Perspective on Variable Elimination
    • Finding Elimination Orderings
    • Variable Elimination (Summary)
• Learning
  

* Super Machine Learning Revision Notes

• Activation Functions
• Gradient Descent
  • Computation Graph
  • Backpropagation
  • Gradients for L2 Regularization (weight decay)
  • Vanishing/Exploding Gradients
  • Mini-Batch Gradient Descent
  • Stochastic Gradient Descent
  • Choosing Mini-Batch Size
  • Gradient Descent with Momentum (always faster than SGD)
  • Gradient Descent with RMSprop
  • Adam (put Momentum and RMSprop together)
  • Learning Rate Decay Methods
  • Batch Normalization
• Parameters
  • Learnable and Hyper Parameters
  • Parameters Initialization
  • Hyper Parameter Tuning
• Regularization
  • L2 Regularization (weight decay)
  • L1 Regularization
  • Dropout (inverted dropout)
  • Early Stopping
• Models
  • Logistic Regression
  • Multi-Class Classification (Softmax Regression)
  • Transfer Learning
  • Multi-task Learning
  • Convolutional Neural Network (CNN)
    • Filter/Kernel
    • Stride
    • Padding (valid and same convolutions)
    • A Convolutional Layer
    • 1*1 Convolution
    • Pooling Layer (Max and Average Pooling)
    • LeNet-5
    • AlexNet
    • VGG-16
    • ResNet (More Advanced and Powerful)
    • Inception Network
    • Object Detection
      • Classification with Localisation
      • Landmark Detection
      • Sliding Windows Detection Algorithm
      • Region Proposal (R-CNN)
      • YOLO Algorithm
        • Bounding Box Predictions (Basics of YOLO)
        • Intersection Over Union
        • Non-max Suppression
        • Anchor Boxes
    • Face Verification
      • One-Shot Learning (Learning a “similarity” function)
        • Siamese Network
        • Triplet Loss
      • Face Recognition/Verification and Binary Classification
    • Neural Style Transfer
  • Sequence Models
    • Recurrent Neural Network Model
    • Gated Recurrent Unit (GRU)
      • GRU (Simplified)
      • GRU (Full)
    • Long Short Term Memory (LSTM)
    • Bidirectional RNN
    • Deep RNN Example
    • Word Embedding
      • One-Hot
      • Embedding Matrix ($E$)
      • Learning Word Embedding
      • Word2Vec & Skip-gram
      • Negative Sampling
      • GloVe Vector
      • Deep Contextualized Word Representations (ELMo, Embeddings from Language Models)
    • Sequence to Sequence Model Example: Translation
      • Pick the most likely sentence (Beam Search)
        • Beam Search
        • Length Normalisation
        • Error Analysis in Beam Search (heuristic search algorithm)
      • Bleu Score
      • Combined Bleu
      • Attention Model
  • Transformer (Attention Is All You Need)
  • Bidirectional Encoder Representations from Transformers (BERT)
• Practical Tips
  • Train/Dev/Test Dataset
  • Over/UnderFitting, Bias/Variance, Comparing to Human-Level Performance, Solutions
  • Mismatched Data Distribution
  • Input Normalization
  • Use a Single Number Model Evaluation Metric
  • Error Analysis (Prioritize Next Steps)

[2017] CRF Layer on the Top of BiLSTM (BiLSTM-CRF)

• CRF Layer on the Top of BiLSTM - 1 Outline and Introduction
• CRF Layer on the Top of BiLSTM - 2 CRF Layer (Emission and Transition Score)
• CRF Layer on the Top of BiLSTM - 3 CRF Loss Function
• CRF Layer on the Top of BiLSTM - 4 Real Path Score
• CRF Layer on the Top of BiLSTM - 5 The Total Score of All the Paths
• CRF Layer on the Top of BiLSTM - 6 Infer the Labels for a New Sentence
• CRF Layer on the Top of BiLSTM - 7 Chainer Implementation Warm Up
• CRF Layer on the Top of BiLSTM - 8 Demo Code

---