👋 Welcome to Notion!

Здесь будут заметки от Дима Сафина и неравнодушных товарищей (Олега Кашурина) для подготовки к экзамену по второму семестру ML. Ждём обратной связи.

1. Text feature extraction: Bag of Words, Bag of Ngrams, Tf-Idf. Normalization Techniques.

2-3. Word embeddings. Word2Vec. CBOW, Skip-Gram. Matrix factorization, PPMI, GloVe algorithm

4. Metric learning, DSSM. Semantic search.

5. Sequence-to-sequence models. Seq2seq training. Decoding strategies (beam search, greedy, sampling)

6. Attention in Seq2seq

7. Machine Translation task. MT metrix, training pipeline

8. Transformer architecture. Self-attention. Transformer block structure. Positional encoding

9. Transfer learning idea. Contextual pretrained representations (CoVe, ELMo)

10. Transfer learning with pretrained models. GPT-1, BERT.

11. Generative pretraining. T5, GPT-2/3/4. Zero-shot and few-shot inference.

Prompting strategies (chain-of-thought, self-consistency).

12. LLM Alignment. Instruction tuning, RLHF

13. LLM Alignment. Contrastive learning: SLiC, DPO

14-15. PEFT: Sparse methods. Pruning, sparse fine-tuning. LoRA, prompt-tuning, multilayer prompt-tuning

16. PEFT: function composition (adapters, routing).