Course in Machine Learning for Language Technologies

• Description
• General Information
• Lecturenotes
• Lab

Description

Für eine gegebene Aufgabe und ein Erfolgsmaß lernt ein Computerprogramm (und damit eine Maschine), wenn es die Aufgabe mit zunehmender Erfahrung besser lösen kann. In diesem Modul lernen die Studierenden das maschinelle Lernen für Sprachtechnologien als eine gezielte Suche in einem Raum potenzieller Hypothesen kennen. Eine Hypothese ist eine Funktion bzw. ihre Parameter, die eine zu erlernende Beziehung zwischen Predictor- und Response-Variablen beschreibt. Die Studierenden sollen einen breiten Überblick über die für Sprachtechnologien relevanten Lernparadigmen gewinnen und die jeweils grundlegenden Konzepte und Theorien verstehen. Die Entwicklung moderner Sprachtechnologien bis hin zu Sprachmodellen soll nachvollzogen, das Erlernte praktisch erprobt und der erzielte Erfolg bewertet werden.

General Information

Organization

• The lecture will take place in attendance. It will also be recorded and uploaded afterwards.
• In addition to the lecture, you will be provided with both theoretical and practical exercises. We will not collect or grade these exercises, but provide you with solutions and helpful tutorial videos (released at the same time). Nevertheless, you will still need to solve the exercises on your own/with a partner. You are responsible for your own learning success.
• We expect you to have studied the lecture, solved the exercises and reviewed the solutions before the corresponding lab session takes place. The lab sessions will be done in a flipped classroom format. This only works if you are well prepared. Please bring a laptop, tablet or smartphone to participate. The lab sessions will not be streamed or recorded.
• Please also communicate with your fellow students to exchange about the course topics, the exercises and solution approaches.

Exam

• The exam will be in German, but you are also allowed to write your answers in English.
• Please remember to bring your passport/Personalausweis and your student ID.
• You are allowed to bring a handwritten sheet (DIN A4, one side only) with useful information into the exam. It must be written by hand by yourself. The sheet will be collected together with the exam at the end of the exam.
• You are also allowed to bring a non-programmable calculator.
• If needed (international students), you may bring a dictionary.

Requirements

Prerequisites for this course are the basic modules about algorithms and data structures, theoretical computer science and mathematics. We will use Python as a programming language.
It might be helpful to refresh the following topics as their use will be required in the course. This list might be incomplete.

• Mathematical set notation
• Boolean formulas
• Matrix multiplication and transposition, matrix-vector multiplication
• Partial derivatives, rules of derivation, Taylor formula
• Straight line equation: Formulating the equation, drawing and reading a plot; quadratic functions
• Laws of exponents, laws of logarithms
• Probabilities: Product rule, conditional probabilities, independence, total probability, Bayes Theorem

Lecturenotes

• Machine Learning » Introduction » Organization, Literature [video]
• Machine Learning » Introduction » Learning Tasks [video]
• Machine Learning » Introduction » Elements of Machine Learning [video]
• Machine Learning » Introduction » Syntax & Model Overview
  
  
• Machine Learning » Machine Learning Basics » Concept Learning [video]
• Machine Learning » Machine Learning Basics » From Regression to Classification [video]
• Machine Learning » Machine Learning Basics » Evaluating Effectiveness [video part 1] [video part 2]
  
  
• Machine Learning » Linear Models » Logistic Regression [video]
• Machine Learning » Linear Models » Overfitting and Regularization [video]
• Machine Learning » Linear Models » Gradient Descent in Detail [video]
  
  
• Machine Learning » Bayesian Learning » Probability Basics [video]
• Machine Learning » Bayesian Learning » Bayes Classifier [video]
• Machine Learning » Bayesian Learning » Frequentist versus Subjectivist
  
  
• Machine Learning » Decision Trees » Decision Trees Basics [video]
• Machine Learning » Decision Trees » Impurity Functions [video part 1] [video part 2]
• Machine Learning » Decision Trees » Decision Trees Algorithms [video]
• Machine Learning » Decision Trees » Decision Trees Pruning [video]
  
  
• Machine Learning » Neural Networks » Perceptron Learning [video]
• Machine Learning » Neural Networks » Multilayer Perceptron [video part 1] [video part 2 (2023)]
• Machine Learning » Neural Networks » Advanced MLPs [video (2023)]

Logbook

Lab

Topic 1: Intro, Python and Maths Basics

Topic 2: Concept Learning & Evaluation

Topic 3: Linear Models

Topic 4: Bayesian Classification

Topic 5: Tree-Based Classification

Topic 6: Neural Networks