Probabilistic Functional Programming
[go to overview]Winter Term 2016 / 2017
In the age of Big Data, Data Science, Machine Learning and Artificial Intelligence, programming with probabilities is a condicio sine qua non. And indeed the number of useful tools and libraries are legion. Disciples in these disciplines usually resort to R, Octave, Python - you name it - to do data analysis. And mastering these tools and libraries will probably land you a high-paying job. However, there is a serious drawback progressing this way - you will have a very hard time understanding how these tools and libraries do what they do and changing what they do remains a painful exercise, more painful that it need be.
Probabilistic functional programming allows developers to program with probabilities in a high-level language that is:
- close to the mathematical concepts one intends to program,
- easy to interpret and to visualize results.
Hence, probabilistic functional programming makes the obstacle of learning how to program with probabilities a lower one, it focuses on the concepts rather than on syntax and it avoids tool-ism, which is the blind-folded approach of calling up something one does not understand entirely - to some extent.
Given the current landscape of tools and libraries, when you become a die-hard-data scientist you will probably not be able to use current day probabilistic programming language like Church a lot. However, you will luckily remember the elegance of the solutions you have seen and truely understand what you have to accomplish.
Schedule
Lecture and Tutorial - Probabilistic Functional Programming (6 ECTS; for Master and Bachelor students in Web Science, Computer Science, Computational Visualistics and Business Informatics)
Students of computer science can use this course for Theory or for Data and Knowledge Engineering
Veranstaltungsnummer: 0432000
| Dozent(in) | Prof. Dr. Steffen Staab |
| Time slots |
Fr 10.15-11.45, Room F413 |
| Dozent(in) | Dr. Mahdi Bohlouli |
| Time slots |
Th 12.15-13.45, Room E524 |
Lectures are hold on Fridays beginning October 28, practical exercises are hold on Thursdays beginning November 3.
First exam will be held on Monday, Feb 20, 2017 at 10:00 AM in room M 201.
Second exam will be held on Friday, April 21, 2017 at 16:00 AM in room K 101. The registration for the second exam will be opened on 1st April and will be closed on 18th April 2017.
Course Slides
| Date | Lecture Topics (to be revised) | Slides |
|---|---|---|
| 28.10 |
Motivation and applications |
Motivation.pdf |
| 4.11. |
Functional Programming with Scheme (1) Functions, lists, let, lambda abstraction, application, recursion |
Scheme.pdf |
| 11.11. |
Functional Programming with Scheme (2) Tail recursion, closures |
Scheme2.pdf |
| 18.11. | Generative models | GenerativeModelsPart1.pdf |
| 24.11 | Generative models - Part 2 | GenerativeModelsPart2.pdf |
| 1.12. | Conditioning | Conditioning.pdf |
| 8.12. | Patterns of Inference | Patterns of Inference.pdf |
| Models for sequences of operations | Sequences of Observations.pdf (updated Dec 15) | |
| 15.12. | Inference about inference | InferenceAboutInference.pdf |
| 13.1. | Algorithms for inference | MCMC.pdf |
| 20.1. | Algorithms for inference - Part 2 | MetropolisHastings.pdf |
| 27.1. | Learning as conditional inference Hierarchical models |
LearningAsConditionalInference.pdf HierarchicalModels.pdprobfunprog-13-hierarchicalmodels.pdff |
| 3.2. | Occams Razor | OccamsRazor.pdf |
| 10.2. | - cancelled - | |
| 17.2. | Finale | update slides of OccamsRazor |
Exercise Slides:
| Date | Lecture Topics (to be revised) | Slides |
|---|---|---|
| 3.11. |
Getting started with Church |
Church_install.pdf |
| 10.11. | Practicing Church and basic instructions | practicing_Church_part1.pdf |
| 17.11. | Practicing Church and Functions | practicing_Church_part2.pdf |
| 25.11. | Discussion on Assignment 1 & further examples | Assignment_Solutions.pdf |
| 01.12. | Generative Models | Generative_Models.pdf |
| 08.12. | Conditioning | Conditioning.pdf |
| 15.12. | Discussion on Assignment 2 & further examples | Assignment02_Solutions.pdf |
| 12.01. | Discussion on Assignment 3 & further examples | Assignment03_Solutions.pdf |
| 19.01. | Discussion on Assignment 4 & further examples | Assignment04_Solutions.pdf |
| 26.01. | Discussion on practical examples | |
| 02.02. | Discussion on Assignment 5 & further examples (updated: 14th February) | Assignment05_Solutions.pdf |
| 09.02. | Discussion on practical examples | |
| 16.02. | Further examples and review | Review.pdf |
Assignments:
| Release Date | Assignment (to be revised) | Submission Deadline |
|---|---|---|
| 09.11. | Assignment Number 01 (20 Points) | 23.11 |
| 02.12. | Assignment Number 02 (20 Points) | 09.12 |
| 13.12. | Assignment Number 03 (14 Points) (updated 22 December) |
01.01. |
| 12.01. | Assignment Number 04 (20 Points) | 17.01. |
| 26.01. | Assignment Number 05 (20 Points) | 30.01. |
| 14.02. | Assignment Number 06 (20 Points) | 17.02. |
Core Literature & Systems
- Abelson, Sussman: Structure and Interpretation of Computer Programs, MIT Press, http://groups.csail.mit.edu/mac/classes/6.001/abelson-sussman-lectures/
- N. D. Goodman and J. B. Tenenbaum (electronic). Probabilistic Models of Cognition. Retrieved June 20, 2016 from http://probmods.org.
Further Literature & Systems
- Wood, F., van de Meent, J. W., & Mansinghka, V. (2014). A New Approach to Probabilistic Programming Inference. In Proceedings of the 17th International conference on Artificial Intelligence and Statistics (1024-1032). BIB PDF
- http://www.robots.ox.ac.uk/~fwood/anglican/
-
Michael Izbicki. HLearn: A Machine Learning Library for Haskell. Retrieved June 20, 2016.
- Extended WebChurch reference summary, WebChurch http://rpubs.com/CMoebus/123012.
- Probabilistic (Bayesian) Programming, University of Oldenburg, https://www.uni-oldenburg.de/en/computingscience/lcs/probabilistic-programming/
Lecturers
- Founder
- Alumnus