Academic Year 2023 - 2024
Semester 2

NUSMods Description

This module introduces the basics of modern information theory. It covers how information can be quantified, and what this quantification tells us about how well we can compress and transmit information without error. It discusses basic error correcting techniques, and information-theoretic cryptography. Topics covered include mathematical techniques, entropy measures, fundamental limits to data compression and noisy-channel coding, examples of error-correcting codes, examples of information theoretic cryptography (commitments, secure computation, key distribution, randomness extraction).

Review

This module is taught by Prof Jonathan Scarlett. It consists of a Midterm, a project and then a final.

I think this is a very nice class for those who want to delve into theoretical computer science. Personally, I found that I could solve the questions, but I was unsure why I do certain things. It is simply not intuitive. The content is the same throughout the past iterations, but I find the exams have gotten increasingly harder.

Tutorial questions were quite challenging, but if you have actually practiced, then there should be no issue. There are hints in the tutorial sheet so it might actually help. Personally, I find that I spent a lot of time solving the questions (yes, it is pretty unintuitive for me). I think the hardest tutorials pertain to Mutual Information, and Channel Coding. Channel coding questions mostly tell us to find the capacity of a certain channel and it is the maximum mutual information for some capacity-maximising probability distribution.

Midterms was not so tough, but I realised Prof deducted marks from me for writing useless stuff. When I was doing the questions, sometimes, I scribble random stuff to get ideas and not erase them anyway but well, he does not seem to like it (or even ignore it). I got quite a high mark which he thinks is good, but falls on the median line.

For the project, it was done individually. For the first part, there were not much of an issue as we had to write code to test out the compression for a file compressor and evaluate its compression capability over different types of files. The second part is just writing about some use case of information theory.

For the finals, well, it was considerably tough. I find myself blanking out certain times. Not sure why but I panicked so early in the exam. No calculators were allowed. I did not want to be messy on the paper so I sketched my solutions in the scratch paper and transferred it over. There was not enough time for me to attempt the last question part for 8 marks. I also did not do 7 marks worth as I was unsure what he wanted by proving equivalence of certain theorem. The final was not my best attempt at all. No expectations for this class, though I did learn alot from this class. I gave up caring about grades at that point. I mainly focused on learning and absorbing as much as possible and enjoying information theory (no matter how unintuitive it is to me).