Essays

A bricolage representing modern statistical mechanics in all its multifaceted chaotic order. Containing numbers (information), particles in a box, Jupiter's great red eye, RNA hairpins, bacteria with wavy flagella. Made in Ideogram V1, with prompt 'An abstract conceptual illustration of a brutalist structure, with sharp angles and a minimalistic design, made of concrete, steel balls, and rebars. The structure appears to be made up of particles in a box, jamming together, creating bits and numbers in the realm of 0-1. The design also features an RNA hairpin, with eddies upon eddies upon eddies, evoking the image of Jupiter's red spot. The random walk of swimming bacteria with wavy flagella is depicted, adding to the complexity of the design. The illustration is presented on a white background, with high contrast and a monochromatic color scheme, reminiscent of a flat style vector SVG art., illustration, conceptual art'.

Statistical Mechanics

math
physics
probability
statistics
biology
How to think like a classical statistical-mechanic, with many examples from gas theory, biology, probability, and information theory. Prerequisites: thermodynamics, calculus, probability, and mathematical maturity.
2024-07-04
117 min

Hole Argument and Inverted Qualia

fun
philosophy
math
physics
biology
The hole argument in general relativity is formally analogous to the inverted qualia problem in philosophy. Like how spacetime points have no existence beyond gauge freedom, qualias have no existence beyond their geometric-functional roles, thus dissolving the hard problem of consciousness.
2024-06-18
63 min

A brutalist sculpture representing classical thermodynamics/neoclassical economics: nonlinear (curved surface) constraint (steel cables tying it down) optimization (the surface struggling to fly towards the sun, touching the solar rays at the closest points). Made in Ideogram V1, with prompt 'A stunning conceptual artwork featuring a white sun casting intense white rays of sunlight, piercing through a curved hyperboloid structure made of concrete. The structure is adorned with straight lines of tight steel cables, creating a strong geometric pattern. The white background and high contrast accentuate the minimalistic vector art style. The piece is reminiscent of Chiaroscuro painting in the style of Piranesi, with a monochrome palette that enhances the drama and depth of the scene., illustration'.

Classical Thermodynamics and Economics

math
physics
philosophy
economics
How to think like a classical thermodynamic-economist, delivered with many illustrations and some sci-fi metaphors. Particular emphasis on what traditional pedagogy gets wrong. Prerequisites: multivariate calculus and mathematical maturity.
2024-05-09
124 min

A brutalist representation of the particle-wave duality of the Hamiltonian--Jacobi equation. The stacked platforms represent the Hamilton principal function according to the field-theoretic wave interpretation, and the steel rebars are the characteristic rays, with balls rolling along them representing the particle interpretation. Made in Ideogram V1 with the following prompt: 'A brutalist terraced mountain made of concrete and steel rebar. There are ripples parallel to the ground. The steel rebars go downhill, and many marbles are rolling down along the steel rebars. White background. High contrast, grayscale, black-and-white minimalistic, in the flat style of vector svg art, conceptual art'

Analytical Mechanics

math
physics
philosophy
What every graduate student should know about analytical mechanics, delivered with economic style and many illustrations. Particular focus on particle-wave duality and old quantum theory. Prerequisites: multivariate calculus and mathematical maturity.
2024-05-04
111 min

A metaphor for real-space renormalization, where the building on the left is coarse-grained to the building on right. The facade of a wide brutalist architecture. On the left, the windows are small and the floors are close together. On the right, the windows are large and the floors are wide apart. High contrast, monochromatic, minimalistic, in the flat style of vector svg art., illustration, conceptual art.

How to do Renormalization

math
physics
scaling
Survival guide for renormalization (RN) theory. How to do it in real space and frequency space. What it all means. Should have something for everyone, from mathematicians to physicists to outsiders, from serious series to pretty pictures. Intended audience: those with two years of undergrad mathematics.
2024-04-11
78 min

Mixture of Experts

AI
scaling
How MoE works, its history, and what it is good for.
2024-01-23
14 min

An abstract representation of the Turing test. On the left is a dark city of obelisks over a bright background, and on the right is the same thing, mirrored and inverted, but with subtle differences due to the randomness of the art generator. It represents the abstract idea of the Turing test: duality, subtle differences, same contours, and complex object made of simple parts. High contrast, monochromatic, minimalistic, in the style of vector svg art.

Predicting AGI by the Turing Test

AI
scaling
math
Minimizing log-perplexity loss is equivalent to maximizing survival length in a Turing test. Assuming compute-loss scaling law, a scaled-up GPT that produces human-like science papers would cost ~200 years of global GDP.
2024-01-20
30 min

Brutalist architecture representing a single perceptron. A hexagonal building in the center representing the unit itself, with multiple arc legs representing the input weights, and antennas projecting off the top representing the output. High contrast, monochromatic, minimalistic, in the style of vector svg art.

The Perceptron Controversy

AI
history
Connectionism died in the 60s from technical limits to scaling, then resurrected in the 80s after backprop allowed scaling. The Minsky–Papert anti-scaling hypothesis explained, psychoanalyzed, and buried.
2024-01-01
113 min

Stylized illustration of a computational graph, with gibberish text that resemble vector calculus, thick and thin lines representing the flow of numbers in the graph. In the center of the vaguely octagonal graph is a letter D, D for derivative. High contrast, monochromatic, minimalistic, in the style of vector svg art.

The Backstory of Backpropagation

AI
math
physics
history
Why backprop was resisted for 20 years: assumption of discretely spiking neurons, goal of synthesizing Boolean logic, fear of local optima, and bad luck. Werbos has the best claim for invention.
2023-12-26
64 min

A grid of electromechanical spider-like robots on a concrete surface, with small cracks and many rounded-headed copper spires like tallgrass. It might be called poetically 'a cybernetic meadow'. Realistic sci-fi illustration.

Cybernetic Artificial Intelligence

AI
cybernetics
math
history
Machine learning and self-reproduction according to Norbert Wiener.
2023-12-23
16 min

The cover of 'Project SOGO: Principles of Hypnotic Bionics'. It is a combination of 'Project PARA: Principles of Neurodynamics' and 'Shoggoth' from Lovecraft's cosmology. The idea is that the book is mythical like the Necronomicon (see Minsky's quote). In the style of a cold-war era RAND corporation report.

Reading Perceptrons

AI
math
A long, hard stare into the math of Perceptrons, the mythical neural network killer.
2023-12-21
25 min

Brutalist architecture representing a neural network. A vaguely hexagonal 3D array of concrete blocks connected by thin 'walkways', small thin steel 'lampposts' topped with balls, and balls rolling around.

Fermi Estimation for Neural Networks

AI
economics
The bitter lesson in bite-sized packets.
2023-12-05
30 min

Neural Scaling Laws by Data Manifold Dimensions

AI
scaling
Neural networks scale they way they do, purely because of data.
2023-11-01
8 min

Grokking Modular Arithmetics

AI
interpretation
Toy model of grokking: tiny neural networks learn modular arithmetics after it has already reached full correctness on the training set.
2023-09-01
8 min

A girl facing away from the viewer, sitting in a datacenter, connected by wires to the server racks. In the middle-top is an ethereal tree-shaped network of light, representing the neural network of Lain. Cyberpunk-style anime art.

Serial Experiments Lain

anime
fun
Lain is the collective subconscious as a neural network.
2022-11-01
13 min

The Racial Algorithmic Bias Controversy

probability
statistics
The racial algorithmic bias controversy, as seen by a mathematician.
2019-07-13
14 min

A picture of iterating the same Lorentz boost for many times on a plus-sign with unit-length arms, creating the outlines of a hyperbola.

The Wigner Rotation in Special Relativity via Hyperbolic Geometry

physics
math
fun
The other special relativity paradox that you have never heard of.
2018-05-01
54 min
No matching items