A random variable rarely arrives in the form you want. You measure a radius but care about the area; you model log-returns but trade prices; you have uniform random numbers but need a normal. Each case is a transformation, a function applied to a random variable, and it changes the distribution in a precise, computable way.
Almost every random sampler, and every modern flow-based or variational generative model, works by transforming one distribution into another. Master how a function reshapes a density and you hold the key to both classical change-of-variables problems and a large slice of generative AI.
Linear Transformations: Shift and Scale
The simplest transform is linear: Y = aX + b. It shifts the mean to aμ + b and scales the standard deviation to |a|σ, but it does not change the shape, a linear function of a normal is still normal.
In the notebook, X ~ Normal(50, 10) becomes Y = 2X + 5 ~ Normal(105, 20): the +5 shifts the center, the ×2 doubles the spread, and the bell shape survives intact. This is the standardization of Standardization & Z-Scores run in reverse.
Nonlinear Transformations: a New Shape
A nonlinear function can change the shape entirely. Exponentiating a normal, Y = eX, produces the right-skewed lognormal, the standard model for incomes, stock prices, and city sizes, quantities that are positive and multiply rather than add.
The symmetric input becomes a one-sided, heavy-right-tailed output, with a skewness of about 1.8 in the notebook. To handle reshaping like this precisely, we need a rule that tracks how a transform stretches and compresses probability, the change-of-variables formula.
The Change-of-Variables Formula
For a one-to-one transform Y = g(X), the density transforms as fY(y) = fX(x) / |g′(x)|. The derivative term, the Jacobian, corrects for how much the transform stretches or squeezes the axis at each point.
Probability is conserved: the same mass that sat in a small interval dx now spreads over a wider interval dy = |g′| dx, so the density must drop by the factor |g′|. For Y = eX with X standard normal, the formula gives fY(y) = φ(ln y)/y, and the notebook confirms it matches SciPy's lognormal to within 10−16. The 1/y is the Jacobian.
Inverse-Transform Sampling
One transform is so useful it deserves its own name. If U is uniform on [0, 1], then X = F−1(U) has cumulative distribution function F. Feeding uniform numbers through an inverse CDF generates samples from any distribution, the engine inside every random sampler.
In the notebook, plain uniforms pushed through the exponential's inverse CDF, X = −ln(1 − U), become exponential samples whose mean (0.997) matches the target. This is how a computer's single uniform generator is turned into a sampler for any distribution with an invertible CDF.
Transformations in Machine Learning & AI
Transforming distributions is not a sideshow in modern AI, it is the main event for generative models. Two ideas in particular, normalizing flows and the reparameterization trick, are change-of-variables made practical.
| Idea (this chapter) | In ML / AI it appears as | Concrete example |
|---|---|---|
| Change of variables / Jacobian | Normalizing flows | chain invertible transforms to model complex densities |
| Differentiable transform of noise | The reparameterization trick | z = μ + σε lets gradients flow through sampling (VAEs) |
| Inverse-transform sampling | Random-variate generation | turning a uniform RNG into any distribution |
| Nonlinear transforms | Activation & link functions | sigmoid, softplus, log transforms of skewed features |
| Log transform | Feature engineering | taming skewed inputs (income, counts) before modeling |
Generative AI is, to a large degree, applied change-of-variables. Normalizing flows build a rich data distribution by chaining many invertible transforms, tracking the Jacobian at each step exactly as in section 3, so they can both sample data and score its exact likelihood. The reparameterization trick writes a random sample as z = μ + σε (a transform of fixed noise ε), which moves the randomness out of the way so gradients can pass through the sampling step, the insight that makes variational autoencoders trainable. Inverse-transform sampling underlies the random generators every simulation relies on, and humble log transforms are the most common fix for skewed features. Knowing how a function reshapes a distribution is what lets researchers design models that create, rather than merely classify.
Reshape distributions in Python
The companion notebook applies a linear transform (a normal stays normal), exponentiates a normal into a lognormal, verifies the change-of-variables Jacobian against SciPy to 16 digits, generates exponential samples by inverting the CDF, and demonstrates the reparameterization trick that lets gradients flow through a sampling step.
View opens the rendered notebook instantly (no setup). Open in Colab runs &
edits it live in your browser. To run locally, install numpy, scipy, and
matplotlib and launch jupyter notebook.
🎓 Key Takeaways
- ✓Linear transforms Y = aX + b shift and scale but keep the shape; a linear function of a normal is normal.
- ✓Nonlinear transforms reshape entirely: exponentiating a normal gives the skewed lognormal.
- ✓Change of variables: fY(y) = fX(x)/|g′(x)|; the Jacobian conserves probability under stretching.
- ✓Inverse-transform sampling: X = F−1(U) turns uniforms into samples from any distribution.
- ✓In ML/AI: normalizing flows track the Jacobian, the reparameterization trick transforms noise for VAEs, and log transforms tame skewed features.
Practice Challenges
Five short challenges, beginner to intermediate. Try them with NumPy/SciPy before checking the solutions.
Linear transform
Celsius temperatures are Normal(20, 5). Fahrenheit is F = 1.8C + 32. Find the mean and standard deviation of F.
Make a lognormal
Let X ~ Normal(0, 1) and Y = eX. Simulate and confirm Y is right-skewed with median 1.
Apply the Jacobian
For Y = X² with X ~ Uniform(0, 1), the CDF of Y is √y. Verify P(Y < 0.25) = 0.5 by simulation.
Build a sampler
Use inverse-transform sampling to generate Exponential(rate = 2) variates from uniforms; check the mean is 0.5.
Reparameterize
Write a sample from Normal(10, 3) as a transform of standard-normal noise and confirm the statistics.
A fully-worked solutions notebook walks through all five challenges, each verified in code. Try them yourself first, then compare.
Quiz: Test Yourself
Eight quick questions on transformations of random variables. Answer them, hit Check Answers, and keep refining until you score 100%. Your progress is saved, so you can hop back to the chapter and return anytime.