GP's are awesome. Usually, if you've got a few points, your first approach to "fitting a curve through them" would be to choose some parametric form and hope that you're not overfitting with e.g. too high a degree of polynomial. But what if you've got something piecewise, with unpredictable pieces? And what if you don't need a writable parametric form for your curve, but all you need is to answer the question "given my data, what's the probability distribution over the y value at x=5, 6, 7, 8, weighted over all likely curves that might fit my data based on how well they fit?"
Then a GP fit on your data will work wonders for you, as essentially an oracle for those kinds of queries. (And you can then just use the means of those distributions sampled at small intervals, if all you want is to throw a single curve on the screen.) Depending on your choices of priors and kernels, you can have it do magical things.
Thanks! I wrote that post! Another friend pointed out yesterday this other post about Gaussian Processes: https://www.jgoertler.com/visual-exploration-gaussian-proces... I think that post has some fun visualizations for showing how different kernels work, but I tend to prefer my explanation. Would love to get more eyes on it and feedback specifically about whether I have any mistakes in there!
Great post. A little typo I found was in the covariance equation (copy pasted when you copy-pasted the equation), there's an extra bracket at the end.
I loved the sentiment at the end:
A Gaussian Process might be useful for you. But please don't assume that it is sophisticated just because the language around it often is, or that its results are automatically true just because they have error bars.
GP's are awesome. Usually, if you've got a few points, your first approach to "fitting a curve through them" would be to choose some parametric form and hope that you're not overfitting with e.g. too high a degree of polynomial. But what if you've got something piecewise, with unpredictable pieces? And what if you don't need a writable parametric form for your curve, but all you need is to answer the question "given my data, what's the probability distribution over the y value at x=5, 6, 7, 8, weighted over all likely curves that might fit my data based on how well they fit?"
Then a GP fit on your data will work wonders for you, as essentially an oracle for those kinds of queries. (And you can then just use the means of those distributions sampled at small intervals, if all you want is to throw a single curve on the screen.) Depending on your choices of priors and kernels, you can have it do magical things.