Turing.jl Overview

Turing.jl: A Powerful Julia Framework for Probabilistic Programming.

Introduction

Turing.jl is a high-level Julia framework for probabilistic programming. It provides a powerful set of tools and abstractions for building and performing inference on probabilistic models. With Turing.jl, you can express complex statistical models concisely and efficiently, and then perform various types of inference to estimate model parameters or make predictions.

History

Turing.jl was first introduced in 2018 by Hong Ge and Kai Xu. It was designed to bring the flexibility and expressiveness of probabilistic programming to Julia, a high-performance programming language. Since its inception, Turing.jl has gained popularity among the Julia community and has been widely used in various domains, including machine learning, Bayesian statistics, and finance.

Features

Turing.jl offers a range of powerful features that make it a versatile framework for probabilistic programming:

1. Modularity: Turing.jl allows you to easily define and compose probabilistic models using a modular and hierarchical approach. You can build complex models by combining simpler model components, making it easy to reuse and extend your code.

   using Turing
   
   @model function linear_regression(x, y)
       σ ~ InverseGamma(2, 3)
       β ~ Normal(0, 1)
       α ~ Normal(0, 1)
       y ~ Normal(α + β * x, σ)
   end

   In the above example, we define a linear regression model using the @model macro provided by Turing.jl. The model consists of three random variables (σ, β, and α) and a likelihood (y). This modular approach allows us to easily add or remove components as needed.

2. Flexible Inference: Turing.jl supports a variety of inference algorithms, including Markov chain Monte Carlo (MCMC), variational inference, and sequential Monte Carlo. This flexibility allows you to choose the most suitable algorithm for your specific problem.

   using Turing, MCMCChains
   
   model = linear_regression(x, y)
   chain = sample(model, NUTS(), 1000)
   summary(chain)

   In this example, we use the NUTS (No-U-Turn Sampler) algorithm to perform MCMC sampling on our linear regression model. The sample function returns a chain of samples from the posterior distribution, which can then be analyzed using the MCMCChains package.

3. Automatic Differentiation: Turing.jl leverages Julia's powerful automatic differentiation capabilities to calculate gradients and perform efficient inference. This allows for fast and accurate estimation of model parameters, even for complex models.

   using Turing, ForwardDiff
   
   model = linear_regression(x, y)
   gradients = gradient(model, [x, y], α)

   In this example, we calculate the gradients of the model with respect to the inputs x and y, and the parameter α. This can be useful for tasks such as gradient-based optimization or sensitivity analysis.

4. Model Checking: Turing.jl provides tools for model checking and diagnostics, allowing you to assess the quality of your model and the convergence of the inference algorithm. This helps identify potential issues and improve the reliability of your results.

   using Turing, MCMCChains, Plots
   
   model = linear_regression(x, y)
   chain = sample(model, NUTS(), 1000)
   plot(chain)

   In this example, we use the plot function from the Plots package to visualize the samples from the posterior distribution. This can provide insights into the uncertainty of the model parameters and help diagnose potential convergence problems.

Examples

Let's explore a few examples to illustrate the power and versatility of Turing.jl.

Example 1: Bayesian Linear Regression

using Turing, Distributions, Random

# Generate synthetic data
Random.seed!(123)
x = rand(Uniform(-5, 5), 100)
y = 2x + rand(Normal(0, 1), 100)

# Define the model
@model function linear_regression(x, y)
    σ ~ InverseGamma(2, 3)
    β ~ Normal(0, 1)
    α ~ Normal(0, 1)
    y ~ Normal(α + β * x, σ)
end

# Perform inference
model = linear_regression(x, y)
chain = sample(model, NUTS(), 1000)

# Summarize the results
@show summary(chain)

In this example, we use Turing.jl to perform Bayesian linear regression on synthetic data. We define a linear regression model with priors on the intercept (α), slope (β), and noise (σ). We then sample from the posterior distribution using the NUTS algorithm and summarize the results.

Example 2: Gaussian Mixture Model

using Turing, Distributions, Random

# Generate synthetic data
Random.seed!(123)
x = [rand(Normal(-5, 1), 100); rand(Normal(5, 1), 100)]

# Define the model
@model function gaussian_mixture(x, K)
    μ = Vector{Real}(undef, K)
    σ = Vector{Real}(undef, K)
    w = Vector{Real}(undef, K)

    for k in 1:K
        μ[k] ~ Normal(0, 10)
        σ[k] ~ Uniform(0, 10)
        w[k] ~ Dirichlet(ones(K))
    end

    z = Vector{Real}(undef, length(x))
    for i in 1:length(x)
        z[i] ~ Categorical(w)
        x[i] ~ Normal(μ[z[i]], σ[z[i]])
    end

    return μ, σ, w
end

# Perform inference
model = gaussian_mixture(x, 2)
chain = sample(model, HMC(), 1000)

# Summarize the results
@show summary(chain)

In this example, we use Turing.jl to perform inference on a Gaussian mixture model. We define a model with parameters for the means (μ), standard deviations (σ), and mixture weights (w) of two Gaussian components. We then sample from the posterior distribution using the Hamiltonian Monte Carlo (HMC) algorithm and summarize the results.

Conclusion

Turing.jl is a powerful Julia framework for probabilistic programming, offering a wide range of features and tools for building and performing inference on probabilistic models. With its modular and flexible design, automatic differentiation capabilities, and model checking tools, Turing.jl enables efficient and reliable probabilistic modeling and inference. Whether you are working on Bayesian statistics, machine learning, or finance, Turing.jl can be a valuable tool in your toolkit.

For more information and detailed documentation, visit the official Turing.jl website: https://turing.ml.