diff --git a/README.md b/README.md index e2ced9a6..de195a95 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,13 @@ -# STMO-ZOO +# Cross-Entropy method +By Ceri-Anne Laureyssens -Welcome to the STMO zoo! This is your final assignment for the course Selected Topics in Mathematical Optimization. Your goal is to implement an optimization method in Julia and contribute this to this repository. To pass, you have to: +The notebook gives an introduction to cross-entropy and its use in the cross-entropy method. Cross-entropy is a metric used to measure the Kullback-Leibler (KL) distance between two probability distributions (f and g). The cross-entropy method is a Monte Carlo method for importance sampling and optimization and is found by minimizing the previously called KL distance in between distribution f and g (parameterized by θ). This is equivalent to choosing θ that minimizes the cross-entropy. -- fork this repo and create a pull request; -- add a module to `src` with **at least one function** -- add at least one unit test to the folder `test`; -- document all your functions and add a page to the documentation page; -- make a notebook in [Pluto](https://github.com/fonsp/Pluto.jl) and add it to `notebooks`; -- perform a small code review of two other students. +The notebook provides an implementation of the cross entropy method for optimizing multivariate time series distributions. Suppose we have a timeseries `X = {x₁, ..., xₙ}` where each `xᵢ` is a vector of dimension `m`. The `cross_entropy_method` function can handle two different scenarios: -Depending on the project you choose some of these individual assignments might be really minimalistic, with other parts larger. For example, if you want to develop an application, say solving the graph coloring problem with Tabu Search, you might have only a single function in the source code (e.g., generating an instance) but have a fairly large notebook with a tutorial. On the other hand, if you work on a method, e.g., implementing Bee Colony Optimization, you might have many functions in the source code, while your notebook is only a demonstration on the test functions. +1. The time series is sampled IID from a single distribution `p`: `xᵢ ~ p(x)`. In this case, the distribution is represented as a `Dict{Symbol, Tuple{Sampleable, Int64}}`. The dictionary will contain `m` symbols, one for each variable in the series. The `Sampleable` object represents `p` and the integer is the length of the timeseries (`N`). +2. The time series is sampled from a different distribution at each timestep `pᵢ`: `xᵢ ~ pᵢ(x)`. In this case, the distribution is also represented as a `Dict{Symbol, Tuple{Sampleable, Int64}}`. -[![Build Status](https://travis-ci.org/MichielStock/STMOZOO.svg?branch=master)](https://travis-ci.org/MichielStock/STMOZOO)[![Coverage Status](https://coveralls.io/repos/github/MichielStock/STMOZOO/badge.svg?branch=master)](https://coveralls.io/github/MichielStock/STMOZOO?branch=master) \ No newline at end of file +Finishing off the notebook provides a fun little example implementing the CE method in the importance sampling technique. + +[![Build Status](https://travis-ci.org/MichielStock/STMOZOO.svg?branch=master)](https://travis-ci.org/MichielStock/STMOZOO)[![Coverage Status](https://coveralls.io/repos/github/MichielStock/STMOZOO/badge.svg?branch=master)](https://coveralls.io/github/MichielStock/STMOZOO?branch=master) diff --git a/notebook/crossentropy.jl b/notebook/crossentropy.jl new file mode 100644 index 00000000..3c75f147 --- /dev/null +++ b/notebook/crossentropy.jl @@ -0,0 +1,1325 @@ +### A Pluto.jl notebook ### +# v0.17.1 + +using Markdown +using InteractiveUtils + +# ╔═╡ 9c2b64cf-c236-46aa-a4d6-02a22c5b7faa +using Distributions + +# ╔═╡ c5ef7103-4aca-49dc-b49b-4cf5ac157701 +using Random.Random + +# ╔═╡ f126a1f8-afc6-4b52-9021-39e946dd0ea2 +md""" +# Cross-Entropy Method + +STMO Exam Project by **Ceri-Anne Laureyssens** +""" + +# ╔═╡ c629d020-2c17-477c-8517-68c39f46fbcb +md""" +### Cross-entropy + +As the title states the method has to do with the cross-entropy. It's a metric used to measure the distance between two probability distributions. These two distributions, as you will read later on, are in fact the original distribution ($$f$$) and the optimized distribution based on elite samples ($$g$$). The distance used to define cross-entropy is called the Kullback-Leibler (KL) distance and measures how one probability distribution is different from a second, reference probability distribution. This KL distance or relative entropy can be easily used to derive the cross-entropy itself and is defined as follows: + +```math +\begin{equation} +D_{KL}(f,g) = E_f[\log\frac{f(X)}{g(X)}] += \int_{x\in χ} f(x)\log f(x)dx - \int_{x\in χ} f(x)\log g(x)dx +\end{equation} +``` + +With $$D$$ being the KL distance, $$E$$ the expectation, and $$X$$ a random variable with support $$χ$$. + +Minimizing this KL distance in between distribution $$f$$ and $$g$$ (parameterized by $$θ$$) is equivalent to choosing $$θ$$ that minimizes the cross-entropy: + +```math +\begin{equation} +H(f,g) = H(f) + D_{KL}(f,g) += -E_f[\log g(X)] += -\int_{x\in χ} f(x)\log g(x|θ)dx +\end{equation} +``` + +With $$H$$($$f$$) being the entropy of distribution $$f$$. + +This assumes that $$f$$ and $$g$$ share the support $$χ$$ and are continuous with respect to $$x$$. The minimization problem becomes the following: + +```math +\begin{equation} +minimize_θ - \int_{x\in χ} f(x)\log g(x|θ)dx +\end{equation} +``` + +Finding this minimum is the goal of the method explained in the next section. +""" + +# ╔═╡ 2c923610-52c4-11ec-367b-2565c12740ab +md""" +### Method description and implementation + +The cross-entropy or CE method is a Monte Carlo method for importance sampling and optimization. +Monte-Carlo algorithms rely on repeated sampling to obtain numerical results and use randomness as underlying concept to solve the given problem. + +The CE method can be used for both combinatorial, such as the travelling salesman problem and the knapsack problem, as continuous problems, with either a static or noisy objective, and is also a valuable asset in rare-event simulation. In the examples called before a brute-force algorithm is not tractable. As such specialized algorithms, like the CE method, can be used because they quickly rule out large parts of the search space. + +The simple two-step process involves generating a random data sample according to a specified mechanism and updating the parameters of the mechanism based on the data to produce a better sample in the next iteration as can be seen in the figure underneath. +""" + +# ╔═╡ 2588e834-f2ee-41d1-b2c3-dbc43a0cda63 +md""" +![](https://github.com/CeriAnneLaureyssens/CrossEntropy.jl/blob/master/notebook/figures/CE_visual.jpg?raw=true) +""" + +# ╔═╡ 9b4da26a-6136-4be7-b3d8-ad573a89c7f9 +md""" +We start with a specified distribution in the left panel from which a fixed amount of samples are randomly derived. These randomly derived samples can be seen in the middle panel. The cross-entropy method makes use of an objective, or loss (as mentioned further down), function ($$S$$) which minimizes the cross-entropy between the known distribution $$f$$ and a proposal distribution $$g$$ parameterized by $$θ$$. + +```math +\begin{equation} +θ_g^* = \text{argmin}_{θ_g} - \int_{x\in χ} I_{(S(x)≥γ)}g(x|θ) \log g(x|θ_g)dx += \text{argmin}_{θ_g} - E_θ[I_{(S(x)≥γ)} \log g(x|θ_g)] +\end{equation} +``` + +With $$I$$ being the indicator function, and $$γ$$ a threshold with which elite samples are determined. + +The grey dots indicate the elite samples. One becomes the elite samples by sorting the outcome of the objective function (done for every sample in the distribution). These elite samples are then selected as the samples of the new distribution, which can be seen in the third panel, and the whole process, starting from the left panel, is repeated until the distribution is optimized. +""" + +# ╔═╡ aba389ff-77b4-4683-9e41-35e8dc429797 +md""" +The optimized distribution parameter $θ_g^*$ is estimated iteratively via the algorithm underneath which runs `max_iter` times and is used here for optimizing multivariate time series distributions. + +Suppose we have a timeseries `X = {x₁, ..., xₙ}` where each `xᵢ` is a vector of dimension `m`. The `cross_entropy_method` function can handle two different scenarios: + +1. The time series is sampled IID from a single distribution `p`: `xᵢ ~ p(x)`. In this case, the distribution is represented as a `Dict{Symbol, Tuple{Sampleable, Int64}}`. The dictionary will contain `m` symbols, one for each variable in the series. The `Sampleable` object represents `p` and the integer is the length of the timeseries (`N`). +2. The time series is sampled from a different distribution at each timestep `pᵢ`: `xᵢ ~ pᵢ(x)`. In this case, the distribution is also represented as a `Dict{Symbol, Tuple{Sampleable, Int64}}`. + +The parameters $θ_{iteration'}$ are defined based on the parameter $θ_{iteration}$. The threshold $γ_{iteration}$ becomes smaller than its initial value, artificially making events less rare under $$X$$ ~ $$g$$($$x$$|$θ_{ iteration}$). +""" + +# ╔═╡ cb36a92a-0888-4993-ba11-ebbc7b05a3f9 +md""" +The `cross_entropy_method` function takes the following input: +- `loss`: The loss or objective function. +- `d_in`: The starting sampling distribution. +- `max_iter`: Maximum number of iterations. +- `N`: Population size (number of samples). +- `elite_thresh`: The threshold below which a sample will be considered elite. To have a fixed number of elite samples set this to `-Inf` and use the `min_elite_samples`. +- `min_elite_samples`: The minimum number of elite samples. +- `max_elite_samples`: The maximum number of allowed elite samples. +- `weight_fn`: A function that specifies the weight of each sample. Use the likelihood ratio when trying to perform importance sampling. +- `rng::AbstractRNG`: The random number generator used. +- `verbose`: Whether or not to print progress. +- `show_progress`: Whether or not to show the progress meter. +- `batched`: Indicates batched loss evaluation (loss function must return an array containing loss values for each sample). +- `add_entropy`: A function that transforms the sampling distribution after fitting. Use it to enforce a maximum level of entropy if converging too quickly. + +And provides you with an output `d`, which is the optimized distribution. +""" + +# ╔═╡ 4115dcd8-9931-4871-bc32-5e6e08450242 +md""" +### Fun little example + +Now the CE method itself is implemented we will try it out on the importance sampling technique. + +**Importance sampling** + +Importance sampling is a general technique for estimating properties of a particular distribution, while only having samples generated from a different distribution than the distribution of interest. + +We start off with choosing a base distribution and a starting sampling distribution. Following this a random time series is sampled from the starting distribution. +""" + +# ╔═╡ d799b586-f338-4614-9799-89ad9d04e695 +md""" +The next two code chunks provide the functions for determining the loss function and determining the likelihood ratio weighting function. In the third code chunk underneath you can see the output of the weighting funtion. +""" + +# ╔═╡ d77ca772-6b3b-43fb-a3a1-459e04bc37fc +# Define a loss function that identifies the values of 3,4,5 as those of interest +function l(d, s) + v = s[:x][1] + -(v in [3,4,5]) +end + +# ╔═╡ f9a0105c-b570-46b2-93a6-af1982fd4966 +md""" +Now we can run the `cross_entropy_method` function with the determined loss function on the sampling distribution. This will return the optimized distribution and can then be plotted in the second code chunk underneath. The plot contains both the base distribution, from which we started, and the optimal distribution as the `cross_entropy_method` funtion returned. `Random.seed!(1234)` is used so the results stay the same each time the code is run. This is done because the CE method uses randomness as underlying concept and thus might provide slightly different results each time its run (sometimes even a vector with different length in comparison to the base distribution, which will provide problems in the underlying plot). +""" + +# ╔═╡ e4cf6503-a9e1-4328-bdd5-d32b4330a6a0 +md""" +### References + +Kloek, T., and van Dijk, H.K. (1978). *Bayesian Estimates of Equation System Parameters: An Application of Integration by Monte Carlo.* Econometrica, 46(1), 1-19. + +Moss, R.J. (2020). *Cross-Entropy Variants for Optimization.* + +Rubinstein, R.Y., and Kroese, D.P. (2004). *The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation, and Machine Learning.* Springer-Verlag, New York. + +Wikipedia (2021). *Combinatorial optimization.* Checked on the 31 January 2022 via https://en.wikipedia.org/wiki/Combinatorial_optimization +""" + +# ╔═╡ 04ef2558-ff4b-4a2d-9f59-2f8b6edde3f0 +md""" +# Appendices + +Underneath you can find all functions this notebook relies on. Please leave these as they are ☺. +""" + +# ╔═╡ abb2b875-0439-49f8-accd-5475ae388a67 +function Distributions.logpdf(d::Dict{Symbol, Vector{Sampleable}}, x, i) + sum([logpdf(d[k][i], x[k][i]) for k in keys(d)]) +end + +# ╔═╡ dbcf3ddc-f4ad-40ed-8c0f-3d7ace8b5069 +function Distributions.logpdf(d::Dict{Symbol, Vector{Sampleable}}, x) + sum([logpdf(d, x, i) for i=1:length(first(x)[2])]) +end + +# ╔═╡ aeb1f33e-1b4b-4808-bcee-79e8c5e4438f +function Distributions.logpdf(d::Dict{Symbol, Tuple{Sampleable, Int64}}, x, i) + sum([logpdf(d[k][1], x[k][i]) for k in keys(d)]) +end + +# ╔═╡ fcb9f3ce-c23a-41ee-9ad2-eaea4699d5bd +function Distributions.logpdf(d::Dict{Symbol, Tuple{Sampleable, Int64}}, x) + sum([logpdf(d, x, i) for i=1:length(first(x)[2])]) +end + +# ╔═╡ c81f3bd0-b576-467c-8621-21e75fa587a5 +function Base.rand(rng::AbstractRNG, d::Dict{Symbol, Vector{Sampleable}}) + Dict(k => rand.(Ref(rng), d[k]) for k in keys(d)) +end + +# ╔═╡ 452fb99f-349e-45b0-bdf8-ada524b6d0db +function Base.rand(rng::AbstractRNG, d::Dict{Symbol, Tuple{Sampleable, Int64}}) + Dict(k => rand(rng, d[k][1], d[k][2]) for k in keys(d)) +end + +# ╔═╡ 05cf58ae-b9f7-4f69-b7e7-64586f1e27a8 +function Base.rand(rng::AbstractRNG, d::Dict{Symbol, Vector{Sampleable}}, N::Int) + [rand(rng, d) for i=1:N] +end + +# ╔═╡ 88ea50a3-02bd-4a90-8d72-cf571273ba7f +function Base.rand(rng::AbstractRNG, d::Dict{Symbol, Tuple{Sampleable, Int64}}, N::Int) + [rand(rng, d) for i=1:N] +end + +# ╔═╡ b0054103-d45f-401c-ba20-7b36729fc53e +begin + # Actual distibution + base_dist = Categorical([0.74899, 0.2, 0.05, 0.001, 0.00001]) + + # Starting sampling distribution + is_dist_0 = Dict{Symbol, Vector{Sampleable}}(:x => [Categorical(5)]) + + # Sample a random time series from the starting distribution + rand(is_dist_0) +end + +# ╔═╡ 7427a6ac-44f1-4fe6-8d0d-e16088391764 +# Define the likelihood ratio weighting function +function w(d, s) + v = s[:x][1] + exp(logpdf(base_dist, v) - logpdf(d, s)) +end + +# ╔═╡ 0160a7ed-7418-4061-bb80-cac0d93c0ac8 +# Show what the weighting function does +begin + s = Dict(:x => [5]) + w(is_dist_0, s) +end + +# ╔═╡ 028f3359-5335-4a13-9b3d-f23084087b8e +function Distributions.fit(d::Dict{Symbol, Vector{Sampleable}}, samples, weights; add_entropy = (x) -> x) + N = length(samples) + new_d = Dict{Symbol, Vector{Sampleable}}() + for s in keys(d) + dtype = typeof(d[s][1]) + m = length(d[s]) + new_d[s] = [add_entropy(fit(dtype, [samples[j][s][i] for j=1:N], weights)) for i=1:m] + end + new_d +end + +# ╔═╡ 199c2691-aeb2-4d0a-b30e-2dd130493038 +function Distributions.fit(d::Dict{Symbol, Tuple{Sampleable, Int64}}, samples, weights; add_entropy = (x)->x) + N = length(samples) + new_d = Dict{Symbol, Tuple{Sampleable, Int64}}() + for s in keys(d) + dtype = typeof(d[s][1]) + m = d[s][2] + all_samples = vcat([samples[j][s][:] for j=1:N]...) + all_weights = vcat([fill(weights[j], length(samples[j][s][:])) for j=1:N]...) + new_d[s] = (add_entropy(fit(dtype, all_samples, all_weights)), m) + end + new_d +end + +# ╔═╡ 6259dd44-2c35-4faf-b340-c69070e7fc48 +# This version uses a vector of distributions for sampling +# N is the number of samples taken +# m is the length of the vector + +# If batched is set to true, loss function must return an array containing loss values for each sample +function cross_entropy_method(loss, + d_in; + max_iter, + N=100, + elite_thresh = -0.99, + min_elite_samples = floor(Int, 0.1*N), + max_elite_samples = typemax(Int64), + weight_fn = (d,x) -> 1., + rng::AbstractRNG = Random.GLOBAL_RNG, + verbose = false, + show_progress = false, + batched = false, + add_entropy = (x)->x + ) + d = deepcopy(d_in) + show_progress ? progress = Progress(max_iter) : nothing + + for iteration in 1:max_iter + # Get samples -> Nxm + samples = rand(rng, d, N) + + # Sort the samples by loss and select elite number + if batched + losses = loss(d,samples) + else + losses = [loss(d, s) for s in samples] + end + + order = sortperm(losses) + losses = losses[order] + N_elite = losses[end] < elite_thresh ? N : findfirst(losses .> elite_thresh) - 1 + N_elite = min(max(N_elite, min_elite_samples), max_elite_samples) + + verbose && println("iteration ", iteration, " of ", max_iter, " N_elite: ", N_elite) + + # Update based on elite samples + elite_samples = samples[order[1:N_elite]] + weights = [weight_fn(d, s) for s in elite_samples] + if all(abs.(weights) .< 1e8) + println("Warning: all weights are zero") + end + d = fit(d, elite_samples, weights, add_entropy = add_entropy) + show_progress && next!(progress) + end + d +end + +# ╔═╡ 06af1708-5e0a-4c5c-a8a5-6d3ba97f2690 +begin + Random.seed!(1234) + + # Run the optimization + is_dist_opt = cross_entropy_method(l, is_dist_0; max_iter = 4, N=10000, weight_fn = w, verbose = true) + + # See the final distribution + is_dist_opt[:x][1].p +end + +# ╔═╡ 0902be86-d083-4c30-815b-f1aed4e92102 +# Plot the base distribution and the optimized distribution +begin + using Plots + x = 1:5 + y = [0.74899, 0.2, 0.05, 0.001, 0.00001] # Base distribution + plot(x, y, title = "Importance sampling", label = "Base distribution") + + z = is_dist_opt[:x][1].p # Optimized distribution + plot!(x, z, label = "Optimal distribution") +end + +# ╔═╡ d60f308c-72c0-4eff-8b21-23ef6803be8d +# Gets a function that adds entropy h to categorical distribution c +function add_categorical_entropy(h::Vector{Float64}) + function add_entropy(c::Categorical) + p = h + p[1:length(c.p)] .+= c.p + Categorical(p ./ sum(p)) + end +end + +# ╔═╡ 00000000-0000-0000-0000-000000000001 +PLUTO_PROJECT_TOML_CONTENTS = """ +[deps] +Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f" +Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80" +Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c" + +[compat] +Distributions = "~0.25.37" +Plots = "~1.25.7" +""" + +# ╔═╡ 00000000-0000-0000-0000-000000000002 +PLUTO_MANIFEST_TOML_CONTENTS = """ +# This file is machine-generated - editing it directly is not advised + +[[Adapt]] +deps = ["LinearAlgebra"] +git-tree-sha1 = "af92965fb30777147966f58acb05da51c5616b5f" +uuid = "79e6a3ab-5dfb-504d-930d-738a2a938a0e" +version = "3.3.3" + +[[ArgTools]] +uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f" + +[[Artifacts]] +uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33" + +[[Base64]] +uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f" + +[[Bzip2_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "19a35467a82e236ff51bc17a3a44b69ef35185a2" +uuid = "6e34b625-4abd-537c-b88f-471c36dfa7a0" +version = "1.0.8+0" + +[[Cairo_jll]] +deps = ["Artifacts", "Bzip2_jll", "Fontconfig_jll", "FreeType2_jll", "Glib_jll", "JLLWrappers", "LZO_jll", "Libdl", "Pixman_jll", "Pkg", "Xorg_libXext_jll", "Xorg_libXrender_jll", "Zlib_jll", "libpng_jll"] +git-tree-sha1 = "4b859a208b2397a7a623a03449e4636bdb17bcf2" +uuid = "83423d85-b0ee-5818-9007-b63ccbeb887a" +version = "1.16.1+1" + +[[ChainRulesCore]] +deps = ["Compat", "LinearAlgebra", "SparseArrays"] +git-tree-sha1 = "d711603452231bad418bd5e0c91f1abd650cba71" +uuid = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4" +version = "1.11.3" + +[[ChangesOfVariables]] +deps = ["ChainRulesCore", "LinearAlgebra", "Test"] +git-tree-sha1 = "bf98fa45a0a4cee295de98d4c1462be26345b9a1" +uuid = "9e997f8a-9a97-42d5-a9f1-ce6bfc15e2c0" +version = "0.1.2" + +[[ColorSchemes]] +deps = ["ColorTypes", "Colors", "FixedPointNumbers", "Random"] +git-tree-sha1 = "6b6f04f93710c71550ec7e16b650c1b9a612d0b6" +uuid = "35d6a980-a343-548e-a6ea-1d62b119f2f4" +version = "3.16.0" + +[[ColorTypes]] +deps = ["FixedPointNumbers", "Random"] +git-tree-sha1 = "024fe24d83e4a5bf5fc80501a314ce0d1aa35597" +uuid = "3da002f7-5984-5a60-b8a6-cbb66c0b333f" +version = "0.11.0" + +[[Colors]] +deps = ["ColorTypes", "FixedPointNumbers", "Reexport"] +git-tree-sha1 = "417b0ed7b8b838aa6ca0a87aadf1bb9eb111ce40" +uuid = "5ae59095-9a9b-59fe-a467-6f913c188581" +version = "0.12.8" + +[[Compat]] +deps = ["Base64", "Dates", "DelimitedFiles", "Distributed", "InteractiveUtils", "LibGit2", "Libdl", "LinearAlgebra", "Markdown", "Mmap", "Pkg", "Printf", "REPL", "Random", "SHA", "Serialization", "SharedArrays", "Sockets", "SparseArrays", "Statistics", "Test", "UUIDs", "Unicode"] +git-tree-sha1 = "44c37b4636bc54afac5c574d2d02b625349d6582" +uuid = "34da2185-b29b-5c13-b0c7-acf172513d20" +version = "3.41.0" + +[[CompilerSupportLibraries_jll]] +deps = ["Artifacts", "Libdl"] +uuid = "e66e0078-7015-5450-92f7-15fbd957f2ae" + +[[Contour]] +deps = ["StaticArrays"] +git-tree-sha1 = "9f02045d934dc030edad45944ea80dbd1f0ebea7" +uuid = "d38c429a-6771-53c6-b99e-75d170b6e991" +version = "0.5.7" + +[[DataAPI]] +git-tree-sha1 = "cc70b17275652eb47bc9e5f81635981f13cea5c8" +uuid = "9a962f9c-6df0-11e9-0e5d-c546b8b5ee8a" +version = "1.9.0" + +[[DataStructures]] +deps = ["Compat", "InteractiveUtils", "OrderedCollections"] +git-tree-sha1 = "3daef5523dd2e769dad2365274f760ff5f282c7d" +uuid = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8" +version = "0.18.11" + +[[DataValueInterfaces]] +git-tree-sha1 = "bfc1187b79289637fa0ef6d4436ebdfe6905cbd6" +uuid = "e2d170a0-9d28-54be-80f0-106bbe20a464" +version = "1.0.0" + +[[Dates]] +deps = ["Printf"] +uuid = "ade2ca70-3891-5945-98fb-dc099432e06a" + +[[DelimitedFiles]] +deps = ["Mmap"] +uuid = "8bb1440f-4735-579b-a4ab-409b98df4dab" + +[[DensityInterface]] +deps = ["InverseFunctions", "Test"] +git-tree-sha1 = "80c3e8639e3353e5d2912fb3a1916b8455e2494b" +uuid = "b429d917-457f-4dbc-8f4c-0cc954292b1d" +version = "0.4.0" + +[[Distributed]] +deps = ["Random", "Serialization", "Sockets"] +uuid = "8ba89e20-285c-5b6f-9357-94700520ee1b" + +[[Distributions]] +deps = ["ChainRulesCore", "DensityInterface", "FillArrays", "LinearAlgebra", "PDMats", "Printf", "QuadGK", "Random", "SparseArrays", "SpecialFunctions", "Statistics", "StatsBase", "StatsFuns", "Test"] +git-tree-sha1 = "6a8dc9f82e5ce28279b6e3e2cea9421154f5bd0d" +uuid = "31c24e10-a181-5473-b8eb-7969acd0382f" +version = "0.25.37" + +[[DocStringExtensions]] +deps = ["LibGit2"] +git-tree-sha1 = "b19534d1895d702889b219c382a6e18010797f0b" +uuid = "ffbed154-4ef7-542d-bbb7-c09d3a79fcae" +version = "0.8.6" + +[[Downloads]] +deps = ["ArgTools", "LibCURL", "NetworkOptions"] +uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6" + +[[EarCut_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "3f3a2501fa7236e9b911e0f7a588c657e822bb6d" +uuid = "5ae413db-bbd1-5e63-b57d-d24a61df00f5" +version = "2.2.3+0" + +[[Expat_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "b3bfd02e98aedfa5cf885665493c5598c350cd2f" +uuid = "2e619515-83b5-522b-bb60-26c02a35a201" +version = "2.2.10+0" + +[[FFMPEG]] +deps = ["FFMPEG_jll"] +git-tree-sha1 = "b57e3acbe22f8484b4b5ff66a7499717fe1a9cc8" +uuid = "c87230d0-a227-11e9-1b43-d7ebe4e7570a" +version = "0.4.1" + +[[FFMPEG_jll]] +deps = ["Artifacts", "Bzip2_jll", "FreeType2_jll", "FriBidi_jll", "JLLWrappers", "LAME_jll", "Libdl", "Ogg_jll", "OpenSSL_jll", "Opus_jll", "Pkg", "Zlib_jll", "libass_jll", "libfdk_aac_jll", "libvorbis_jll", "x264_jll", "x265_jll"] +git-tree-sha1 = "d8a578692e3077ac998b50c0217dfd67f21d1e5f" +uuid = "b22a6f82-2f65-5046-a5b2-351ab43fb4e5" +version = "4.4.0+0" + +[[FillArrays]] +deps = ["LinearAlgebra", "Random", "SparseArrays", "Statistics"] +git-tree-sha1 = "8756f9935b7ccc9064c6eef0bff0ad643df733a3" +uuid = "1a297f60-69ca-5386-bcde-b61e274b549b" +version = "0.12.7" + +[[FixedPointNumbers]] +deps = ["Statistics"] +git-tree-sha1 = "335bfdceacc84c5cdf16aadc768aa5ddfc5383cc" +uuid = "53c48c17-4a7d-5ca2-90c5-79b7896eea93" +version = "0.8.4" + +[[Fontconfig_jll]] +deps = ["Artifacts", "Bzip2_jll", "Expat_jll", "FreeType2_jll", "JLLWrappers", "Libdl", "Libuuid_jll", "Pkg", "Zlib_jll"] +git-tree-sha1 = "21efd19106a55620a188615da6d3d06cd7f6ee03" +uuid = "a3f928ae-7b40-5064-980b-68af3947d34b" +version = "2.13.93+0" + +[[Formatting]] +deps = ["Printf"] +git-tree-sha1 = "8339d61043228fdd3eb658d86c926cb282ae72a8" +uuid = "59287772-0a20-5a39-b81b-1366585eb4c0" +version = "0.4.2" + +[[FreeType2_jll]] +deps = ["Artifacts", "Bzip2_jll", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"] +git-tree-sha1 = "87eb71354d8ec1a96d4a7636bd57a7347dde3ef9" +uuid = "d7e528f0-a631-5988-bf34-fe36492bcfd7" +version = "2.10.4+0" + +[[FriBidi_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "aa31987c2ba8704e23c6c8ba8a4f769d5d7e4f91" +uuid = "559328eb-81f9-559d-9380-de523a88c83c" +version = "1.0.10+0" + +[[GLFW_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Libglvnd_jll", "Pkg", "Xorg_libXcursor_jll", "Xorg_libXi_jll", "Xorg_libXinerama_jll", "Xorg_libXrandr_jll"] +git-tree-sha1 = "0c603255764a1fa0b61752d2bec14cfbd18f7fe8" +uuid = "0656b61e-2033-5cc2-a64a-77c0f6c09b89" +version = "3.3.5+1" + +[[GR]] +deps = ["Base64", "DelimitedFiles", "GR_jll", "HTTP", "JSON", "Libdl", "LinearAlgebra", "Pkg", "Printf", "Random", "RelocatableFolders", "Serialization", "Sockets", "Test", "UUIDs"] +git-tree-sha1 = "4a740db447aae0fbeb3ee730de1afbb14ac798a1" +uuid = "28b8d3ca-fb5f-59d9-8090-bfdbd6d07a71" +version = "0.63.1" + +[[GR_jll]] +deps = ["Artifacts", "Bzip2_jll", "Cairo_jll", "FFMPEG_jll", "Fontconfig_jll", "GLFW_jll", "JLLWrappers", "JpegTurbo_jll", "Libdl", "Libtiff_jll", "Pixman_jll", "Pkg", "Qt5Base_jll", "Zlib_jll", "libpng_jll"] +git-tree-sha1 = "aa22e1ee9e722f1da183eb33370df4c1aeb6c2cd" +uuid = "d2c73de3-f751-5644-a686-071e5b155ba9" +version = "0.63.1+0" + +[[GeometryBasics]] +deps = ["EarCut_jll", "IterTools", "LinearAlgebra", "StaticArrays", "StructArrays", "Tables"] +git-tree-sha1 = "58bcdf5ebc057b085e58d95c138725628dd7453c" +uuid = "5c1252a2-5f33-56bf-86c9-59e7332b4326" +version = "0.4.1" + +[[Gettext_jll]] +deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "Libiconv_jll", "Pkg", "XML2_jll"] +git-tree-sha1 = "9b02998aba7bf074d14de89f9d37ca24a1a0b046" +uuid = "78b55507-aeef-58d4-861c-77aaff3498b1" +version = "0.21.0+0" + +[[Glib_jll]] +deps = ["Artifacts", "Gettext_jll", "JLLWrappers", "Libdl", "Libffi_jll", "Libiconv_jll", "Libmount_jll", "PCRE_jll", "Pkg", "Zlib_jll"] +git-tree-sha1 = "a32d672ac2c967f3deb8a81d828afc739c838a06" +uuid = "7746bdde-850d-59dc-9ae8-88ece973131d" +version = "2.68.3+2" + +[[Graphite2_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "344bf40dcab1073aca04aa0df4fb092f920e4011" +uuid = "3b182d85-2403-5c21-9c21-1e1f0cc25472" +version = "1.3.14+0" + +[[Grisu]] +git-tree-sha1 = "53bb909d1151e57e2484c3d1b53e19552b887fb2" +uuid = "42e2da0e-8278-4e71-bc24-59509adca0fe" +version = "1.0.2" + +[[HTTP]] +deps = ["Base64", "Dates", "IniFile", "Logging", "MbedTLS", "NetworkOptions", "Sockets", "URIs"] +git-tree-sha1 = "0fa77022fe4b511826b39c894c90daf5fce3334a" +uuid = "cd3eb016-35fb-5094-929b-558a96fad6f3" +version = "0.9.17" + +[[HarfBuzz_jll]] +deps = ["Artifacts", "Cairo_jll", "Fontconfig_jll", "FreeType2_jll", "Glib_jll", "Graphite2_jll", "JLLWrappers", "Libdl", "Libffi_jll", "Pkg"] +git-tree-sha1 = "129acf094d168394e80ee1dc4bc06ec835e510a3" +uuid = "2e76f6c2-a576-52d4-95c1-20adfe4de566" +version = "2.8.1+1" + +[[IniFile]] +deps = ["Test"] +git-tree-sha1 = "098e4d2c533924c921f9f9847274f2ad89e018b8" +uuid = "83e8ac13-25f8-5344-8a64-a9f2b223428f" +version = "0.5.0" + +[[InteractiveUtils]] +deps = ["Markdown"] +uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240" + +[[InverseFunctions]] +deps = ["Test"] +git-tree-sha1 = "a7254c0acd8e62f1ac75ad24d5db43f5f19f3c65" +uuid = "3587e190-3f89-42d0-90ee-14403ec27112" +version = "0.1.2" + +[[IrrationalConstants]] +git-tree-sha1 = "7fd44fd4ff43fc60815f8e764c0f352b83c49151" +uuid = "92d709cd-6900-40b7-9082-c6be49f344b6" +version = "0.1.1" + +[[IterTools]] +git-tree-sha1 = "fa6287a4469f5e048d763df38279ee729fbd44e5" +uuid = "c8e1da08-722c-5040-9ed9-7db0dc04731e" +version = "1.4.0" + +[[IteratorInterfaceExtensions]] +git-tree-sha1 = "a3f24677c21f5bbe9d2a714f95dcd58337fb2856" +uuid = "82899510-4779-5014-852e-03e436cf321d" +version = "1.0.0" + +[[JLLWrappers]] +deps = ["Preferences"] +git-tree-sha1 = "642a199af8b68253517b80bd3bfd17eb4e84df6e" +uuid = "692b3bcd-3c85-4b1f-b108-f13ce0eb3210" +version = "1.3.0" + +[[JSON]] +deps = ["Dates", "Mmap", "Parsers", "Unicode"] +git-tree-sha1 = "8076680b162ada2a031f707ac7b4953e30667a37" +uuid = "682c06a0-de6a-54ab-a142-c8b1cf79cde6" +version = "0.21.2" + +[[JpegTurbo_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "d735490ac75c5cb9f1b00d8b5509c11984dc6943" +uuid = "aacddb02-875f-59d6-b918-886e6ef4fbf8" +version = "2.1.0+0" + +[[LAME_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "f6250b16881adf048549549fba48b1161acdac8c" +uuid = "c1c5ebd0-6772-5130-a774-d5fcae4a789d" +version = "3.100.1+0" + +[[LZO_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "e5b909bcf985c5e2605737d2ce278ed791b89be6" +uuid = "dd4b983a-f0e5-5f8d-a1b7-129d4a5fb1ac" +version = "2.10.1+0" + +[[LaTeXStrings]] +git-tree-sha1 = "f2355693d6778a178ade15952b7ac47a4ff97996" +uuid = "b964fa9f-0449-5b57-a5c2-d3ea65f4040f" +version = "1.3.0" + +[[Latexify]] +deps = ["Formatting", "InteractiveUtils", "LaTeXStrings", "MacroTools", "Markdown", "Printf", "Requires"] +git-tree-sha1 = "a8f4f279b6fa3c3c4f1adadd78a621b13a506bce" +uuid = "23fbe1c1-3f47-55db-b15f-69d7ec21a316" +version = "0.15.9" + +[[LibCURL]] +deps = ["LibCURL_jll", "MozillaCACerts_jll"] +uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21" + +[[LibCURL_jll]] +deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"] +uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0" + +[[LibGit2]] +deps = ["Base64", "NetworkOptions", "Printf", "SHA"] +uuid = "76f85450-5226-5b5a-8eaa-529ad045b433" + +[[LibSSH2_jll]] +deps = ["Artifacts", "Libdl", "MbedTLS_jll"] +uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8" + +[[Libdl]] +uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb" + +[[Libffi_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "0b4a5d71f3e5200a7dff793393e09dfc2d874290" +uuid = "e9f186c6-92d2-5b65-8a66-fee21dc1b490" +version = "3.2.2+1" + +[[Libgcrypt_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Libgpg_error_jll", "Pkg"] +git-tree-sha1 = "64613c82a59c120435c067c2b809fc61cf5166ae" +uuid = "d4300ac3-e22c-5743-9152-c294e39db1e4" +version = "1.8.7+0" + +[[Libglvnd_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll", "Xorg_libXext_jll"] +git-tree-sha1 = "7739f837d6447403596a75d19ed01fd08d6f56bf" +uuid = "7e76a0d4-f3c7-5321-8279-8d96eeed0f29" +version = "1.3.0+3" + +[[Libgpg_error_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "c333716e46366857753e273ce6a69ee0945a6db9" +uuid = "7add5ba3-2f88-524e-9cd5-f83b8a55f7b8" +version = "1.42.0+0" + +[[Libiconv_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "42b62845d70a619f063a7da093d995ec8e15e778" +uuid = "94ce4f54-9a6c-5748-9c1c-f9c7231a4531" +version = "1.16.1+1" + +[[Libmount_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "9c30530bf0effd46e15e0fdcf2b8636e78cbbd73" +uuid = "4b2f31a3-9ecc-558c-b454-b3730dcb73e9" +version = "2.35.0+0" + +[[Libtiff_jll]] +deps = ["Artifacts", "JLLWrappers", "JpegTurbo_jll", "Libdl", "Pkg", "Zlib_jll", "Zstd_jll"] +git-tree-sha1 = "340e257aada13f95f98ee352d316c3bed37c8ab9" +uuid = "89763e89-9b03-5906-acba-b20f662cd828" +version = "4.3.0+0" + +[[Libuuid_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "7f3efec06033682db852f8b3bc3c1d2b0a0ab066" +uuid = "38a345b3-de98-5d2b-a5d3-14cd9215e700" +version = "2.36.0+0" + +[[LinearAlgebra]] +deps = ["Libdl"] +uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e" + +[[LogExpFunctions]] +deps = ["ChainRulesCore", "ChangesOfVariables", "DocStringExtensions", "InverseFunctions", "IrrationalConstants", "LinearAlgebra"] +git-tree-sha1 = "e5718a00af0ab9756305a0392832c8952c7426c1" +uuid = "2ab3a3ac-af41-5b50-aa03-7779005ae688" +version = "0.3.6" + +[[Logging]] +uuid = "56ddb016-857b-54e1-b83d-db4d58db5568" + +[[MacroTools]] +deps = ["Markdown", "Random"] +git-tree-sha1 = "3d3e902b31198a27340d0bf00d6ac452866021cf" +uuid = "1914dd2f-81c6-5fcd-8719-6d5c9610ff09" +version = "0.5.9" + +[[Markdown]] +deps = ["Base64"] +uuid = "d6f4376e-aef5-505a-96c1-9c027394607a" + +[[MbedTLS]] +deps = ["Dates", "MbedTLS_jll", "Random", "Sockets"] +git-tree-sha1 = "1c38e51c3d08ef2278062ebceade0e46cefc96fe" +uuid = "739be429-bea8-5141-9913-cc70e7f3736d" +version = "1.0.3" + +[[MbedTLS_jll]] +deps = ["Artifacts", "Libdl"] +uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1" + +[[Measures]] +git-tree-sha1 = "e498ddeee6f9fdb4551ce855a46f54dbd900245f" +uuid = "442fdcdd-2543-5da2-b0f3-8c86c306513e" +version = "0.3.1" + +[[Missings]] +deps = ["DataAPI"] +git-tree-sha1 = "bf210ce90b6c9eed32d25dbcae1ebc565df2687f" +uuid = "e1d29d7a-bbdc-5cf2-9ac0-f12de2c33e28" +version = "1.0.2" + +[[Mmap]] +uuid = "a63ad114-7e13-5084-954f-fe012c677804" + +[[MozillaCACerts_jll]] +uuid = "14a3606d-f60d-562e-9121-12d972cd8159" + +[[NaNMath]] +git-tree-sha1 = "f755f36b19a5116bb580de457cda0c140153f283" +uuid = "77ba4419-2d1f-58cd-9bb1-8ffee604a2e3" +version = "0.3.6" + +[[NetworkOptions]] +uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908" + +[[Ogg_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "887579a3eb005446d514ab7aeac5d1d027658b8f" +uuid = "e7412a2a-1a6e-54c0-be00-318e2571c051" +version = "1.3.5+1" + +[[OpenLibm_jll]] +deps = ["Artifacts", "Libdl"] +uuid = "05823500-19ac-5b8b-9628-191a04bc5112" + +[[OpenSSL_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "648107615c15d4e09f7eca16307bc821c1f718d8" +uuid = "458c3c95-2e84-50aa-8efc-19380b2a3a95" +version = "1.1.13+0" + +[[OpenSpecFun_jll]] +deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "13652491f6856acfd2db29360e1bbcd4565d04f1" +uuid = "efe28fd5-8261-553b-a9e1-b2916fc3738e" +version = "0.5.5+0" + +[[Opus_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "51a08fb14ec28da2ec7a927c4337e4332c2a4720" +uuid = "91d4177d-7536-5919-b921-800302f37372" +version = "1.3.2+0" + +[[OrderedCollections]] +git-tree-sha1 = "85f8e6578bf1f9ee0d11e7bb1b1456435479d47c" +uuid = "bac558e1-5e72-5ebc-8fee-abe8a469f55d" +version = "1.4.1" + +[[PCRE_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "b2a7af664e098055a7529ad1a900ded962bca488" +uuid = "2f80f16e-611a-54ab-bc61-aa92de5b98fc" +version = "8.44.0+0" + +[[PDMats]] +deps = ["LinearAlgebra", "SparseArrays", "SuiteSparse"] +git-tree-sha1 = "ee26b350276c51697c9c2d88a072b339f9f03d73" +uuid = "90014a1f-27ba-587c-ab20-58faa44d9150" +version = "0.11.5" + +[[Parsers]] +deps = ["Dates"] +git-tree-sha1 = "92f91ba9e5941fc781fecf5494ac1da87bdac775" +uuid = "69de0a69-1ddd-5017-9359-2bf0b02dc9f0" +version = "2.2.0" + +[[Pixman_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "b4f5d02549a10e20780a24fce72bea96b6329e29" +uuid = "30392449-352a-5448-841d-b1acce4e97dc" +version = "0.40.1+0" + +[[Pkg]] +deps = ["Artifacts", "Dates", "Downloads", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"] +uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f" + +[[PlotThemes]] +deps = ["PlotUtils", "Requires", "Statistics"] +git-tree-sha1 = "a3a964ce9dc7898193536002a6dd892b1b5a6f1d" +uuid = "ccf2f8ad-2431-5c83-bf29-c5338b663b6a" +version = "2.0.1" + +[[PlotUtils]] +deps = ["ColorSchemes", "Colors", "Dates", "Printf", "Random", "Reexport", "Statistics"] +git-tree-sha1 = "6f1b25e8ea06279b5689263cc538f51331d7ca17" +uuid = "995b91a9-d308-5afd-9ec6-746e21dbc043" +version = "1.1.3" + +[[Plots]] +deps = ["Base64", "Contour", "Dates", "Downloads", "FFMPEG", "FixedPointNumbers", "GR", "GeometryBasics", "JSON", "Latexify", "LinearAlgebra", "Measures", "NaNMath", "PlotThemes", "PlotUtils", "Printf", "REPL", "Random", "RecipesBase", "RecipesPipeline", "Reexport", "Requires", "Scratch", "Showoff", "SparseArrays", "Statistics", "StatsBase", "UUIDs", "UnicodeFun", "Unzip"] +git-tree-sha1 = "7e4920a7d4323b8ffc3db184580598450bde8a8e" +uuid = "91a5bcdd-55d7-5caf-9e0b-520d859cae80" +version = "1.25.7" + +[[Preferences]] +deps = ["TOML"] +git-tree-sha1 = "2cf929d64681236a2e074ffafb8d568733d2e6af" +uuid = "21216c6a-2e73-6563-6e65-726566657250" +version = "1.2.3" + +[[Printf]] +deps = ["Unicode"] +uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7" + +[[Qt5Base_jll]] +deps = ["Artifacts", "CompilerSupportLibraries_jll", "Fontconfig_jll", "Glib_jll", "JLLWrappers", "Libdl", "Libglvnd_jll", "OpenSSL_jll", "Pkg", "Xorg_libXext_jll", "Xorg_libxcb_jll", "Xorg_xcb_util_image_jll", "Xorg_xcb_util_keysyms_jll", "Xorg_xcb_util_renderutil_jll", "Xorg_xcb_util_wm_jll", "Zlib_jll", "xkbcommon_jll"] +git-tree-sha1 = "ad368663a5e20dbb8d6dc2fddeefe4dae0781ae8" +uuid = "ea2cea3b-5b76-57ae-a6ef-0a8af62496e1" +version = "5.15.3+0" + +[[QuadGK]] +deps = ["DataStructures", "LinearAlgebra"] +git-tree-sha1 = "78aadffb3efd2155af139781b8a8df1ef279ea39" +uuid = "1fd47b50-473d-5c70-9696-f719f8f3bcdc" +version = "2.4.2" + +[[REPL]] +deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"] +uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb" + +[[Random]] +deps = ["Serialization"] +uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c" + +[[RecipesBase]] +git-tree-sha1 = "6bf3f380ff52ce0832ddd3a2a7b9538ed1bcca7d" +uuid = "3cdcf5f2-1ef4-517c-9805-6587b60abb01" +version = "1.2.1" + +[[RecipesPipeline]] +deps = ["Dates", "NaNMath", "PlotUtils", "RecipesBase"] +git-tree-sha1 = "37c1631cb3cc36a535105e6d5557864c82cd8c2b" +uuid = "01d81517-befc-4cb6-b9ec-a95719d0359c" +version = "0.5.0" + +[[Reexport]] +git-tree-sha1 = "45e428421666073eab6f2da5c9d310d99bb12f9b" +uuid = "189a3867-3050-52da-a836-e630ba90ab69" +version = "1.2.2" + +[[RelocatableFolders]] +deps = ["SHA", "Scratch"] +git-tree-sha1 = "cdbd3b1338c72ce29d9584fdbe9e9b70eeb5adca" +uuid = "05181044-ff0b-4ac5-8273-598c1e38db00" +version = "0.1.3" + +[[Requires]] +deps = ["UUIDs"] +git-tree-sha1 = "838a3a4188e2ded87a4f9f184b4b0d78a1e91cb7" +uuid = "ae029012-a4dd-5104-9daa-d747884805df" +version = "1.3.0" + +[[Rmath]] +deps = ["Random", "Rmath_jll"] +git-tree-sha1 = "bf3188feca147ce108c76ad82c2792c57abe7b1f" +uuid = "79098fc4-a85e-5d69-aa6a-4863f24498fa" +version = "0.7.0" + +[[Rmath_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "68db32dff12bb6127bac73c209881191bf0efbb7" +uuid = "f50d1b31-88e8-58de-be2c-1cc44531875f" +version = "0.3.0+0" + +[[SHA]] +uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce" + +[[Scratch]] +deps = ["Dates"] +git-tree-sha1 = "0b4b7f1393cff97c33891da2a0bf69c6ed241fda" +uuid = "6c6a2e73-6563-6170-7368-637461726353" +version = "1.1.0" + +[[Serialization]] +uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b" + +[[SharedArrays]] +deps = ["Distributed", "Mmap", "Random", "Serialization"] +uuid = "1a1011a3-84de-559e-8e89-a11a2f7dc383" + +[[Showoff]] +deps = ["Dates", "Grisu"] +git-tree-sha1 = "91eddf657aca81df9ae6ceb20b959ae5653ad1de" +uuid = "992d4aef-0814-514b-bc4d-f2e9a6c4116f" +version = "1.0.3" + +[[Sockets]] +uuid = "6462fe0b-24de-5631-8697-dd941f90decc" + +[[SortingAlgorithms]] +deps = ["DataStructures"] +git-tree-sha1 = "b3363d7460f7d098ca0912c69b082f75625d7508" +uuid = "a2af1166-a08f-5f64-846c-94a0d3cef48c" +version = "1.0.1" + +[[SparseArrays]] +deps = ["LinearAlgebra", "Random"] +uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf" + +[[SpecialFunctions]] +deps = ["ChainRulesCore", "IrrationalConstants", "LogExpFunctions", "OpenLibm_jll", "OpenSpecFun_jll"] +git-tree-sha1 = "e08890d19787ec25029113e88c34ec20cac1c91e" +uuid = "276daf66-3868-5448-9aa4-cd146d93841b" +version = "2.0.0" + +[[StaticArrays]] +deps = ["LinearAlgebra", "Random", "Statistics"] +git-tree-sha1 = "2884859916598f974858ff01df7dfc6c708dd895" +uuid = "90137ffa-7385-5640-81b9-e52037218182" +version = "1.3.3" + +[[Statistics]] +deps = ["LinearAlgebra", "SparseArrays"] +uuid = "10745b16-79ce-11e8-11f9-7d13ad32a3b2" + +[[StatsAPI]] +git-tree-sha1 = "d88665adc9bcf45903013af0982e2fd05ae3d0a6" +uuid = "82ae8749-77ed-4fe6-ae5f-f523153014b0" +version = "1.2.0" + +[[StatsBase]] +deps = ["DataAPI", "DataStructures", "LinearAlgebra", "LogExpFunctions", "Missings", "Printf", "Random", "SortingAlgorithms", "SparseArrays", "Statistics", "StatsAPI"] +git-tree-sha1 = "51383f2d367eb3b444c961d485c565e4c0cf4ba0" +uuid = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91" +version = "0.33.14" + +[[StatsFuns]] +deps = ["ChainRulesCore", "InverseFunctions", "IrrationalConstants", "LogExpFunctions", "Reexport", "Rmath", "SpecialFunctions"] +git-tree-sha1 = "bedb3e17cc1d94ce0e6e66d3afa47157978ba404" +uuid = "4c63d2b9-4356-54db-8cca-17b64c39e42c" +version = "0.9.14" + +[[StructArrays]] +deps = ["Adapt", "DataAPI", "StaticArrays", "Tables"] +git-tree-sha1 = "d21f2c564b21a202f4677c0fba5b5ee431058544" +uuid = "09ab397b-f2b6-538f-b94a-2f83cf4a842a" +version = "0.6.4" + +[[SuiteSparse]] +deps = ["Libdl", "LinearAlgebra", "Serialization", "SparseArrays"] +uuid = "4607b0f0-06f3-5cda-b6b1-a6196a1729e9" + +[[TOML]] +deps = ["Dates"] +uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76" + +[[TableTraits]] +deps = ["IteratorInterfaceExtensions"] +git-tree-sha1 = "c06b2f539df1c6efa794486abfb6ed2022561a39" +uuid = "3783bdb8-4a98-5b6b-af9a-565f29a5fe9c" +version = "1.0.1" + +[[Tables]] +deps = ["DataAPI", "DataValueInterfaces", "IteratorInterfaceExtensions", "LinearAlgebra", "TableTraits", "Test"] +git-tree-sha1 = "bb1064c9a84c52e277f1096cf41434b675cd368b" +uuid = "bd369af6-aec1-5ad0-b16a-f7cc5008161c" +version = "1.6.1" + +[[Tar]] +deps = ["ArgTools", "SHA"] +uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e" + +[[Test]] +deps = ["InteractiveUtils", "Logging", "Random", "Serialization"] +uuid = "8dfed614-e22c-5e08-85e1-65c5234f0b40" + +[[URIs]] +git-tree-sha1 = "97bbe755a53fe859669cd907f2d96aee8d2c1355" +uuid = "5c2747f8-b7ea-4ff2-ba2e-563bfd36b1d4" +version = "1.3.0" + +[[UUIDs]] +deps = ["Random", "SHA"] +uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4" + +[[Unicode]] +uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5" + +[[UnicodeFun]] +deps = ["REPL"] +git-tree-sha1 = "53915e50200959667e78a92a418594b428dffddf" +uuid = "1cfade01-22cf-5700-b092-accc4b62d6e1" +version = "0.4.1" + +[[Unzip]] +git-tree-sha1 = "34db80951901073501137bdbc3d5a8e7bbd06670" +uuid = "41fe7b60-77ed-43a1-b4f0-825fd5a5650d" +version = "0.1.2" + +[[Wayland_jll]] +deps = ["Artifacts", "Expat_jll", "JLLWrappers", "Libdl", "Libffi_jll", "Pkg", "XML2_jll"] +git-tree-sha1 = "3e61f0b86f90dacb0bc0e73a0c5a83f6a8636e23" +uuid = "a2964d1f-97da-50d4-b82a-358c7fce9d89" +version = "1.19.0+0" + +[[Wayland_protocols_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "66d72dc6fcc86352f01676e8f0f698562e60510f" +uuid = "2381bf8a-dfd0-557d-9999-79630e7b1b91" +version = "1.23.0+0" + +[[XML2_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Libiconv_jll", "Pkg", "Zlib_jll"] +git-tree-sha1 = "1acf5bdf07aa0907e0a37d3718bb88d4b687b74a" +uuid = "02c8fc9c-b97f-50b9-bbe4-9be30ff0a78a" +version = "2.9.12+0" + +[[XSLT_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Libgcrypt_jll", "Libgpg_error_jll", "Libiconv_jll", "Pkg", "XML2_jll", "Zlib_jll"] +git-tree-sha1 = "91844873c4085240b95e795f692c4cec4d805f8a" +uuid = "aed1982a-8fda-507f-9586-7b0439959a61" +version = "1.1.34+0" + +[[Xorg_libX11_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxcb_jll", "Xorg_xtrans_jll"] +git-tree-sha1 = "5be649d550f3f4b95308bf0183b82e2582876527" +uuid = "4f6342f7-b3d2-589e-9d20-edeb45f2b2bc" +version = "1.6.9+4" + +[[Xorg_libXau_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "4e490d5c960c314f33885790ed410ff3a94ce67e" +uuid = "0c0b7dd1-d40b-584c-a123-a41640f87eec" +version = "1.0.9+4" + +[[Xorg_libXcursor_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXfixes_jll", "Xorg_libXrender_jll"] +git-tree-sha1 = "12e0eb3bc634fa2080c1c37fccf56f7c22989afd" +uuid = "935fb764-8cf2-53bf-bb30-45bb1f8bf724" +version = "1.2.0+4" + +[[Xorg_libXdmcp_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "4fe47bd2247248125c428978740e18a681372dd4" +uuid = "a3789734-cfe1-5b06-b2d0-1dd0d9d62d05" +version = "1.1.3+4" + +[[Xorg_libXext_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"] +git-tree-sha1 = "b7c0aa8c376b31e4852b360222848637f481f8c3" +uuid = "1082639a-0dae-5f34-9b06-72781eeb8cb3" +version = "1.3.4+4" + +[[Xorg_libXfixes_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"] +git-tree-sha1 = "0e0dc7431e7a0587559f9294aeec269471c991a4" +uuid = "d091e8ba-531a-589c-9de9-94069b037ed8" +version = "5.0.3+4" + +[[Xorg_libXi_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXext_jll", "Xorg_libXfixes_jll"] +git-tree-sha1 = "89b52bc2160aadc84d707093930ef0bffa641246" +uuid = "a51aa0fd-4e3c-5386-b890-e753decda492" +version = "1.7.10+4" + +[[Xorg_libXinerama_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXext_jll"] +git-tree-sha1 = "26be8b1c342929259317d8b9f7b53bf2bb73b123" +uuid = "d1454406-59df-5ea1-beac-c340f2130bc3" +version = "1.1.4+4" + +[[Xorg_libXrandr_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXext_jll", "Xorg_libXrender_jll"] +git-tree-sha1 = "34cea83cb726fb58f325887bf0612c6b3fb17631" +uuid = "ec84b674-ba8e-5d96-8ba1-2a689ba10484" +version = "1.5.2+4" + +[[Xorg_libXrender_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"] +git-tree-sha1 = "19560f30fd49f4d4efbe7002a1037f8c43d43b96" +uuid = "ea2f1a96-1ddc-540d-b46f-429655e07cfa" +version = "0.9.10+4" + +[[Xorg_libpthread_stubs_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "6783737e45d3c59a4a4c4091f5f88cdcf0908cbb" +uuid = "14d82f49-176c-5ed1-bb49-ad3f5cbd8c74" +version = "0.1.0+3" + +[[Xorg_libxcb_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "XSLT_jll", "Xorg_libXau_jll", "Xorg_libXdmcp_jll", "Xorg_libpthread_stubs_jll"] +git-tree-sha1 = "daf17f441228e7a3833846cd048892861cff16d6" +uuid = "c7cfdc94-dc32-55de-ac96-5a1b8d977c5b" +version = "1.13.0+3" + +[[Xorg_libxkbfile_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"] +git-tree-sha1 = "926af861744212db0eb001d9e40b5d16292080b2" +uuid = "cc61e674-0454-545c-8b26-ed2c68acab7a" +version = "1.1.0+4" + +[[Xorg_xcb_util_image_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"] +git-tree-sha1 = "0fab0a40349ba1cba2c1da699243396ff8e94b97" +uuid = "12413925-8142-5f55-bb0e-6d7ca50bb09b" +version = "0.4.0+1" + +[[Xorg_xcb_util_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxcb_jll"] +git-tree-sha1 = "e7fd7b2881fa2eaa72717420894d3938177862d1" +uuid = "2def613f-5ad1-5310-b15b-b15d46f528f5" +version = "0.4.0+1" + +[[Xorg_xcb_util_keysyms_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"] +git-tree-sha1 = "d1151e2c45a544f32441a567d1690e701ec89b00" +uuid = "975044d2-76e6-5fbe-bf08-97ce7c6574c7" +version = "0.4.0+1" + +[[Xorg_xcb_util_renderutil_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"] +git-tree-sha1 = "dfd7a8f38d4613b6a575253b3174dd991ca6183e" +uuid = "0d47668e-0667-5a69-a72c-f761630bfb7e" +version = "0.3.9+1" + +[[Xorg_xcb_util_wm_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"] +git-tree-sha1 = "e78d10aab01a4a154142c5006ed44fd9e8e31b67" +uuid = "c22f9ab0-d5fe-5066-847c-f4bb1cd4e361" +version = "0.4.1+1" + +[[Xorg_xkbcomp_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxkbfile_jll"] +git-tree-sha1 = "4bcbf660f6c2e714f87e960a171b119d06ee163b" +uuid = "35661453-b289-5fab-8a00-3d9160c6a3a4" +version = "1.4.2+4" + +[[Xorg_xkeyboard_config_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xkbcomp_jll"] +git-tree-sha1 = "5c8424f8a67c3f2209646d4425f3d415fee5931d" +uuid = "33bec58e-1273-512f-9401-5d533626f822" +version = "2.27.0+4" + +[[Xorg_xtrans_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "79c31e7844f6ecf779705fbc12146eb190b7d845" +uuid = "c5fb5394-a638-5e4d-96e5-b29de1b5cf10" +version = "1.4.0+3" + +[[Zlib_jll]] +deps = ["Libdl"] +uuid = "83775a58-1f1d-513f-b197-d71354ab007a" + +[[Zstd_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "cc4bf3fdde8b7e3e9fa0351bdeedba1cf3b7f6e6" +uuid = "3161d3a3-bdf6-5164-811a-617609db77b4" +version = "1.5.0+0" + +[[libass_jll]] +deps = ["Artifacts", "Bzip2_jll", "FreeType2_jll", "FriBidi_jll", "HarfBuzz_jll", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"] +git-tree-sha1 = "5982a94fcba20f02f42ace44b9894ee2b140fe47" +uuid = "0ac62f75-1d6f-5e53-bd7c-93b484bb37c0" +version = "0.15.1+0" + +[[libfdk_aac_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "daacc84a041563f965be61859a36e17c4e4fcd55" +uuid = "f638f0a6-7fb0-5443-88ba-1cc74229b280" +version = "2.0.2+0" + +[[libpng_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"] +git-tree-sha1 = "94d180a6d2b5e55e447e2d27a29ed04fe79eb30c" +uuid = "b53b4c65-9356-5827-b1ea-8c7a1a84506f" +version = "1.6.38+0" + +[[libvorbis_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Ogg_jll", "Pkg"] +git-tree-sha1 = "b910cb81ef3fe6e78bf6acee440bda86fd6ae00c" +uuid = "f27f6e37-5d2b-51aa-960f-b287f2bc3b7a" +version = "1.3.7+1" + +[[nghttp2_jll]] +deps = ["Artifacts", "Libdl"] +uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d" + +[[p7zip_jll]] +deps = ["Artifacts", "Libdl"] +uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0" + +[[x264_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "4fea590b89e6ec504593146bf8b988b2c00922b2" +uuid = "1270edf5-f2f9-52d2-97e9-ab00b5d0237a" +version = "2021.5.5+0" + +[[x265_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] +git-tree-sha1 = "ee567a171cce03570d77ad3a43e90218e38937a9" +uuid = "dfaa095f-4041-5dcd-9319-2fabd8486b76" +version = "3.5.0+0" + +[[xkbcommon_jll]] +deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Wayland_jll", "Wayland_protocols_jll", "Xorg_libxcb_jll", "Xorg_xkeyboard_config_jll"] +git-tree-sha1 = "ece2350174195bb31de1a63bea3a41ae1aa593b6" +uuid = "d8fb68d0-12a3-5cfd-a85a-d49703b185fd" +version = "0.9.1+5" +""" + +# ╔═╡ Cell order: +# ╟─f126a1f8-afc6-4b52-9021-39e946dd0ea2 +# ╟─c629d020-2c17-477c-8517-68c39f46fbcb +# ╟─2c923610-52c4-11ec-367b-2565c12740ab +# ╟─2588e834-f2ee-41d1-b2c3-dbc43a0cda63 +# ╟─9b4da26a-6136-4be7-b3d8-ad573a89c7f9 +# ╟─aba389ff-77b4-4683-9e41-35e8dc429797 +# ╟─cb36a92a-0888-4993-ba11-ebbc7b05a3f9 +# ╠═6259dd44-2c35-4faf-b340-c69070e7fc48 +# ╟─4115dcd8-9931-4871-bc32-5e6e08450242 +# ╠═b0054103-d45f-401c-ba20-7b36729fc53e +# ╟─d799b586-f338-4614-9799-89ad9d04e695 +# ╠═d77ca772-6b3b-43fb-a3a1-459e04bc37fc +# ╠═7427a6ac-44f1-4fe6-8d0d-e16088391764 +# ╠═0160a7ed-7418-4061-bb80-cac0d93c0ac8 +# ╟─f9a0105c-b570-46b2-93a6-af1982fd4966 +# ╠═06af1708-5e0a-4c5c-a8a5-6d3ba97f2690 +# ╠═0902be86-d083-4c30-815b-f1aed4e92102 +# ╟─e4cf6503-a9e1-4328-bdd5-d32b4330a6a0 +# ╟─04ef2558-ff4b-4a2d-9f59-2f8b6edde3f0 +# ╠═9c2b64cf-c236-46aa-a4d6-02a22c5b7faa +# ╠═abb2b875-0439-49f8-accd-5475ae388a67 +# ╠═dbcf3ddc-f4ad-40ed-8c0f-3d7ace8b5069 +# ╠═aeb1f33e-1b4b-4808-bcee-79e8c5e4438f +# ╠═fcb9f3ce-c23a-41ee-9ad2-eaea4699d5bd +# ╠═c5ef7103-4aca-49dc-b49b-4cf5ac157701 +# ╠═c81f3bd0-b576-467c-8621-21e75fa587a5 +# ╠═452fb99f-349e-45b0-bdf8-ada524b6d0db +# ╠═05cf58ae-b9f7-4f69-b7e7-64586f1e27a8 +# ╠═88ea50a3-02bd-4a90-8d72-cf571273ba7f +# ╠═028f3359-5335-4a13-9b3d-f23084087b8e +# ╠═199c2691-aeb2-4d0a-b30e-2dd130493038 +# ╠═d60f308c-72c0-4eff-8b21-23ef6803be8d +# ╟─00000000-0000-0000-0000-000000000001 +# ╟─00000000-0000-0000-0000-000000000002 diff --git a/notebook/example.jl b/notebook/example.jl deleted file mode 100644 index 950417f0..00000000 --- a/notebook/example.jl +++ /dev/null @@ -1,159 +0,0 @@ -### A Pluto.jl notebook ### -# v0.14.8 - -using Markdown -using InteractiveUtils - -# ╔═╡ 5ad5c202-20f8-11eb-23f1-4f38b687c285 -using STMOZOO.Example - -# ╔═╡ d2d007b8-20f8-11eb-0ddd-1181d4565a85 -using Plots - -# ╔═╡ 45189a82-20fa-11eb-0423-05ce1b84639d -using Zygote - -# ╔═╡ 171fee18-20f6-11eb-37e5-2d04caea8c35 -md""" -# Example: solving quadratic systems - -By Michiel Stock - -## Introduction - -In this notebook, I have chosen to write code so minimize *quadratic functions* of the form: - -$$f(\mathbf{x}) = \frac{1}{2} \mathbf{x}^\intercal P\mathbf{x} + \mathbf{q} \cdot \mathbf{x} + r\,,$$ - -where $P \succ 0$. - -Hence, solving - -$$\min_\mathbf{x}\, f(\mathbf{x})\,.$$ -""" - -# ╔═╡ fb4aeb8c-20f7-11eb-0444-259de7b76883 -md""" - -## Derivation - -The gradient of the quadratic function is - -$$\nabla f(\mathbf{x})=P\mathbf{x} +\mathbf{q}\,.$$ - -Setting this to zero gives - -$$\mathbf{x}^\star=-P^{-1}\mathbf{q}\,.$$ - -We also know that $\nabla^2f(\mathbf{x}) = P \succ 0$, so $\mathbf{x}^\star$ is a minimizer. -""" - -# ╔═╡ 52e87238-20f8-11eb-2ea0-27ee1208d3c3 -md""" -## Illustration - -We will use a simple 2D example to illustrate this code. -""" - -# ╔═╡ e9a9d69e-20f8-11eb-0d9d-330ee5e9cf25 -P = [7 -1; -1 2] - -# ╔═╡ fdd4e550-20f8-11eb-227b-25f36708484d -q = [-14.0, -2.0] - -# ╔═╡ 025fd6e8-20f9-11eb-3e7d-3519f3c4b58f -r = 1 - -# ╔═╡ 096eff98-20f9-11eb-1e61-99d5714895ba -md"We have a function to generate the quadratic function." - -# ╔═╡ 165509ca-20f9-11eb-107c-550cbba0f0e9 -f_quadr = quadratic_function(P, q, r) - -# ╔═╡ 1fffc82a-20f9-11eb-198c-c160d7dac87d -f_quadr([2, 1]) - -# ╔═╡ 26ab6ce2-20f9-11eb-1836-1756b290e5e3 -md"No more need to remember the formulla for the minimizer! Just use `solve_quadratic_system`!" - -# ╔═╡ 49832a8e-20f9-11eb-0841-19a40a12db18 -x_star = solve_quadratic_system(P, q, r) - -# ╔═╡ 55e0e274-20f9-11eb-36c0-753f228f7e9b -begin - contourf(-20:0.1:20, -20:0.1:20, (x, y) -> f_quadr([x,y]), color=:speed) - scatter!([x_star[1]], [x_star[2]], label="minimizer") -end - -# ╔═╡ b1551758-20f9-11eb-3e8f-ff9a7127d7f8 -md""" -## Approximating non-quadratic functions - -We can approximate non-quadratic functions by a quadratic function: The second order Taylor approximation $\hat{f}$ of a function $f$ at $\mathbf{x}$ is -$$f(\mathbf{x}+\mathbf{v})\approx\hat{f}(\mathbf{x}+\mathbf{v}) = f(\mathbf{x}) + \nabla f(\mathbf{x})^\top \mathbf{v} + \frac{1}{2} \mathbf{v}^\top \nabla^2 f(\mathbf{x}) \mathbf{v}\,.$$ - -Let us use this idea for the Rosenbrock function. -""" - -# ╔═╡ 41d8f1dc-20fa-11eb-3586-a989427c1fd6 -f_nonquadr((x1, x2); a=1, b=5) = (a-x1)^2 + b * (x2 - x1^2)^2 - -# ╔═╡ 4ed4215e-20fa-11eb-11ee-f7741591163c -x = [0.0, 0.0] - -# ╔═╡ 56af99ee-20fa-11eb-0240-69c675efb78c -fx = f_nonquadr(x) - -# ╔═╡ 6c5473b4-20fa-11eb-327b-51ac560530eb -∇fx = f_nonquadr'(x) - -# ╔═╡ 7518c2c0-20fa-11eb-32c0-a9db2a91cbc5 -∇²fx = Zygote.hessian(f_nonquadr, x) - -# ╔═╡ 34027942-20fb-11eb-261e-3b991ce4c9f8 -v = solve_quadratic_system(∇²fx, ∇fx, fx) - -# ╔═╡ 3bbeb85c-20fc-11eb-04d0-fb12d8ace50a -f̂(x′) = quadratic_function(∇²fx, ∇fx, fx)(x′ .- x) - -# ╔═╡ 8623ac1a-20fa-11eb-2d45-49cce0fdac86 -begin - plot_nonquadr = contourf(-2:0.01:2, -2:0.01:2, (x, y) -> f_nonquadr([x,y]), color=:speed, title="non-quadratic function") - scatter!(plot_nonquadr, [x[1]], [x[2]], label="x") - scatter!(plot_nonquadr, [x[1]+v[1]], [x[2]+v[2]], label="x + v") - - plot_approx = contourf(-2:0.01:2, -2:0.01:2, (x, y) -> f̂([x,y]), color=:speed, - title="quadratic approximation") - scatter!(plot_approx, [x[1]], [x[2]], label="x") - scatter!(plot_approx, [x[1]+v[1]], [x[2]+v[2]], label="x + v") - - plot(plot_nonquadr, plot_approx, layout=(2,1), size=(600, 800)) - -end - - -# ╔═╡ Cell order: -# ╟─171fee18-20f6-11eb-37e5-2d04caea8c35 -# ╠═5ad5c202-20f8-11eb-23f1-4f38b687c285 -# ╠═d2d007b8-20f8-11eb-0ddd-1181d4565a85 -# ╟─fb4aeb8c-20f7-11eb-0444-259de7b76883 -# ╟─52e87238-20f8-11eb-2ea0-27ee1208d3c3 -# ╠═e9a9d69e-20f8-11eb-0d9d-330ee5e9cf25 -# ╠═fdd4e550-20f8-11eb-227b-25f36708484d -# ╠═025fd6e8-20f9-11eb-3e7d-3519f3c4b58f -# ╟─096eff98-20f9-11eb-1e61-99d5714895ba -# ╠═165509ca-20f9-11eb-107c-550cbba0f0e9 -# ╠═1fffc82a-20f9-11eb-198c-c160d7dac87d -# ╟─26ab6ce2-20f9-11eb-1836-1756b290e5e3 -# ╠═49832a8e-20f9-11eb-0841-19a40a12db18 -# ╠═55e0e274-20f9-11eb-36c0-753f228f7e9b -# ╟─b1551758-20f9-11eb-3e8f-ff9a7127d7f8 -# ╠═41d8f1dc-20fa-11eb-3586-a989427c1fd6 -# ╠═45189a82-20fa-11eb-0423-05ce1b84639d -# ╠═4ed4215e-20fa-11eb-11ee-f7741591163c -# ╠═56af99ee-20fa-11eb-0240-69c675efb78c -# ╠═6c5473b4-20fa-11eb-327b-51ac560530eb -# ╠═7518c2c0-20fa-11eb-32c0-a9db2a91cbc5 -# ╠═34027942-20fb-11eb-261e-3b991ce4c9f8 -# ╠═3bbeb85c-20fc-11eb-04d0-fb12d8ace50a -# ╟─8623ac1a-20fa-11eb-2d45-49cce0fdac86 diff --git a/notebook/figures/CE_visual.jpg b/notebook/figures/CE_visual.jpg new file mode 100644 index 00000000..80aa67c0 Binary files /dev/null and b/notebook/figures/CE_visual.jpg differ