From 0a4943d1fb010e21ae6d80dcd953919405635fe5 Mon Sep 17 00:00:00 2001 From: "Documenter.jl" Date: Wed, 29 May 2024 00:56:42 +0000 Subject: [PATCH] build based on 09287f5 --- dev/.documenter-siteinfo.json | 2 +- dev/api/internal/index.html | 12 ++--- dev/api/public/index.html | 48 +++++++++---------- dev/contributing/index.html | 2 +- dev/examples-overview/index.html | 2 +- dev/generated/asia-network/main/index.html | 16 +++---- .../hard-core-lattice-gas/main/index.html | 28 +++++------ dev/generated/performance-tips/index.html | 12 ++--- dev/index.html | 2 +- dev/performance-evaluation/index.html | 2 +- dev/probabilistic-inference/index.html | 2 +- dev/search_index.js | 2 +- dev/tensor-networks/index.html | 2 +- dev/uai-file-formats/index.html | 2 +- 14 files changed, 67 insertions(+), 67 deletions(-) diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json index 87f1b0b..0a1eb07 100644 --- a/dev/.documenter-siteinfo.json +++ b/dev/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.10.3","generation_timestamp":"2024-05-22T03:23:22","documenter_version":"1.4.1"}} \ No newline at end of file +{"documenter":{"julia_version":"1.10.3","generation_timestamp":"2024-05-29T00:56:39","documenter_version":"1.4.1"}} \ No newline at end of file diff --git a/dev/api/internal/index.html b/dev/api/internal/index.html index 6cc5f6c..20d905a 100644 --- a/dev/api/internal/index.html +++ b/dev/api/internal/index.html @@ -1,5 +1,5 @@ -Internal · TensorInference.jl

Internal API

Index

Types

Functions

Types

TensorInference.Factor — Type
struct Factor{T, N}

Fields

  • vars
  • vals

Encodes a discrete function over the set of variables vars that maps each instantiation of vars into a nonnegative number in vals.

source
TensorInference.Samples — Type
struct Samples{L} <: AbstractVector{SubArray{Float64, 1, Matrix{Float64}, Tuple{Base.Slice{Base.OneTo{Int64}}, Int64}, true}}

Fields

  • samples::Matrix{Int64}

  • labels::Vector

  • setmask::BitVector

The sampled configurations are stored in samples, which is a vector of vector. labels is a vector of variable names for labeling configurations. The setmask is an boolean indicator to denote whether the sampling process of a variable is complete.

source

Functions

TensorInference.backward_sampling! — Method
backward_sampling!(
+Internal · TensorInference.jl
Internal API
Index
Types
Functions
Types
TensorInference.Factor — Type
struct Factor{T, N}
Fields
  • vars
  • vals
Encodes a discrete function over the set of variables vars that maps each instantiation of vars into a nonnegative number in vals.
source
TensorInference.Samples — Type
struct Samples{L} <: AbstractVector{SubArray{Float64, 1, Matrix{Float64}, Tuple{Base.Slice{Base.OneTo{Int64}}, Int64}, true}}
Fields
  • samples::Matrix{Int64}
  • labels::Vector
  • setmask::BitVector
The sampled configurations are stored in samples, which is a vector of vector. labels is a vector of variable names for labeling configurations. The setmask is an boolean indicator to denote whether the sampling process of a variable is complete.
source
Functions
TensorInference.backward_sampling! — Method
backward_sampling!(
     ixs,
     xs::Tuple,
     iy,
@@ -7,7 +7,7 @@
     samples::TensorInference.Samples,
     size_dict
 ) -> TensorInference.Samples
-
The backward process for sampling configurations.
  • ixs and xs are labels and tensor data for input tensors,
  • iy and y are labels and tensor data for the output tensor,
  • samples is the samples generated for eliminated variables,
  • size_dict is a key-value map from tensor label to dimension size.
source
TensorInference.backward_tropical! — Method
backward_tropical!(
+
The backward process for sampling configurations.
  • ixs and xs are labels and tensor data for input tensors,
  • iy and y are labels and tensor data for the output tensor,
  • samples is the samples generated for eliminated variables,
  • size_dict is a key-value map from tensor label to dimension size.
source
TensorInference.backward_tropical! — Method
backward_tropical!(
     ixs,
     xs::Tuple,
     iy,
@@ -15,12 +15,12 @@
     ymask,
     size_dict
 ) -> Vector{Any}
-
The backward rule for tropical einsum.
  • ixs and xs are labels and tensor data for input tensors,
  • iy and y are labels and tensor data for the output tensor,
  • ymask is the boolean mask for gradients,
  • size_dict is a key-value map from tensor label to dimension size.
source
TensorInference.parse_mar_solution_file — Method
parse_mar_solution_file(
+
The backward rule for tropical einsum.
  • ixs and xs are labels and tensor data for input tensors,
  • iy and y are labels and tensor data for the output tensor,
  • ymask is the boolean mask for gradients,
  • size_dict is a key-value map from tensor label to dimension size.
source
TensorInference.parse_mar_solution_file — Method
parse_mar_solution_file(
     rawlines::Vector{String};
     factor_eltype
 ) -> Vector{Vector{Float64}}
-
Parse the solution marginals of all variables from the UAI MAR solution file. The order of the variables is the same as in the model definition.
The UAI file formats are defined in: https://uaicompetition.github.io/uci-2022/file-formats/
source
TensorInference.read_query_file — Method
read_query_file(
+
Parse the solution marginals of all variables from the UAI MAR solution file. The order of the variables is the same as in the model definition.
The UAI file formats are defined in: https://uaicompetition.github.io/uci-2022/file-formats/
source
TensorInference.read_query_file — Method
read_query_file(
     query_filepath::AbstractString
 ) -> Vector{Int64}
-
Return the query variables in query_filepath. If the passed file path is an empty string, return an empty vector.
The UAI file formats are defined in: https://uaicompetition.github.io/uci-2022/file-formats/
source

+

Return the query variables in query_filepath. If the passed file path is an empty string, return an empty vector.

The UAI file formats are defined in: https://uaicompetition.github.io/uci-2022/file-formats/

source
diff --git a/dev/api/public/index.html b/dev/api/public/index.html index 45448c1..fdfec3d 100644 --- a/dev/api/public/index.html +++ b/dev/api/public/index.html @@ -1,5 +1,5 @@ -Public · TensorInference.jl

Public API

Index

Modules

Types

Functions

Modules

TensorInference — Module
source

Types

OMEinsumContractionOrders.GreedyMethod — Type
GreedyMethod{MT}
+Public · TensorInference.jl
Public API
Index
Modules
Types
Functions
Modules
TensorInference — Module
source
Types
OMEinsumContractionOrders.GreedyMethod — Type
GreedyMethod{MT}
 GreedyMethod(; method=MinSpaceOut(), nrepeat=10)
The fast but poor greedy optimizer. Input arguments are
  • method is MinSpaceDiff() or MinSpaceOut.
    • MinSpaceOut choose one of the contraction that produces a minimum output tensor size,
    • MinSpaceDiff choose one of the contraction that decrease the space most.
  • nrepeat is the number of repeatition, returns the best contraction order.
source
OMEinsumContractionOrders.KaHyParBipartite — Type
KaHyParBipartite{RT,IT,GM}
 KaHyParBipartite(; sc_target, imbalances=collect(0.0:0.005:0.8),
     max_group_size=40, greedy_config=GreedyMethod())
Optimize the einsum code contraction order using the KaHyPar + Greedy approach. This program first recursively cuts the tensors into several groups using KaHyPar, with maximum group size specifed by max_group_size and maximum space complexity specified by sc_target, Then finds the contraction order inside each group with the greedy search algorithm. Other arguments are
  • sc_target is the target space complexity, defined as log2(number of elements in the largest tensor),
  • imbalances is a KaHyPar parameter that controls the group sizes in hierarchical bipartition,
  • max_group_size is the maximum size that allowed to used greedy search,
  • greedy_config is a greedy optimizer.
References
source
OMEinsumContractionOrders.MergeGreedy — Type
MergeGreedy <: CodeSimplifier
@@ -8,19 +8,19 @@
 SABipartite(; sc_target=25, ntrials=50, βs=0.1:0.2:15.0, niters=1000
     max_group_size=40, greedy_config=GreedyMethod(), initializer=:random)
Optimize the einsum code contraction order using the Simulated Annealing bipartition + Greedy approach. This program first recursively cuts the tensors into several groups using simulated annealing, with maximum group size specifed by max_group_size and maximum space complexity specified by sc_target, Then finds the contraction order inside each group with the greedy search algorithm. Other arguments are
  • size_dict, a dictionary that specifies leg dimensions,
  • sc_target is the target space complexity, defined as log2(number of elements in the largest tensor),
  • max_group_size is the maximum size that allowed to used greedy search,
  • βs is a list of inverse temperature 1/T,
  • niters is the number of iteration in each temperature,
  • ntrials is the number of repetition (with different random seeds),
  • greedy_config configures the greedy method,
  • initializer, the partition configuration initializer, one can choose :random or :greedy (slow but better).
References
source
OMEinsumContractionOrders.TreeSA — Type
TreeSA{RT,IT,GM}
 TreeSA(; sc_target=20, βs=collect(0.01:0.05:15), ntrials=10, niters=50,
-    sc_weight=1.0, rw_weight=0.2, initializer=:greedy, greedy_config=GreedyMethod(; nrepeat=1))
Optimize the einsum contraction pattern using the simulated annealing on tensor expression tree.
  • sc_target is the target space complexity,
  • ntrials, βs and niters are annealing parameters, doing ntrials indepedent annealings, each has inverse tempteratures specified by βs, in each temperature, do niters updates of the tree.
  • sc_weight is the relative importance factor of space complexity in the loss compared with the time complexity.
  • rw_weight is the relative importance factor of memory read and write in the loss compared with the time complexity.
  • initializer specifies how to determine the initial configuration, it can be :greedy or :random. If it is using :greedy method to generate the initial configuration, it also uses two extra arguments greedy_method and greedy_nrepeat.
  • nslices is the number of sliced legs, default is 0.
  • fixed_slices is a vector of sliced legs, default is [].
References
source
TensorInference.MMAPModel — Type
struct MMAPModel{LT, AT<:AbstractArray}
Computing the most likely assignment to the query variables,  Xₘ ⊆ X after marginalizing out the remaining variables Xₛ = X \ Xₘ.
\[{\rm MMAP}(X_i|E=e) = \arg \max_{X_M} \sum_{X_S} \prod_{F} f(x_M, x_S, e)\]
Fields
  • vars is the query variables in the tensor network.
  • code is the tropical tensor network contraction pattern.
  • tensors is the tensors fed into the tensor network.
  • clusters is the clusters, each element of this cluster is a TensorNetworkModel instance for marginalizing certain variables.
  • evidence is a dictionary to specifiy degree of freedoms fixed to certain values, which should not have overlap with the query variables.
source
TensorInference.RescaledArray — Type
struct RescaledArray{T, N, AT<:AbstractArray{T, N}} <: AbstractArray{T, N}
RescaledArray(α, T) -> RescaledArray
An array data type with a log-prefactor, and a l∞-normalized storage, i.e. the maximum element in a tensor is 1. This tensor type can avoid the potential underflow/overflow of numbers in a tensor network. The constructor RescaledArray(α, T) creates a rescaled array that equal to exp(α) * T.
source
TensorInference.TensorNetworkModel — Type
struct TensorNetworkModel{LT, ET, MT<:AbstractArray}
Probabilistic modeling with a tensor network.
Fields
  • vars are the degrees of freedom in the tensor network.
  • code is the tensor network contraction pattern.
  • tensors are the tensors fed into the tensor network, the leading tensors are unity tensors associated with mars.
  • evidence is a dictionary used to specify degrees of freedom that are fixed to certain values.
  • mars is a vector, each element is a vector of variables to compute marginal probabilities.
source
TensorInference.UAIModel — Type
struct UAIModel{ET, FT<:(TensorInference.Factor{ET})}
Fields
  • nvars is the number of variables,
  • cards is a vector of cardinalities for variables,
  • factors is a vector of factors,
source
Functions
OMEinsumContractionOrders.contraction_complexity — Function
contraction_complexity(tensor_network)
Returns the contraction complexity of a tensor newtork model.
source
contraction_complexity(eincode, size_dict) -> ContractionComplexity
Returns the time, space and read-write complexity of the einsum contraction. The returned object contains 3 fields:
  • time complexity tc defined as log2(number of element-wise multiplications).
  • space complexity sc defined as log2(size of the maximum intermediate tensor).
  • read-write complexity rwc defined as log2(the number of read-write operations).
source
TensorInference.get_cards — Function
get_cards(tn::TensorNetworkModel; fixedisone) -> Vector
-
Get the cardinalities of variables in this tensor network.
source
get_cards(mmap::MMAPModel; fixedisone) -> Vector
-
source
TensorInference.get_vars — Function
get_vars(tn::TensorNetworkModel) -> Vector
-
Get the variables in this tensor network, they are also known as legs, labels, or degree of freedoms.
source
get_vars(mmap::MMAPModel) -> Vector
-
source
TensorInference.log_probability — Function
log_probability(
+    sc_weight=1.0, rw_weight=0.2, initializer=:greedy, greedy_config=GreedyMethod(; nrepeat=1))
Optimize the einsum contraction pattern using the simulated annealing on tensor expression tree.
  • sc_target is the target space complexity,
  • ntrials, βs and niters are annealing parameters, doing ntrials indepedent annealings, each has inverse tempteratures specified by βs, in each temperature, do niters updates of the tree.
  • sc_weight is the relative importance factor of space complexity in the loss compared with the time complexity.
  • rw_weight is the relative importance factor of memory read and write in the loss compared with the time complexity.
  • initializer specifies how to determine the initial configuration, it can be :greedy or :random. If it is using :greedy method to generate the initial configuration, it also uses two extra arguments greedy_method and greedy_nrepeat.
  • nslices is the number of sliced legs, default is 0.
  • fixed_slices is a vector of sliced legs, default is [].
References
source
TensorInference.MMAPModel — Type
struct MMAPModel{LT, AT<:AbstractArray}
Computing the most likely assignment to the query variables,  Xₘ ⊆ X after marginalizing out the remaining variables Xₛ = X \ Xₘ.
\[{\rm MMAP}(X_i|E=e) = \arg \max_{X_M} \sum_{X_S} \prod_{F} f(x_M, x_S, e)\]
Fields
  • vars is the query variables in the tensor network.
  • code is the tropical tensor network contraction pattern.
  • tensors is the tensors fed into the tensor network.
  • clusters is the clusters, each element of this cluster is a TensorNetworkModel instance for marginalizing certain variables.
  • evidence is a dictionary to specifiy degree of freedoms fixed to certain values, which should not have overlap with the query variables.
source
TensorInference.RescaledArray — Type
struct RescaledArray{T, N, AT<:AbstractArray{T, N}} <: AbstractArray{T, N}
RescaledArray(α, T) -> RescaledArray
An array data type with a log-prefactor, and a l∞-normalized storage, i.e. the maximum element in a tensor is 1. This tensor type can avoid the potential underflow/overflow of numbers in a tensor network. The constructor RescaledArray(α, T) creates a rescaled array that equal to exp(α) * T.
source
TensorInference.TensorNetworkModel — Type
struct TensorNetworkModel{LT, ET, MT<:AbstractArray}
Probabilistic modeling with a tensor network.
Fields
  • vars are the degrees of freedom in the tensor network.
  • code is the tensor network contraction pattern.
  • tensors are the tensors fed into the tensor network, the leading tensors are unity tensors associated with mars.
  • evidence is a dictionary used to specify degrees of freedom that are fixed to certain values.
  • mars is a vector, each element is a vector of variables to compute marginal probabilities.
source
TensorInference.UAIModel — Type
struct UAIModel{ET, FT<:(TensorInference.Factor{ET})}
Fields
  • nvars is the number of variables,
  • cards is a vector of cardinalities for variables,
  • factors is a vector of factors,
source
Functions
OMEinsumContractionOrders.contraction_complexity — Function
contraction_complexity(tensor_network)
Returns the contraction complexity of a tensor newtork model.
source
contraction_complexity(eincode, size_dict) -> ContractionComplexity
Returns the time, space and read-write complexity of the einsum contraction. The returned object contains 3 fields:
  • time complexity tc defined as log2(number of element-wise multiplications).
  • space complexity sc defined as log2(size of the maximum intermediate tensor).
  • read-write complexity rwc defined as log2(the number of read-write operations).
source
TensorInference.get_cards — Function
get_cards(tn::TensorNetworkModel; fixedisone) -> Vector
+
Get the cardinalities of variables in this tensor network.
source
get_cards(mmap::MMAPModel; fixedisone) -> Vector
+
source
TensorInference.get_vars — Function
get_vars(tn::TensorNetworkModel) -> Vector
+
Get the variables in this tensor network, they are also known as legs, labels, or degree of freedoms.
source
get_vars(mmap::MMAPModel) -> Vector
+
source
TensorInference.log_probability — Function
log_probability(
     tn::TensorNetworkModel,
     config::Union{Dict, AbstractVector}
 ) -> Real
-
Evaluate the log probability (or partition function) of config.
source
log_probability(
+
Evaluate the log probability (or partition function) of config.
source
log_probability(
     tn::TensorNetworkModel;
     usecuda
 ) -> AbstractArray
-
Evaluate the log probability (or partition function). It is the logged version of probability, which is less likely to overflow.
source
TensorInference.marginals — Function
marginals(
     tn::TensorNetworkModel;
     usecuda,
     rescale
@@ -51,61 +51,61 @@
 julia> marginals(tn2)
 Dict{Vector{Int64}, Matrix{Float64}} with 2 entries:
   [2, 3] => [0.025 0.025; 0.475 0.475]
-  [3, 4] => [0.05 0.45; 0.005 0.495]
In this example, we first set the evidence for variable 1 to 0 and then query the marginals of all individual variables. The returned dictionary has keys that correspond to the queried variables and values that represent their marginals. These marginals are vectors, with each entry corresponding to the probability of the variable taking a specific value. In this example, the possible values are 0 or 1. For the evidence variable 1, the marginal is always [1.0] since its value is fixed at 0.
Next, we specify the marginal variables to query as variables 2 and 3, and variables 3 and 4, respectively. The joint marginals may or may not affect the contraction time and space. In this example, the contraction space complexity increases from 2^{2.0} to 2^{5.0}, and the contraction time complexity increases from 2^{5.977} to 2^{7.781}. The output marginals are the joint probabilities of the queried variables, represented by tensors.
source
TensorInference.maximum_logp — Function
maximum_logp(
+  [3, 4] => [0.05 0.45; 0.005 0.495]
In this example, we first set the evidence for variable 1 to 0 and then query the marginals of all individual variables. The returned dictionary has keys that correspond to the queried variables and values that represent their marginals. These marginals are vectors, with each entry corresponding to the probability of the variable taking a specific value. In this example, the possible values are 0 or 1. For the evidence variable 1, the marginal is always [1.0] since its value is fixed at 0.
Next, we specify the marginal variables to query as variables 2 and 3, and variables 3 and 4, respectively. The joint marginals may or may not affect the contraction time and space. In this example, the contraction space complexity increases from 2^{2.0} to 2^{5.0}, and the contraction time complexity increases from 2^{5.977} to 2^{7.781}. The output marginals are the joint probabilities of the queried variables, represented by tensors.
source
TensorInference.maximum_logp — Function
maximum_logp(
     tn::TensorNetworkModel;
     usecuda
 ) -> AbstractArray{<:Real}
-
Returns an output array containing largest log-probabilities.
source
TensorInference.probability — Function
probability(
     tn::TensorNetworkModel;
     usecuda,
     rescale
 ) -> AbstractArray
-
Contract the tensor network and return an array of probability of evidence. Precisely speaking, the return value is the partition function, which may not be l1-normalized.
If the openvars of the input tensor networks is zero, the array rank is zero. Otherwise, the return values corresponds to marginal probabilities.
source
TensorInference.dataset_from_artifact — Function
dataset_from_artifact(
+
Contract the tensor network and return an array of probability of evidence. Precisely speaking, the return value is the partition function, which may not be l1-normalized.
If the openvars of the input tensor networks is zero, the array rank is zero. Otherwise, the return values corresponds to marginal probabilities.
source
TensorInference.dataset_from_artifact — Function
dataset_from_artifact(
     artifact_name::AbstractString
 ) -> Dict{String, Dict{String, Dict{Int64, ArtifactProblemSpec}}}
-
Helper function that captures the problem names that belong to problem_set for the given task.
source
TensorInference.problem_from_artifact — Function
problem_from_artifact(
     artifact_name::String,
     task::String,
     problem_set::String,
     problem_id::Int64
 ) -> ArtifactProblemSpec
-
Get artifact from artifact name, task name, problem set name and problem id.
source
TensorInference.read_solution — Function
read_solution(
     problem::ArtifactProblemSpec;
     factor_eltype
 ) -> Any
-
Return the solution in the artifact.
The UAI file formats are defined in: https://uaicompetition.github.io/uci-2022/file-formats/
source
TensorInference.read_queryvars — Function
read_queryvars(
+
Return the solution in the artifact.
The UAI file formats are defined in: https://uaicompetition.github.io/uci-2022/file-formats/
source
TensorInference.read_model_file — Function
read_model_file(
     model_filepath::AbstractString;
     factor_eltype
 ) -> UAIModel
-
Parse the problem instance found in model_filepath defined in the UAI model format. If the provided file path is empty, return nothing.
The UAI file formats are defined in: https://uaicompetition.github.io/uci-2022/file-formats/
source
TensorInference.read_evidence_file — Function
read_evidence_file(
+
Parse the problem instance found in model_filepath defined in the UAI model format. If the provided file path is empty, return nothing.
The UAI file formats are defined in: https://uaicompetition.github.io/uci-2022/file-formats/
source
TensorInference.read_evidence_file — Function
read_evidence_file(
     evidence_filepath::AbstractString
 ) -> Tuple{Vector{Int64}, Vector{Int64}}
-
Return the observed variables and values in evidence_filepath. If the passed file path is an empty string, return empty vectors.
The UAI file formats are defined in: https://uaicompetition.github.io/uci-2022/file-formats/
source
TensorInference.read_td_file — Function
read_td_file(
+
Return the observed variables and values in evidence_filepath. If the passed file path is an empty string, return empty vectors.
The UAI file formats are defined in: https://uaicompetition.github.io/uci-2022/file-formats/
source
TensorInference.read_td_file — Function
read_td_file(
     td_filepath::AbstractString
 ) -> Tuple{Int64, Int64, Int64, Vector{Vector{Int64}}, Vector{Vector{Int64}}}
-
Parse a tree decomposition instance described the PACE format.
The PACE file format is defined in: https://pacechallenge.org/2017/treewidth/
source
TensorInference.sample — Function
sample(
+
Parse a tree decomposition instance described the PACE format.
The PACE file format is defined in: https://pacechallenge.org/2017/treewidth/
source
TensorInference.sample — Function
sample(
     tn::TensorNetworkModel,
     n::Int64;
     usecuda
 ) -> TensorInference.Samples
-
Generate samples from a tensor network based probabilistic model. Returns a vector of vector, each element being a configurations defined on get_vars(tn).
Arguments
  • tn is the tensor network model.
  • n is the number of samples to be returned.
source
TensorInference.update_evidence! — Function
update_evidence!(
+
Generate samples from a tensor network based probabilistic model. Returns a vector of vector, each element being a configurations defined on get_vars(tn).
Arguments
  • tn is the tensor network model.
  • n is the number of samples to be returned.
source
TensorInference.update_evidence! — Function
update_evidence!(
     tnet::TensorNetworkModel,
     evidence::Dict
 ) -> TensorNetworkModel
-
Update the evidence of a tensor network model, without changing the set of observed variables!
Arguments
  • tnet is the TensorNetworkModel instance.
  • evidence is the new evidence, the keys must be a subset of existing evidence.
source
TensorInference.update_temperature — Function
update_temperature(
+
Update the evidence of a tensor network model, without changing the set of observed variables!
Arguments
  • tnet is the TensorNetworkModel instance.
  • evidence is the new evidence, the keys must be a subset of existing evidence.
source
TensorInference.update_temperature — Function
update_temperature(
     tnet::TensorNetworkModel,
     problem::GenericTensorNetworks.GraphProblem,
     β::Real
 ) -> TensorNetworkModel
-
Update the temperature of a tensor network model. The program will regenerate tensors from the problem, without repeated optimizing the contraction order.
Arguments
  • tnet is the TensorNetworkModel instance.
  • problem is the target constraint satisfiability problem.
  • β is the inverse temperature.
source

+

Update the temperature of a tensor network model. The program will regenerate tensors from the problem, without repeated optimizing the contraction order.

Arguments

  • tnet is the TensorNetworkModel instance.
  • problem is the target constraint satisfiability problem.
  • β is the inverse temperature.
source
diff --git a/dev/contributing/index.html b/dev/contributing/index.html index 0cab51d..18bb152 100644 --- a/dev/contributing/index.html +++ b/dev/contributing/index.html @@ -1,2 +1,2 @@ -Contributing · TensorInference.jl

Contributing

Welcome and thank you for considering contributing to TensorInference.jl.

By following these guidelines, you make it easier for everyone to work together. It shows you value the time of the people who create and manage this open-source project. In return, we'll show you the same respect by quickly looking at your issues, reviewing your changes, and helping you with your pull requests.

Getting Started

Contributions are made to this repo via Issues and Pull Requests (PRs). A few general guidelines that cover both:

  • Search for existing Issues and PRs before creating your own.
  • We do our best to solve problems quickly. Still, some issues take longer to understand. Posting a comment can help, especially if you need a quick fix.

Issues

Issues should be used to report problems with the package, ask for a new feature, or discuss possible changes before creating a Pull Request (PR). When you open a new issue, please include all the details needed to look into it.

If you find an Issue that addresses the problem you're having, please add your own reproduction information to the existing issue rather than creating a new one.

Pull Requests

Pull Requests (PRs) to our package are always welcome. Submitting a PR is a quick way to have your changes considered for the next software release. Generally, your Pull Request should:

  • Either fix or add the functionality in question OR address widespread whitespace/style issues, not both.
  • Add unit or integration tests for any fixed or changed features.
  • Minimize the number of changed lines to address a single concern.
  • Include documentation in the repo, especially for implementations of new features.

For major changes that affect core functionality or would require a new major release, we recommend opening an Issue to discuss your ideas before creating a Pull Request. While this step is optional, it can save everyone time in both the creation and review phases.

In general, we follow the "fork-and-pull" Git workflow

  1. Fork the repository to your own GitHub account.
  2. Clone the project to your machine.
  3. Create a branch locally with a succinct but descriptive name.
  4. Commit changes to the branch.
  5. Follow any formatting and testing guidelines specific to this repo.
  6. Push changes to your fork.
  7. Open a PR in our repository and complete the PR template to help us efficiently review the changes.
+Contributing · TensorInference.jl

Contributing

Welcome and thank you for considering contributing to TensorInference.jl.

By following these guidelines, you make it easier for everyone to work together. It shows you value the time of the people who create and manage this open-source project. In return, we'll show you the same respect by quickly looking at your issues, reviewing your changes, and helping you with your pull requests.

Getting Started

Contributions are made to this repo via Issues and Pull Requests (PRs). A few general guidelines that cover both:

  • Search for existing Issues and PRs before creating your own.
  • We do our best to solve problems quickly. Still, some issues take longer to understand. Posting a comment can help, especially if you need a quick fix.

Issues

Issues should be used to report problems with the package, ask for a new feature, or discuss possible changes before creating a Pull Request (PR). When you open a new issue, please include all the details needed to look into it.

If you find an Issue that addresses the problem you're having, please add your own reproduction information to the existing issue rather than creating a new one.

Pull Requests

Pull Requests (PRs) to our package are always welcome. Submitting a PR is a quick way to have your changes considered for the next software release. Generally, your Pull Request should:

  • Either fix or add the functionality in question OR address widespread whitespace/style issues, not both.
  • Add unit or integration tests for any fixed or changed features.
  • Minimize the number of changed lines to address a single concern.
  • Include documentation in the repo, especially for implementations of new features.

For major changes that affect core functionality or would require a new major release, we recommend opening an Issue to discuss your ideas before creating a Pull Request. While this step is optional, it can save everyone time in both the creation and review phases.

In general, we follow the "fork-and-pull" Git workflow

  1. Fork the repository to your own GitHub account.
  2. Clone the project to your machine.
  3. Create a branch locally with a succinct but descriptive name.
  4. Commit changes to the branch.
  5. Follow any formatting and testing guidelines specific to this repo.
  6. Push changes to your fork.
  7. Open a PR in our repository and complete the PR template to help us efficiently review the changes.
diff --git a/dev/examples-overview/index.html b/dev/examples-overview/index.html index a105fb3..79011ff 100644 --- a/dev/examples-overview/index.html +++ b/dev/examples-overview/index.html @@ -1,2 +1,2 @@ -Overview · TensorInference.jl
+Overview · TensorInference.jl
diff --git a/dev/generated/asia-network/main/index.html b/dev/generated/asia-network/main/index.html index 86e128c..2456e35 100644 --- a/dev/generated/asia-network/main/index.html +++ b/dev/generated/asia-network/main/index.html @@ -42,18 +42,18 @@ 7 8

Set the evidence: Assume that the "X-ray" result (variable 7) is negative. Since setting the evidence may affect the contraction order of the tensor network, recompute it.

tn = TensorNetworkModel(model, evidence = Dict(7 => 0))
TensorNetworkModel{Int64, OMEinsum.DynamicNestedEinsum{Int64}, Array{Float64}}
 variables: 1, 2, 3, 4, 5, 6, 7 (evidence → 0), 8
-contraction time = 2^6.0, space = 2^2.0, read-write = 2^7.066

Calculate the maximum log-probability among all configurations.

maximum_logp(tn)
0-dimensional Array{Float64, 0}:
--3.65222179200233

Generate 10 samples from the posterior distribution.

sample(tn, 10)
10-element TensorInference.Samples{Int64}:
- [0, 1, 0, 0, 1, 0, 0, 0]
+contraction time = 2^6.044, space = 2^2.0, read-write = 2^7.109

Calculate the maximum log-probability among all configurations.

maximum_logp(tn)
0-dimensional Array{Float64, 0}:
+-3.6522217920023303

Generate 10 samples from the posterior distribution.

sample(tn, 10)
10-element TensorInference.Samples{Int64}:
+ [1, 1, 0, 1, 1, 1, 0, 1]
+ [1, 1, 0, 1, 1, 1, 0, 0]
  [1, 1, 1, 1, 1, 1, 0, 1]
  [1, 1, 0, 0, 0, 0, 0, 0]
- [1, 1, 1, 1, 1, 1, 0, 0]
+ [1, 0, 0, 1, 1, 0, 0, 1]
  [1, 1, 0, 0, 0, 0, 0, 0]
+ [1, 1, 0, 1, 0, 1, 0, 0]
  [1, 1, 0, 0, 0, 0, 0, 0]
  [1, 1, 1, 1, 1, 1, 0, 1]
- [1, 1, 1, 0, 1, 0, 0, 0]
- [1, 1, 0, 1, 1, 1, 0, 1]
- [1, 1, 0, 0, 1, 0, 0, 1]

Retrieve both the maximum log-probability and the most probable configuration.

logp, cfg = most_probable_config(tn)
(-3.65222179200233, [1, 1, 0, 0, 0, 0, 0, 0])

Compute the most probable values of certain variables (e.g., 4 and 7) while marginalizing over others. This is known as Maximum a Posteriori (MAP) estimation.

mmap = MMAPModel(model, evidence=Dict(7=>0), queryvars=[4,7])
MMAPModel{Int64, Array{Float64}}
+ [1, 1, 0, 0, 0, 0, 0, 0]

Retrieve both the maximum log-probability and the most probable configuration.

logp, cfg = most_probable_config(tn)
(-3.6522217920023303, [1, 1, 0, 0, 0, 0, 0, 0])

Compute the most probable values of certain variables (e.g., 4 and 7) while marginalizing over others. This is known as Maximum a Posteriori (MAP) estimation.

mmap = MMAPModel(model, evidence=Dict(7=>0), queryvars=[4,7])
MMAPModel{Int64, Array{Float64}}
 variables: 4, 7 (evidence → 0)
 query variables: [[1, 2, 6, 5, 3, 8]]
-contraction time = 2^5.977, space = 2^2.0, read-write = 2^6.989

Get the most probable configurations for variables 4 and 7.

most_probable_config(mmap)
(-2.8754627318176693, [1, 0])

Compute the total log-probability of having lung cancer. The results suggest that the probability is roughly half.

log_probability(mmap, [1, 0]), log_probability(mmap, [0, 0])
(-2.8754627318176693, -2.9206248010671856)

This page was generated using Literate.jl.

+contraction time = 2^6.0, space = 2^2.0, read-write = 2^7.0

Get the most probable configurations for variables 4 and 7.

most_probable_config(mmap)
(-2.8754627318176693, [1, 0])

Compute the total log-probability of having lung cancer. The results suggest that the probability is roughly half.

log_probability(mmap, [1, 0]), log_probability(mmap, [0, 0])
(-2.8754627318176693, -2.9206248010671856)

This page was generated using Literate.jl.

diff --git a/dev/generated/hard-core-lattice-gas/main/index.html b/dev/generated/hard-core-lattice-gas/main/index.html index 0061f74..1ecd68d 100644 --- a/dev/generated/hard-core-lattice-gas/main/index.html +++ b/dev/generated/hard-core-lattice-gas/main/index.html @@ -113,7 +113,7 @@ β = 3.0 pmodel = TensorNetworkModel(problem, β)
TensorNetworkModel{Int64, OMEinsum.DynamicNestedEinsum{Int64}, Array{Float64}}
 variables: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100
-contraction time = 2^18.036, space = 2^11.0, read-write = 2^15.814

The partition function of this statistical model can be computed with the probability function.

partition_func = probability(pmodel)
exp(107.42267890429402) * fill(1.0)

The default return value is a log-rescaled tensor. Use indexing to get the real value.

partition_func[]
4.4985927541461473e46

The marginal probabilities can be computed with the marginals function, which measures how likely a site is occupied.

mars = marginals(pmodel)
+contraction time = 2^18.013, space = 2^12.0, read-write = 2^15.986

The partition function of this statistical model can be computed with the probability function.

partition_func = probability(pmodel)
exp(107.42267890429403) * fill(1.0)

The default return value is a log-rescaled tensor. Use indexing to get the real value.

partition_func[]
4.4985927541462117e46

The marginal probabilities can be computed with the marginals function, which measures how likely a site is occupied.

mars = marginals(pmodel)
 show_graph(graph, sites; vertex_colors=[(b = mars[[i]][2]; (1-b, 1-b, 1-b)) for i in 1:nv(graph)], texts=fill("", nv(graph)))

The can see the sites at the corner is more likely to be occupied. To obtain two-site correlations, one can set the variables to query marginal probabilities manually.

pmodel2 = TensorNetworkModel(problem, β; mars=[[e.src, e.dst] for e in edges(graph)])
 mars = marginals(pmodel2);

We show the probability that both sites on an edge are not occupied

show_graph(graph, sites; edge_colors=[(b = mars[[e.src, e.dst]][1, 1]; (1-b, 1-b, 1-b)) for e in edges(graph)], texts=fill("", nv(graph)), config=GraphDisplayConfig(; edge_line_width=5))

The most likely configuration

The MAP and MMAP can be used to get the most likely configuration given an evidence. The relavant function is most_probable_config. If we fix the vertex configuration at one corner to be one, we get the most probably configuration as bellow.

pmodel3 = TensorNetworkModel(problem, β; evidence=Dict(1=>1))
 mars = marginals(pmodel3)
@@ -144,17 +144,17 @@
    0
    0
    0
-   1
    0
-   4
-   6
-  16
-  80
- 123
- 174
- 226
- 177
- 127
-  54
-  12
-   0

This page was generated using Literate.jl.

+ 0 + 0 + 9 + 21 + 45 + 124 + 202 + 216 + 176 + 128 + 60 + 19 + 0

This page was generated using Literate.jl.

diff --git a/dev/generated/performance-tips/index.html b/dev/generated/performance-tips/index.html index f8d2c32..f683aca 100644 --- a/dev/generated/performance-tips/index.html +++ b/dev/generated/performance-tips/index.html @@ -1,13 +1,13 @@ Performance tips · TensorInference.jl

Performance Tips

Optimize contraction orders

Let us use a problem instance from the "Promedus" dataset of the UAI 2014 competition as an example.

using TensorInference
 problem = problem_from_artifact("uai2014", "MAR", "Promedus", 11)
-model, evidence = read_model(problem), read_evidence(problem);

Next, we select the tensor network contraction order optimizer.

optimizer = TreeSA(ntrials = 1, niters = 5, βs = 0.1:0.3:100)
TreeSA{Int64, StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, GreedyMethod{OMEinsumContractionOrders.MinSpaceOut}, Any}(20, 0.1:0.3:100.0, 1, 5, 1.0, 0.2, :greedy, 0, Any[], GreedyMethod{OMEinsumContractionOrders.MinSpaceOut}(OMEinsumContractionOrders.MinSpaceOut(), 1))

Here, we choose the local search based TreeSA algorithm, which often finds the smallest time/space complexity and supports slicing. One can type ?TreeSA in a Julia REPL for more information about how to configure the hyper-parameters of the TreeSA method, while the detailed algorithm explanation is in arXiv: 2108.05665. Alternative tensor network contraction order optimizers include

tn = TensorNetworkModel(model; optimizer, evidence);

The returned object tn contains a field code that specifies the tensor network with optimized contraction order. To check the contraction complexity, please type

contraction_complexity(tn)
Time complexity: 2^19.330221020205514
-Space complexity: 2^14.0
-Read-write complexity: 2^17.78831687691557

The returned object contains log2 values of the number of multiplications, the number elements in the largest tensor during contraction and the number of read-write operations to tensor elements.

probability(tn)
exp(-19.322038772705977) * fill(1.0)

Using the slicing technique to reduce the memory cost

For large scale applications, it is also possible to slice over certain degrees of freedom to reduce the space complexity, i.e. loop and accumulate over certain degrees of freedom so that one can have a smaller tensor network inside the loop due to the removal of these degrees of freedom. In the TreeSA optimizer, one can set nslices to a value larger than zero to turn on this feature. As a comparison we slice over 5 degrees of freedom, which can reduce the space complexity by at most 5. In this application, the slicing achieves the largest possible space complexity reduction 5, while the time and read-write complexity are only increased by less than 1, i.e. the peak memory usage is reduced by a factor $32$, while the (theoretical) computing time is increased by at a factor $< 2$.

optimizer = TreeSA(ntrials = 1, niters = 5, βs = 0.1:0.3:100, nslices=5)
+model, evidence = read_model(problem), read_evidence(problem);

Next, we select the tensor network contraction order optimizer.

optimizer = TreeSA(ntrials = 1, niters = 5, βs = 0.1:0.3:100)
TreeSA{Int64, StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, GreedyMethod{OMEinsumContractionOrders.MinSpaceOut}, Any}(20, 0.1:0.3:100.0, 1, 5, 1.0, 0.2, :greedy, 0, Any[], GreedyMethod{OMEinsumContractionOrders.MinSpaceOut}(OMEinsumContractionOrders.MinSpaceOut(), 1))

Here, we choose the local search based TreeSA algorithm, which often finds the smallest time/space complexity and supports slicing. One can type ?TreeSA in a Julia REPL for more information about how to configure the hyper-parameters of the TreeSA method, while the detailed algorithm explanation is in arXiv: 2108.05665. Alternative tensor network contraction order optimizers include

tn = TensorNetworkModel(model; optimizer, evidence);

The returned object tn contains a field code that specifies the tensor network with optimized contraction order. To check the contraction complexity, please type

contraction_complexity(tn)
Time complexity: 2^21.503163838633586
+Space complexity: 2^16.0
+Read-write complexity: 2^18.649049128960407

The returned object contains log2 values of the number of multiplications, the number elements in the largest tensor during contraction and the number of read-write operations to tensor elements.

probability(tn)
exp(-19.322038772705984) * fill(1.0)

Using the slicing technique to reduce the memory cost

For large scale applications, it is also possible to slice over certain degrees of freedom to reduce the space complexity, i.e. loop and accumulate over certain degrees of freedom so that one can have a smaller tensor network inside the loop due to the removal of these degrees of freedom. In the TreeSA optimizer, one can set nslices to a value larger than zero to turn on this feature. As a comparison we slice over 5 degrees of freedom, which can reduce the space complexity by at most 5. In this application, the slicing achieves the largest possible space complexity reduction 5, while the time and read-write complexity are only increased by less than 1, i.e. the peak memory usage is reduced by a factor $32$, while the (theoretical) computing time is increased by at a factor $< 2$.

optimizer = TreeSA(ntrials = 1, niters = 5, βs = 0.1:0.3:100, nslices=5)
 tn = TensorNetworkModel(model; optimizer, evidence);
-contraction_complexity(tn)
Time complexity: 2^22.58280353572975
+contraction_complexity(tn)
Time complexity: 2^22.021695728226906
 Space complexity: 2^11.0
-Read-write complexity: 2^21.01447821617116

Faster Tropical tensor contraction to speed up MAP and MMAP

One can enjoy the BLAS level speed provided by TropicalGEMM by importing the package with

using TropicalGEMM

The benchmark in the TropicalGEMM repo shows this performance is close to the theoretical optimal value. Its implementation on GPU is under development in Github repo CuTropicalGEMM.jl as a part of Open Source Promotion Plan summer program.

Working with GPUs

To upload the computation to GPU, you just add using CUDA before calling the solve function, and set the keyword argument usecuda to true.

julia> using CUDA
+Read-write complexity: 2^20.849576142049628

Faster Tropical tensor contraction to speed up MAP and MMAP

One can enjoy the BLAS level speed provided by TropicalGEMM by importing the package with

using TropicalGEMM

The benchmark in the TropicalGEMM repo shows this performance is close to the theoretical optimal value. Its implementation on GPU is under development in Github repo CuTropicalGEMM.jl as a part of Open Source Promotion Plan summer program.

Working with GPUs

To upload the computation to GPU, you just add using CUDA before calling the solve function, and set the keyword argument usecuda to true.

julia> using CUDA
 [ Info: OMEinsum loaded the CUDA module successfully
 
-julia> marginals(tn; usecuda = true);

Functions support usecuda keyword argument includes

Benchmarks

For performance metrics, see the Performance evaluation section.

This page was generated using Literate.jl.

+julia> marginals(tn; usecuda = true);

Functions support usecuda keyword argument includes

Benchmarks

For performance metrics, see the Performance evaluation section.

This page was generated using Literate.jl.

diff --git a/dev/index.html b/dev/index.html index a183fc2..476d051 100644 --- a/dev/index.html +++ b/dev/index.html @@ -1,2 +1,2 @@ -Home · TensorInference.jl

TensorInference.jl

TensorInference is a standalone solver written in Julia, that harnesses tensor-based technology to implement state-of-the-art algorithms for probabilistic inference in graphical models.

Package features

Solutions to the most common probabilistic inference tasks, including:

  • Probability of evidence (PR): Calculates the total probability of the observed evidence across all possible states of the unobserved variables.

  • Marginal inference (MAR): Computes the probability distribution of a subset of variables, ignoring the states of all other variables.

  • Maximum a Posteriori Probability estimation (MAP): Finds the most probable state of a subset of unobserved variables given some observed evidence.

  • Marginal Maximum a Posteriori (MMAP): Finds the most probable state of a subset of variables, averaging out the uncertainty over the remaining ones.

Why TensorInference.jl

A major challenge in developing intelligent systems is the ability to reason under uncertainty, a challenge that appears in many real-world problems across various domains, including artificial intelligence, medical diagnosis, computer vision, computational biology, and natural language processing. Reasoning under uncertainty involves calculating the probabilities of relevant variables while taking into account any information that is acquired. This process, which can be thought of as drawing global insights from local observations, is known as probabilistic inference.

Probabilistic graphical models (PGMs) provide a unified framework to perform probabilistic inference. These models use graphs to represent the joint probability distribution of complex systems in a concise manner by exploiting the conditional independence between variables in the model. Additionally, they form the foundation for various algorithms that enable efficient probabilistic inference.

However, even with the representational aid of PGMs, performing probabilistic inference remains an intractable endeavor on many real-world models. The reason is that performing probabilistic inference involves complex combinatorial optimization problems in very high dimensional spaces. To tackle these challenges, more efficient and scalable inference algorithms are needed.

As an attempt to tackle the aforementioned challenges, we present TensorInference.jl, a Julia package for probabilistic inference that combines the representational capabilities of PGMs with the computational power of tensor networks. By harnessing the best of both worlds, TensorInference.jl aims to enhance the performance of probabilistic inference, thereby expanding the tractability spectrum of exact inference for more complex, real-world models.

Outline

+Home · TensorInference.jl

TensorInference.jl

TensorInference is a standalone solver written in Julia, that harnesses tensor-based technology to implement state-of-the-art algorithms for probabilistic inference in graphical models.

Package features

Solutions to the most common probabilistic inference tasks, including:

  • Probability of evidence (PR): Calculates the total probability of the observed evidence across all possible states of the unobserved variables.

  • Marginal inference (MAR): Computes the probability distribution of a subset of variables, ignoring the states of all other variables.

  • Maximum a Posteriori Probability estimation (MAP): Finds the most probable state of a subset of unobserved variables given some observed evidence.

  • Marginal Maximum a Posteriori (MMAP): Finds the most probable state of a subset of variables, averaging out the uncertainty over the remaining ones.

Why TensorInference.jl

A major challenge in developing intelligent systems is the ability to reason under uncertainty, a challenge that appears in many real-world problems across various domains, including artificial intelligence, medical diagnosis, computer vision, computational biology, and natural language processing. Reasoning under uncertainty involves calculating the probabilities of relevant variables while taking into account any information that is acquired. This process, which can be thought of as drawing global insights from local observations, is known as probabilistic inference.

Probabilistic graphical models (PGMs) provide a unified framework to perform probabilistic inference. These models use graphs to represent the joint probability distribution of complex systems in a concise manner by exploiting the conditional independence between variables in the model. Additionally, they form the foundation for various algorithms that enable efficient probabilistic inference.

However, even with the representational aid of PGMs, performing probabilistic inference remains an intractable endeavor on many real-world models. The reason is that performing probabilistic inference involves complex combinatorial optimization problems in very high dimensional spaces. To tackle these challenges, more efficient and scalable inference algorithms are needed.

As an attempt to tackle the aforementioned challenges, we present TensorInference.jl, a Julia package for probabilistic inference that combines the representational capabilities of PGMs with the computational power of tensor networks. By harnessing the best of both worlds, TensorInference.jl aims to enhance the performance of probabilistic inference, thereby expanding the tractability spectrum of exact inference for more complex, real-world models.

Outline

diff --git a/dev/performance-evaluation/index.html b/dev/performance-evaluation/index.html index b1caeb7..6e2a9c3 100644 --- a/dev/performance-evaluation/index.html +++ b/dev/performance-evaluation/index.html @@ -1,2 +1,2 @@ -Performance evaluation · TensorInference.jl

Performance evaluation

The graph below illustrates a comparison of the runtime performance of TensorInference.jl against Merlin [marinescu2022merlin], libDAI [mooij2010libdai], and JunctionTrees.jl [roa2022partial] libraries, specifically for the task of computing the marginal probabilities of all variables. Both Merlin and libDAI have previously participated in UAI inference competitions [gal2010summary][gogate2014uai], achieving favorable results. Additionally, we compared against JunctionTrees.jl, the predecessor of TensorInference.jl. The experiments were conducted on an Intel Core i9–9900K CPU @3.60GHz with 64 GB of RAM. Performance comparisons for other tasks will be added in the near future.

The benchmark problems are arranged along the x-axis in ascending order of complexity, measured by the induced tree width. On average, TensorInference.jl achieves a speedup of 20 times across all problems. Notably, for the 10 most complex problems, the average speedup increases to 148 times, highlighting its superior scalability. The graph features a fitted linear curve in log-space to underscore the exponential improvement in computation time achieved by TensorInference.jl in comparison to the other alternatives. This speedup is primarily due to our package's unique approach: while traditional solvers typically focus only on minimizing space complexity (as quantified by the induced tree width), TensorInference.jl is designed to optimize for both time and space complexity.

References

+Performance evaluation · TensorInference.jl

Performance evaluation

The graph below illustrates a comparison of the runtime performance of TensorInference.jl against Merlin [marinescu2022merlin], libDAI [mooij2010libdai], and JunctionTrees.jl [roa2022partial] libraries, specifically for the task of computing the marginal probabilities of all variables. Both Merlin and libDAI have previously participated in UAI inference competitions [gal2010summary][gogate2014uai], achieving favorable results. Additionally, we compared against JunctionTrees.jl, the predecessor of TensorInference.jl. The experiments were conducted on an Intel Core i9–9900K CPU @3.60GHz with 64 GB of RAM. Performance comparisons for other tasks will be added in the near future.

The benchmark problems are arranged along the x-axis in ascending order of complexity, measured by the induced tree width. On average, TensorInference.jl achieves a speedup of 20 times across all problems. Notably, for the 10 most complex problems, the average speedup increases to 148 times, highlighting its superior scalability. The graph features a fitted linear curve in log-space to underscore the exponential improvement in computation time achieved by TensorInference.jl in comparison to the other alternatives. This speedup is primarily due to our package's unique approach: while traditional solvers typically focus only on minimizing space complexity (as quantified by the induced tree width), TensorInference.jl is designed to optimize for both time and space complexity.

References

diff --git a/dev/probabilistic-inference/index.html b/dev/probabilistic-inference/index.html index 1e1d32f..4e263a4 100644 --- a/dev/probabilistic-inference/index.html +++ b/dev/probabilistic-inference/index.html @@ -3,4 +3,4 @@ \setminus V_i} \prod_{\phi \in \bm{\phi}} \phi(V^{\prime\prime},\bm{e}) }{ PR(\bm{V}^{\prime} \mid \bm{E}=\bm{e}) }\]

This task involves computing the marginal probability of a subset of variables, integrating out the others. In other words, it computes the probability distribution of some variables of interest regardless of the states of all other variables. This is useful when we're interested in the probabilities of some specific variables in the model, but not the entire model.

Maximum a Posteriori Probability estimation (MAP)

Computing the most likely assignment to all variables given evidence:

\[MAP(V_i \mid \bm{E}=\bm{e}) = \arg \max_{V^{\prime} \in \bm{V}^{\prime}} \prod_{\phi \in \bm{\phi}} \phi(V^{\prime},\bm{e})\]

In the MAP task, given some observed variables, the goal is to find the most probable assignment of values to some subset of the unobserved variables. It provides the states of variables that maximize the posterior probability given some observed evidence. This is often used when we want the most likely explanation or prediction according to the model.

Marginal Maximum a Posteriori (MMAP)

Computing the most likely assignment to the query variables, $\bm{Q} \subset \bm{V}^{\prime}$ after marginalizing out the remaining variables $\bm{Z} = \bm{V}^{\prime} \setminus \bm{Q}$, also known as hidden or latent variables:

\[MMAP(V_i \mid \bm{E}=e) = \arg \max_{Q \in \bm{Q}} \sum_{Z \in \bm{Z}} -\prod_{\phi \in \bm{\phi}} \phi(Q, Z, e)\]

This task is essentially a combination of the MAR and MAP tasks. The MMAP task involves finding the most probable assignment (the MAP estimate) for a subset of the variables, while marginalizing over (summing out) the remaining variables. This task is useful when we want to know the most likely state of some variables, but there's some uncertainty over others that we need to average out.

+\prod_{\phi \in \bm{\phi}} \phi(Q, Z, e)\]

This task is essentially a combination of the MAR and MAP tasks. The MMAP task involves finding the most probable assignment (the MAP estimate) for a subset of the variables, while marginalizing over (summing out) the remaining variables. This task is useful when we want to know the most likely state of some variables, but there's some uncertainty over others that we need to average out.

diff --git a/dev/search_index.js b/dev/search_index.js index 2c8d8fa..817f041 100644 --- a/dev/search_index.js +++ b/dev/search_index.js @@ -1,3 +1,3 @@ var documenterSearchIndex = {"docs": -[{"location":"contributing/#Contributing","page":"Contributing","title":"Contributing","text":"","category":"section"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Welcome and thank you for considering contributing to TensorInference.jl.","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"By following these guidelines, you make it easier for everyone to work together. It shows you value the time of the people who create and manage this open-source project. In return, we'll show you the same respect by quickly looking at your issues, reviewing your changes, and helping you with your pull requests.","category":"page"},{"location":"contributing/#Getting-Started","page":"Contributing","title":"Getting Started","text":"","category":"section"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Contributions are made to this repo via Issues and Pull Requests (PRs). A few general guidelines that cover both:","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Search for existing Issues and PRs before creating your own.\nWe do our best to solve problems quickly. Still, some issues take longer to understand. Posting a comment can help, especially if you need a quick fix.","category":"page"},{"location":"contributing/#Issues","page":"Contributing","title":"Issues","text":"","category":"section"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Issues should be used to report problems with the package, ask for a new feature, or discuss possible changes before creating a Pull Request (PR). When you open a new issue, please include all the details needed to look into it.","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"If you find an Issue that addresses the problem you're having, please add your own reproduction information to the existing issue rather than creating a new one. ","category":"page"},{"location":"contributing/#Pull-Requests","page":"Contributing","title":"Pull Requests","text":"","category":"section"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Pull Requests (PRs) to our package are always welcome. Submitting a PR is a quick way to have your changes considered for the next software release. Generally, your Pull Request should:","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Either fix or add the functionality in question OR address widespread whitespace/style issues, not both.\nAdd unit or integration tests for any fixed or changed features.\nMinimize the number of changed lines to address a single concern.\nInclude documentation in the repo, especially for implementations of new features.","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"For major changes that affect core functionality or would require a new major release, we recommend opening an Issue to discuss your ideas before creating a Pull Request. While this step is optional, it can save everyone time in both the creation and review phases.","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"In general, we follow the \"fork-and-pull\" Git workflow","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Fork the repository to your own GitHub account.\nClone the project to your machine.\nCreate a branch locally with a succinct but descriptive name.\nCommit changes to the branch.\nFollow any formatting and testing guidelines specific to this repo.\nPush changes to your fork.\nOpen a PR in our repository and complete the PR template to help us efficiently review the changes.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"EditURL = \"main.jl\"","category":"page"},{"location":"generated/hard-core-lattice-gas/main/#The-hard-core-lattice-gas","page":"Hard-core Lattice Gas","title":"The hard core lattice gas","text":"","category":"section"},{"location":"generated/hard-core-lattice-gas/main/#Hard-core-lattice-gas-problem","page":"Hard-core Lattice Gas","title":"Hard-core lattice gas problem","text":"","category":"section"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"Hard-core lattice gas refers to a model used in statistical physics to study the behavior of particles on a lattice, where the particles are subject to an exclusion principle known as the \"hard-core\" interaction that characterized by a blockade radius. Distances between two particles can not be smaller than this radius.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"Nath T, Rajesh R. Multiple phase transitions in extended hard-core lattice gas models in two dimensions[J]. Physical Review E, 2014, 90(1): 012120.\nFernandes H C M, Arenzon J J, Levin Y. Monte Carlo simulations of two-dimensional hard core lattice gases[J]. The Journal of chemical physics, 2007, 126(11).","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"Let define a 10 times 10 triangular lattice, with unit vectors","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"beginalign*\nvec a = left(beginmatrix1 0endmatrixright)\nvec b = left(beginmatrixfrac12 fracsqrt32endmatrixright)\nendalign*","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"a, b = (1, 0), (0.5, 0.5*sqrt(3))\nNa, Nb = 10, 10\nsites = vec([50 .* (a .* i .+ b .* j) for i=1:Na, j=1:Nb])","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"100-element Vector{Tuple{Float64, Float64}}:\n (75.0, 43.30127018922193)\n (125.0, 43.30127018922193)\n (175.0, 43.30127018922193)\n (225.0, 43.30127018922193)\n (275.0, 43.30127018922193)\n (325.0, 43.30127018922193)\n (375.0, 43.30127018922193)\n (425.0, 43.30127018922193)\n (475.0, 43.30127018922193)\n (525.0, 43.30127018922193)\n (100.0, 86.60254037844386)\n (150.0, 86.60254037844386)\n (200.0, 86.60254037844386)\n (250.0, 86.60254037844386)\n (300.0, 86.60254037844386)\n (350.0, 86.60254037844386)\n (400.0, 86.60254037844386)\n (450.0, 86.60254037844386)\n (500.0, 86.60254037844386)\n (550.0, 86.60254037844386)\n (125.0, 129.9038105676658)\n (175.0, 129.9038105676658)\n (225.0, 129.9038105676658)\n (275.0, 129.9038105676658)\n (325.0, 129.9038105676658)\n (375.0, 129.9038105676658)\n (425.0, 129.9038105676658)\n (475.0, 129.9038105676658)\n (525.0, 129.9038105676658)\n (575.0, 129.9038105676658)\n (150.0, 173.20508075688772)\n (200.0, 173.20508075688772)\n (250.0, 173.20508075688772)\n (300.0, 173.20508075688772)\n (350.0, 173.20508075688772)\n (400.0, 173.20508075688772)\n (450.0, 173.20508075688772)\n (500.0, 173.20508075688772)\n (550.0, 173.20508075688772)\n (600.0, 173.20508075688772)\n (175.0, 216.50635094610965)\n (225.0, 216.50635094610965)\n (275.0, 216.50635094610965)\n (325.0, 216.50635094610965)\n (375.0, 216.50635094610965)\n (425.0, 216.50635094610965)\n (475.0, 216.50635094610965)\n (525.0, 216.50635094610965)\n (575.0, 216.50635094610965)\n (625.0, 216.50635094610965)\n (200.0, 259.8076211353316)\n (250.0, 259.8076211353316)\n (300.0, 259.8076211353316)\n (350.0, 259.8076211353316)\n (400.0, 259.8076211353316)\n (450.0, 259.8076211353316)\n (500.0, 259.8076211353316)\n (550.0, 259.8076211353316)\n (600.0, 259.8076211353316)\n (650.0, 259.8076211353316)\n (225.0, 303.1088913245535)\n (275.0, 303.1088913245535)\n (325.0, 303.1088913245535)\n (375.0, 303.1088913245535)\n (425.0, 303.1088913245535)\n (475.0, 303.1088913245535)\n (525.0, 303.1088913245535)\n (575.0, 303.1088913245535)\n (625.0, 303.1088913245535)\n (675.0, 303.1088913245535)\n (250.0, 346.41016151377545)\n (300.0, 346.41016151377545)\n (350.0, 346.41016151377545)\n (400.0, 346.41016151377545)\n (450.0, 346.41016151377545)\n (500.0, 346.41016151377545)\n (550.0, 346.41016151377545)\n (600.0, 346.41016151377545)\n (650.0, 346.41016151377545)\n (700.0, 346.41016151377545)\n (275.0, 389.71143170299734)\n (325.0, 389.71143170299734)\n (375.0, 389.71143170299734)\n (425.0, 389.71143170299734)\n (475.0, 389.71143170299734)\n (525.0, 389.71143170299734)\n (575.0, 389.71143170299734)\n (625.0, 389.71143170299734)\n (675.0, 389.71143170299734)\n (725.0, 389.71143170299734)\n (300.0, 433.0127018922193)\n (350.0, 433.0127018922193)\n (400.0, 433.0127018922193)\n (450.0, 433.0127018922193)\n (500.0, 433.0127018922193)\n (550.0, 433.0127018922193)\n (600.0, 433.0127018922193)\n (650.0, 433.0127018922193)\n (700.0, 433.0127018922193)\n (750.0, 433.0127018922193)","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"There exists blockade interactions between hard-core particles. We connect two lattice sites within blockade radius by an edge. Two ends of an edge can not both be occupied by particles.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"blockade_radius = 55\nusing GenericTensorNetworks: show_graph, unit_disk_graph\nusing GenericTensorNetworks.Graphs: edges, nv\ngraph = unit_disk_graph(vec(sites), blockade_radius)\nshow_graph(graph, sites; texts=fill(\"\", length(sites)))","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"(Image: )","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"These constraints defines an independent set problem that characterized by the following energy based model. Let G = (V E) be a graph, where V is the set of vertices and E is the set of edges. The energy model for the hard-core lattice gas problem is","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"E(mathbfn) = -sum_i in Vw_i n_i + U sum_(i j) in E n_i n_j","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"where n_i in 0 1 is the number of particles at site i, and w_i is the weight associated with it. For unweighted graphs, the weights are uniform. U is the repulsive interaction strength between two particles. To represent the independence constraint, we let U = infty, i.e. coexitence of two particles at two sites connected by an edge is completely forbidden. The solution space hard-core lattice gas is equivalent to that of an independent set problem. The independent set problem involves finding a set of vertices in a graph such that no two vertices in the set are adjacent (i.e., there is no edge connecting them). One can create a tensor network based modeling of an independent set problem with package GenericTensorNetworks.jl.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"using GenericTensorNetworks\nproblem = IndependentSet(graph)","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"GenericTensorNetworks.IndependentSet{GenericTensorNetworks.UnitWeight}(Graphs.SimpleGraphs.SimpleGraph{Int64}(261, [[2, 11], [1, 3, 11, 12], [2, 4, 12, 13], [3, 5, 13, 14], [4, 6, 14, 15], [5, 7, 15, 16], [6, 8, 16, 17], [7, 9, 17, 18], [8, 10, 18, 19], [9, 19, 20], [1, 2, 12, 21], [2, 3, 11, 13, 21, 22], [3, 4, 12, 14, 22, 23], [4, 5, 13, 15, 23, 24], [5, 6, 14, 16, 24, 25], [6, 7, 15, 17, 25, 26], [7, 8, 16, 18, 26, 27], [8, 9, 17, 19, 27, 28], [9, 10, 18, 20, 28, 29], [10, 19, 29, 30], [11, 12, 22, 31], [12, 13, 21, 23, 31, 32], [13, 14, 22, 24, 32, 33], [14, 15, 23, 25, 33, 34], [15, 16, 24, 26, 34, 35], [16, 17, 25, 27, 35, 36], [17, 18, 26, 28, 36, 37], [18, 19, 27, 29, 37, 38], [19, 20, 28, 30, 38, 39], [20, 29, 39, 40], [21, 22, 32, 41], [22, 23, 31, 33, 41, 42], [23, 24, 32, 34, 42, 43], [24, 25, 33, 35, 43, 44], [25, 26, 34, 36, 44, 45], [26, 27, 35, 37, 45, 46], [27, 28, 36, 38, 46, 47], [28, 29, 37, 39, 47, 48], [29, 30, 38, 40, 48, 49], [30, 39, 49, 50], [31, 32, 42, 51], [32, 33, 41, 43, 51, 52], [33, 34, 42, 44, 52, 53], [34, 35, 43, 45, 53, 54], [35, 36, 44, 46, 54, 55], [36, 37, 45, 47, 55, 56], [37, 38, 46, 48, 56, 57], [38, 39, 47, 49, 57, 58], [39, 40, 48, 50, 58, 59], [40, 49, 59, 60], [41, 42, 52, 61], [42, 43, 51, 53, 61, 62], [43, 44, 52, 54, 62, 63], [44, 45, 53, 55, 63, 64], [45, 46, 54, 56, 64, 65], [46, 47, 55, 57, 65, 66], [47, 48, 56, 58, 66, 67], [48, 49, 57, 59, 67, 68], [49, 50, 58, 60, 68, 69], [50, 59, 69, 70], [51, 52, 62, 71], [52, 53, 61, 63, 71, 72], [53, 54, 62, 64, 72, 73], [54, 55, 63, 65, 73, 74], [55, 56, 64, 66, 74, 75], [56, 57, 65, 67, 75, 76], [57, 58, 66, 68, 76, 77], [58, 59, 67, 69, 77, 78], [59, 60, 68, 70, 78, 79], [60, 69, 79, 80], [61, 62, 72, 81], [62, 63, 71, 73, 81, 82], [63, 64, 72, 74, 82, 83], [64, 65, 73, 75, 83, 84], [65, 66, 74, 76, 84, 85], [66, 67, 75, 77, 85, 86], [67, 68, 76, 78, 86, 87], [68, 69, 77, 79, 87, 88], [69, 70, 78, 80, 88, 89], [70, 79, 89, 90], [71, 72, 82, 91], [72, 73, 81, 83, 91, 92], [73, 74, 82, 84, 92, 93], [74, 75, 83, 85, 93, 94], [75, 76, 84, 86, 94, 95], [76, 77, 85, 87, 95, 96], [77, 78, 86, 88, 96, 97], [78, 79, 87, 89, 97, 98], [79, 80, 88, 90, 98, 99], [80, 89, 99, 100], [81, 82, 92], [82, 83, 91, 93], [83, 84, 92, 94], [84, 85, 93, 95], [85, 86, 94, 96], [86, 87, 95, 97], [87, 88, 96, 98], [88, 89, 97, 99], [89, 90, 98, 100], [90, 99]]), GenericTensorNetworks.UnitWeight())","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"There are plenty of discussions related to solution space properties in the GenericTensorNetworks documentaion page. In this example, we show how to use TensorInference to use probabilistic inference for understand the finite temperature properties of this statistical model. We use TensorNetworkModel to convert a combinatorial optimization problem to a probabilistic model. Here, we let the inverse temperature be beta = 3.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/#Probabilistic-modeling-correlation-functions","page":"Hard-core Lattice Gas","title":"Probabilistic modeling correlation functions","text":"","category":"section"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"using TensorInference\nβ = 3.0\npmodel = TensorNetworkModel(problem, β)","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"TensorNetworkModel{Int64, OMEinsum.DynamicNestedEinsum{Int64}, Array{Float64}}\nvariables: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100\ncontraction time = 2^18.036, space = 2^11.0, read-write = 2^15.814","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"The partition function of this statistical model can be computed with the probability function.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"partition_func = probability(pmodel)","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"exp(107.42267890429402) * fill(1.0)","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"The default return value is a log-rescaled tensor. Use indexing to get the real value.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"partition_func[]","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"4.4985927541461473e46","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"The marginal probabilities can be computed with the marginals function, which measures how likely a site is occupied.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"mars = marginals(pmodel)\nshow_graph(graph, sites; vertex_colors=[(b = mars[[i]][2]; (1-b, 1-b, 1-b)) for i in 1:nv(graph)], texts=fill(\"\", nv(graph)))","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"(Image: )","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"The can see the sites at the corner is more likely to be occupied. To obtain two-site correlations, one can set the variables to query marginal probabilities manually.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"pmodel2 = TensorNetworkModel(problem, β; mars=[[e.src, e.dst] for e in edges(graph)])\nmars = marginals(pmodel2);","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"We show the probability that both sites on an edge are not occupied","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"show_graph(graph, sites; edge_colors=[(b = mars[[e.src, e.dst]][1, 1]; (1-b, 1-b, 1-b)) for e in edges(graph)], texts=fill(\"\", nv(graph)), config=GraphDisplayConfig(; edge_line_width=5))","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"(Image: )","category":"page"},{"location":"generated/hard-core-lattice-gas/main/#The-most-likely-configuration","page":"Hard-core Lattice Gas","title":"The most likely configuration","text":"","category":"section"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"The MAP and MMAP can be used to get the most likely configuration given an evidence. The relavant function is most_probable_config. If we fix the vertex configuration at one corner to be one, we get the most probably configuration as bellow.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"pmodel3 = TensorNetworkModel(problem, β; evidence=Dict(1=>1))\nmars = marginals(pmodel3)\nlogp, config = most_probable_config(pmodel3)","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"(102.0, [1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1])","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"The log probability is 102. Let us visualize the configuration.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"show_graph(graph, sites; vertex_colors=[(1-b, 1-b, 1-b) for b in config], texts=fill(\"\", nv(graph)))","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"(Image: )","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"The number of particles is","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"sum(config)","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"34","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"Otherwise, we will get a suboptimal configuration.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"pmodel3 = TensorNetworkModel(problem, β; evidence=Dict(1=>0))\nlogp2, config2 = most_probable_config(pmodel)","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"(102.0, [1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1])","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"The log probability is 99, which is much smaller.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"show_graph(graph, sites; vertex_colors=[(1-b, 1-b, 1-b) for b in config2], texts=fill(\"\", nv(graph)))","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"(Image: )","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"The number of particles is","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"sum(config2)\n\n# Sampling configurations","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"34","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"One can ue sample to generate samples from hard-core lattice gas at finite temperature. The return value is a matrix, with the columns correspond to different samples.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"configs = sample(pmodel3, 1000)\nsizes = sum.(configs)\n[count(==(i), sizes) for i=0:34] # counting sizes","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"35-element Vector{Int64}:\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 1\n 0\n 4\n 6\n 16\n 80\n 123\n 174\n 226\n 177\n 127\n 54\n 12\n 0","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"This page was generated using Literate.jl.","category":"page"},{"location":"tensor-networks/#Tensor-networks","page":"Tensor Networks","title":"Tensor networks","text":"","category":"section"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"We now introduce the core ideas of tensor networks, highlighting their connections with probabilistic graphical models (PGM) to align the terminology between them.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"For our purposes, a tensor is equivalent to the concept of a factor as defined in the PGM domain, which we detail more formally below.","category":"page"},{"location":"tensor-networks/#What-is-a-tensor?","page":"Tensor Networks","title":"What is a tensor?","text":"","category":"section"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"Definition: A tensor T is defined as:","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"T prod_V in bmV mathcalD_V rightarrow textttnumber","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"Here, the function T maps each possible instantiation of the random variables in its scope bmV to a generic number type. In the context of tensor networks, a minimum requirement is that the number type is a commutative semiring. To define a commutative semiring with the addition operation oplus and the multiplication operation odot on a set S, the following relations must hold for any arbitrary three elements a b c in S.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"newcommandmymathbb1mathbb1\nbeginalign*\n(a oplus b) oplus c = a oplus (b oplus c) hspace5emtexttriangleright commutative monoid oplus with identity mymathbb0\na oplus mymathbb0 = mymathbb0 oplus a = a \na oplus b = b oplus a \n\n(a odot b) odot c = a odot (b odot c) hspace5emtexttriangleright commutative monoid odot with identity mymathbb1\na odot mymathbb1 = mymathbb1 odot a = a \na odot b = b odot a \n\na odot (boplus c) = aodot b oplus aodot c hspace5emtexttriangleright left and right distributive\n(aoplus b) odot c = aodot c oplus bodot c \n\na odot mymathbb0 = mymathbb0 odot a = mymathbb0\nendalign*","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"Tensors are represented using multidimensional arrays of numbers with labeled dimensions. These labels correspond to the array's indices, which in turn represent the set of random variables that the tensor is a function of. Thus, in this context, the terms label, index, and variable are synonymous and hence used interchangeably.","category":"page"},{"location":"tensor-networks/#What-is-a-tensor-network?","page":"Tensor Networks","title":"What is a tensor network?","text":"","category":"section"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"We now turn our attention to defining a tensor network, a mathematical object used to represent a multilinear map between tensors. This concept is widely employed in fields like condensed matter physics [Orus2014][Pfeifer2014], quantum simulation [Markov2008][Pan2022], and even in solving combinatorial optimization problems [Liu2023]. It's worth noting that we use a generalized version of the conventional notation, most commonly known through the eisnum function, which is commonly used in high-performance computing. Packages that implement this conventional notation include","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"numpy\nOMEinsum.jl\nPyTorch\nTensorFlow","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"This approach allows us to represent a broader range of sum-product multilinear operations between tensors, thus meeting the requirements of the PGM field.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"Definition[Liu2023]: A tensor network is a multilinear map represented by the triple mathcalN = (Lambda mathcalT bmsigma_0), where:","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"Lambda is the set of variables present in the network mathcalN.\nmathcalT = T^(k)_bmsigma_k _k=1^M is the set of input tensors, where each tensor T^(k)_bmsigma_k is identified by a superscript (k) and has an associated scope bmsigma_k.\nbmsigma_0 specifies the scope of the output tensor.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"More specifically, each tensor T^(k)_bmsigma_k in mathcalT is labeled by a string bmsigma_k in Lambda^r left(T^(k) right), where r left(T^(k) right) is the rank of T^(k). The multilinear map, also known as the contraction, applied to this triple is defined as","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"textttcontract(Lambda mathcalT bmsigma_0) = sum_bmsigma_Lambda\nsetminus bmsigma_0 prod_k=1^M T^(k)_bmsigma_k","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"Notably, the summation extends over all instantiations of the variables that are not part of the output tensor.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"As an example, consider matrix multiplication, which can be specified as a tensor network contraction:","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":" (AB)_ik = textttcontractleft(ijk A_ij B_jk ikright)","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"Here, matrices A and B are input tensors labeled by strings ij jk in i j k^2. The output tensor is labeled by string ik. Summations run over indices Lambda setminus ik = j. The contraction corresponds to","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":" textttcontractleft(ijk A_ij B_jk ikright) = sum_j\n A_ijB_jk","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"In the einsum notation commonly used in various programming languages, this is equivalent to ij, jk -> ik.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"Diagrammatically, a tensor network can be represented as an open hypergraph. In this diagram, a tensor maps to a vertex, and a variable maps to a hyperedge. Tensors sharing the same variable are connected by the same hyperedge for that variable. The diagrammatic representation of matrix multiplication is:","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"using TikzPictures\n\ntp = TikzPicture(\n L\"\"\"\n \\matrix[row sep=0.8cm,column sep=0.8cm,ampersand replacement= \\& ] {\n \\node (1) {}; \\&\n \\node (a) [mytensor] {$A$}; \\&\n \\node (b) [mytensor] {$B$}; \\&\n \\node (2) {}; \\&\n \\\\\n };\n \\draw [myedge, color=c01] (1) edge node[below] {$i$} (a);\n \\draw [myedge, color=c02] (a) edge node[below] {$j$} (b);\n \\draw [myedge, color=c03] (b) edge node[below] {$k$} (2);\n \"\"\",\n options=\"every node/.style={scale=2.0}\",\n preamble=\"\\\\input{\" * joinpath(@__DIR__, \"assets\", \"preambles\", \"the-tensor-network\") * \"}\",\n)\nsave(SVG(\"the-tensor-network1\"), tp)","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"In this diagram, we use different colors to denote different hyperedges. Hyperedges for i and j are left open to denote variables in the output string bmsigma_0. The reason we use hyperedges rather than regular edges will become clear in the following star contraction example.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":" textttcontract(ijkl A_il B_jl C_kl ijk) = sum_lA_il\n B_jl C_kl","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"The equivalent einsum notation employed by many programming languages is il, jl, kl -> ijk.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"Since the variable l is shared across all three tensors, a simple graph can't capture the diagram's complexity. The more appropriate hypergraph representation is shown below.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"using TikzPictures\n\ntp = TikzPicture(\n L\"\"\"\n \\matrix[row sep=0.4cm,column sep=0.4cm,ampersand replacement= \\& ] {\n \\&\n \\&\n \\node[color=c01] (j) {$j$}; \\&\n \\&\n \\&\n \\\\\n \\&\n \\&\n \\node (b) [mytensor] {$B$}; \\&\n \\&\n \\&\n \\\\\n \\node[color=c03] (i) {$i$}; \\&\n \\node (a) [mytensor] {$A$}; \\&\n \\node[color=c02] (l) {$l$}; \\&\n \\node (c) [mytensor] {$C$}; \\&\n \\node[color=c04] (k) {$k$}; \\&\n \\\\\n };\n \\draw [myedge, color=c01] (j) edge (b);\n \\draw [myedge, color=c02] (b) edge (l);\n \\draw [myedge, color=c03] (i) edge (a);\n \\draw [myedge, color=c02] (a) edge (l);\n \\draw [myedge, color=c02] (l) edge (c);\n \\draw [myedge, color=c04] (c) edge (k);\n \"\"\",\n options=\"every node/.style={scale=2.0}\",\n preamble=\"\\\\input{\" * joinpath(@__DIR__, \"assets\", \"preambles\", \"the-tensor-network\") * \"}\",\n)\nsave(SVG(\"the-tensor-network2\"), tp)","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"As a final note, our definition of a tensor network allows for repeated indices within the same tensor, which translates to self-loops in their corresponding diagrams.","category":"page"},{"location":"tensor-networks/#Tensor-network-contraction-orders","page":"Tensor Networks","title":"Tensor network contraction orders","text":"","category":"section"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"The performance of a tensor network contraction depends on the order in which the tensors are contracted. The order of contraction is usually specified by binary trees, where the leaves are the input tensors and the internal nodes represent the order of contraction. The root of the tree is the output tensor.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"Numerous approaches have been proposed to determine efficient contraction orderings, which include:","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"Greedy algorithms\nBreadth-first search and Dynamic programming [Pfeifer2014]\nGraph bipartitioning [Gray2021]\nLocal search [Kalachev2021]","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"Some of these have been implemented in the OMEinsum package. Please check Performance Tips for more details.","category":"page"},{"location":"tensor-networks/#References","page":"Tensor Networks","title":"References","text":"","category":"section"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"[Orus2014]: Orús R. A practical introduction to tensor networks: Matrix product states and projected entangled pair states[J]. Annals of physics, 2014, 349: 117-158.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"[Markov2008]: Markov I L, Shi Y. Simulating quantum computation by contracting tensor networks[J]. SIAM Journal on Computing, 2008, 38(3): 963-981.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"[Pfeifer2014]: Pfeifer R N C, Haegeman J, Verstraete F. Faster identification of optimal contraction sequences for tensor networks[J]. Physical Review E, 2014, 90(3): 033315.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"[Gray2021]: Gray J, Kourtis S. Hyper-optimized tensor network contraction[J]. Quantum, 2021, 5: 410.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"[Kalachev2021]: Kalachev G, Panteleev P, Yung M H. Multi-tensor contraction for XEB verification of quantum circuits[J]. arXiv:2108.05665, 2021.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"[Pan2022]: Pan F, Chen K, Zhang P. Solving the sampling problem of the sycamore quantum circuits[J]. Physical Review Letters, 2022, 129(9): 090502.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"[Liu2023]: Liu J G, Gao X, Cain M, et al. Computing solution space properties of combinatorial optimization problems via generic tensor networks[J]. SIAM Journal on Scientific Computing, 2023, 45(3): A1239-A1270.","category":"page"},{"location":"api/public/#Public-API","page":"Public","title":"Public API","text":"","category":"section"},{"location":"api/public/#Index","page":"Public","title":"Index","text":"","category":"section"},{"location":"api/public/","page":"Public","title":"Public","text":"Modules","category":"page"},{"location":"api/public/","page":"Public","title":"Public","text":"Pages = [\"public.md\"]\nOrder = [:module]","category":"page"},{"location":"api/public/","page":"Public","title":"Public","text":"Types","category":"page"},{"location":"api/public/","page":"Public","title":"Public","text":"Pages = [\"public.md\"]\nOrder = [:type]","category":"page"},{"location":"api/public/","page":"Public","title":"Public","text":"Functions","category":"page"},{"location":"api/public/","page":"Public","title":"Public","text":"Pages = [\"public.md\"]\nOrder = [:function]","category":"page"},{"location":"api/public/#Modules","page":"Public","title":"Modules","text":"","category":"section"},{"location":"api/public/","page":"Public","title":"Public","text":"TensorInference","category":"page"},{"location":"api/public/#TensorInference","page":"Public","title":"TensorInference","text":"Main module for TensorInference.jl – A toolbox for probabilistic inference using contraction of tensor networks.\n\nExports\n\nArtifactProblemSpec\nGreedyMethod\nKaHyParBipartite\nMMAPModel\nMergeGreedy\nMergeVectors\nRescaledArray\nSABipartite\nTensorNetworkModel\nTreeSA\nUAIModel\ncontraction_complexity\ndataset_from_artifact\nget_cards\nget_vars\nlog_probability\nmarginals\nmaximum_logp\nmost_probable_config\nprobability\nproblem_from_artifact\nread_evidence\nread_evidence_file\nread_model\nread_model_file\nread_queryvars\nread_solution\nread_td_file\nsample\nupdate_evidence!\nupdate_temperature\n\n\n\n\n\n","category":"module"},{"location":"api/public/#Types","page":"Public","title":"Types","text":"","category":"section"},{"location":"api/public/","page":"Public","title":"Public","text":"GreedyMethod\nKaHyParBipartite\nMergeGreedy\nMergeVectors\nSABipartite\nTreeSA\nMMAPModel\nRescaledArray\nTensorNetworkModel\nArtifactProblemSpec\nUAIModel","category":"page"},{"location":"api/public/#OMEinsumContractionOrders.GreedyMethod","page":"Public","title":"OMEinsumContractionOrders.GreedyMethod","text":"GreedyMethod{MT}\nGreedyMethod(; method=MinSpaceOut(), nrepeat=10)\n\nThe fast but poor greedy optimizer. Input arguments are\n\nmethod is MinSpaceDiff() or MinSpaceOut.\nMinSpaceOut choose one of the contraction that produces a minimum output tensor size,\nMinSpaceDiff choose one of the contraction that decrease the space most.\nnrepeat is the number of repeatition, returns the best contraction order.\n\n\n\n\n\n","category":"type"},{"location":"api/public/#OMEinsumContractionOrders.KaHyParBipartite","page":"Public","title":"OMEinsumContractionOrders.KaHyParBipartite","text":"KaHyParBipartite{RT,IT,GM}\nKaHyParBipartite(; sc_target, imbalances=collect(0.0:0.005:0.8),\n max_group_size=40, greedy_config=GreedyMethod())\n\nOptimize the einsum code contraction order using the KaHyPar + Greedy approach. This program first recursively cuts the tensors into several groups using KaHyPar, with maximum group size specifed by max_group_size and maximum space complexity specified by sc_target, Then finds the contraction order inside each group with the greedy search algorithm. Other arguments are\n\nsc_target is the target space complexity, defined as log2(number of elements in the largest tensor),\nimbalances is a KaHyPar parameter that controls the group sizes in hierarchical bipartition,\nmax_group_size is the maximum size that allowed to used greedy search,\ngreedy_config is a greedy optimizer.\n\nReferences\n\nHyper-optimized tensor network contraction\nSimulating the Sycamore quantum supremacy circuits\n\n\n\n\n\n","category":"type"},{"location":"api/public/#OMEinsumContractionOrders.MergeGreedy","page":"Public","title":"OMEinsumContractionOrders.MergeGreedy","text":"MergeGreedy <: CodeSimplifier\nMergeGreedy(; threshhold=-1e-12)\n\nContraction code simplifier (in order to reduce the time of calling optimizers) that merges tensors greedily if the space complexity of merged tensors is reduced (difference smaller than the threshhold).\n\n\n\n\n\n","category":"type"},{"location":"api/public/#OMEinsumContractionOrders.MergeVectors","page":"Public","title":"OMEinsumContractionOrders.MergeVectors","text":"MergeVectors <: CodeSimplifier\nMergeVectors()\n\nContraction code simplifier (in order to reduce the time of calling optimizers) that merges vectors to closest tensors.\n\n\n\n\n\n","category":"type"},{"location":"api/public/#OMEinsumContractionOrders.SABipartite","page":"Public","title":"OMEinsumContractionOrders.SABipartite","text":"SABipartite{RT,BT}\nSABipartite(; sc_target=25, ntrials=50, βs=0.1:0.2:15.0, niters=1000\n max_group_size=40, greedy_config=GreedyMethod(), initializer=:random)\n\nOptimize the einsum code contraction order using the Simulated Annealing bipartition + Greedy approach. This program first recursively cuts the tensors into several groups using simulated annealing, with maximum group size specifed by max_group_size and maximum space complexity specified by sc_target, Then finds the contraction order inside each group with the greedy search algorithm. Other arguments are\n\nsize_dict, a dictionary that specifies leg dimensions,\nsc_target is the target space complexity, defined as log2(number of elements in the largest tensor),\nmax_group_size is the maximum size that allowed to used greedy search,\nβs is a list of inverse temperature 1/T,\nniters is the number of iteration in each temperature,\nntrials is the number of repetition (with different random seeds),\ngreedy_config configures the greedy method,\ninitializer, the partition configuration initializer, one can choose :random or :greedy (slow but better).\n\nReferences\n\nHyper-optimized tensor network contraction\n\n\n\n\n\n","category":"type"},{"location":"api/public/#OMEinsumContractionOrders.TreeSA","page":"Public","title":"OMEinsumContractionOrders.TreeSA","text":"TreeSA{RT,IT,GM}\nTreeSA(; sc_target=20, βs=collect(0.01:0.05:15), ntrials=10, niters=50,\n sc_weight=1.0, rw_weight=0.2, initializer=:greedy, greedy_config=GreedyMethod(; nrepeat=1))\n\nOptimize the einsum contraction pattern using the simulated annealing on tensor expression tree.\n\nsc_target is the target space complexity,\nntrials, βs and niters are annealing parameters, doing ntrials indepedent annealings, each has inverse tempteratures specified by βs, in each temperature, do niters updates of the tree.\nsc_weight is the relative importance factor of space complexity in the loss compared with the time complexity.\nrw_weight is the relative importance factor of memory read and write in the loss compared with the time complexity.\ninitializer specifies how to determine the initial configuration, it can be :greedy or :random. If it is using :greedy method to generate the initial configuration, it also uses two extra arguments greedy_method and greedy_nrepeat.\nnslices is the number of sliced legs, default is 0.\nfixed_slices is a vector of sliced legs, default is [].\n\nReferences\n\nRecursive Multi-Tensor Contraction for XEB Verification of Quantum Circuits\n\n\n\n\n\n","category":"type"},{"location":"api/public/#TensorInference.MMAPModel","page":"Public","title":"TensorInference.MMAPModel","text":"struct MMAPModel{LT, AT<:AbstractArray}\n\nComputing the most likely assignment to the query variables, Xₘ ⊆ X after marginalizing out the remaining variables Xₛ = X \\ Xₘ.\n\nrm MMAP(X_iE=e) = arg max_X_M sum_X_S prod_F f(x_M x_S e)\n\nFields\n\nvars is the query variables in the tensor network.\ncode is the tropical tensor network contraction pattern.\ntensors is the tensors fed into the tensor network.\nclusters is the clusters, each element of this cluster is a TensorNetworkModel instance for marginalizing certain variables.\nevidence is a dictionary to specifiy degree of freedoms fixed to certain values, which should not have overlap with the query variables.\n\n\n\n\n\n","category":"type"},{"location":"api/public/#TensorInference.RescaledArray","page":"Public","title":"TensorInference.RescaledArray","text":"struct RescaledArray{T, N, AT<:AbstractArray{T, N}} <: AbstractArray{T, N}\n\nRescaledArray(α, T) -> RescaledArray\n\nAn array data type with a log-prefactor, and a l∞-normalized storage, i.e. the maximum element in a tensor is 1. This tensor type can avoid the potential underflow/overflow of numbers in a tensor network. The constructor RescaledArray(α, T) creates a rescaled array that equal to exp(α) * T.\n\n\n\n\n\n","category":"type"},{"location":"api/public/#TensorInference.TensorNetworkModel","page":"Public","title":"TensorInference.TensorNetworkModel","text":"struct TensorNetworkModel{LT, ET, MT<:AbstractArray}\n\nProbabilistic modeling with a tensor network.\n\nFields\n\nvars are the degrees of freedom in the tensor network.\ncode is the tensor network contraction pattern.\ntensors are the tensors fed into the tensor network, the leading tensors are unity tensors associated with mars.\nevidence is a dictionary used to specify degrees of freedom that are fixed to certain values.\nmars is a vector, each element is a vector of variables to compute marginal probabilities.\n\n\n\n\n\n","category":"type"},{"location":"api/public/#TensorInference.ArtifactProblemSpec","page":"Public","title":"TensorInference.ArtifactProblemSpec","text":"struct ArtifactProblemSpec\n\nSpecify the UAI models from the artifacts. It can be used as the input of read_model.\n\nFields\n\nartifact_path::String\ntask::String\nproblem_set::String\nproblem_id::Int64\n\n\n\n\n\n","category":"type"},{"location":"api/public/#TensorInference.UAIModel","page":"Public","title":"TensorInference.UAIModel","text":"struct UAIModel{ET, FT<:(TensorInference.Factor{ET})}\n\nFields\n\nnvars is the number of variables,\ncards is a vector of cardinalities for variables,\nfactors is a vector of factors,\n\n\n\n\n\n","category":"type"},{"location":"api/public/#Functions","page":"Public","title":"Functions","text":"","category":"section"},{"location":"api/public/","page":"Public","title":"Public","text":"contraction_complexity\nget_cards\nget_vars\nlog_probability\nmarginals\nmaximum_logp\nmost_probable_config\nprobability\ndataset_from_artifact\nproblem_from_artifact\nread_model\nread_evidence\nread_solution\nread_queryvars\nread_model_file\nread_evidence_file\nread_td_file\nsample\nupdate_evidence!\nupdate_temperature","category":"page"},{"location":"api/public/#OMEinsumContractionOrders.contraction_complexity","page":"Public","title":"OMEinsumContractionOrders.contraction_complexity","text":"contraction_complexity(tensor_network)\n\nReturns the contraction complexity of a tensor newtork model.\n\n\n\n\n\ncontraction_complexity(eincode, size_dict) -> ContractionComplexity\n\nReturns the time, space and read-write complexity of the einsum contraction. The returned object contains 3 fields:\n\ntime complexity tc defined as log2(number of element-wise multiplications).\nspace complexity sc defined as log2(size of the maximum intermediate tensor).\nread-write complexity rwc defined as log2(the number of read-write operations).\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.get_cards","page":"Public","title":"TensorInference.get_cards","text":"get_cards(tn::TensorNetworkModel; fixedisone) -> Vector\n\n\nGet the cardinalities of variables in this tensor network.\n\n\n\n\n\nget_cards(mmap::MMAPModel; fixedisone) -> Vector\n\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.get_vars","page":"Public","title":"TensorInference.get_vars","text":"get_vars(tn::TensorNetworkModel) -> Vector\n\n\nGet the variables in this tensor network, they are also known as legs, labels, or degree of freedoms.\n\n\n\n\n\nget_vars(mmap::MMAPModel) -> Vector\n\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.log_probability","page":"Public","title":"TensorInference.log_probability","text":"log_probability(\n tn::TensorNetworkModel,\n config::Union{Dict, AbstractVector}\n) -> Real\n\n\nEvaluate the log probability (or partition function) of config.\n\n\n\n\n\nlog_probability(\n tn::TensorNetworkModel;\n usecuda\n) -> AbstractArray\n\n\nEvaluate the log probability (or partition function). It is the logged version of probability, which is less likely to overflow.\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.marginals","page":"Public","title":"TensorInference.marginals","text":"marginals(\n tn::TensorNetworkModel;\n usecuda,\n rescale\n) -> Dict{Vector{Int64}}\n\n\nQueries the marginals of the variables in a TensorNetworkModel. The function returns a dictionary, where the keys are the variables and the values are their respective marginals. A marginal is a probability distribution over a subset of variables, obtained by integrating or summing over the remaining variables in the model. By default, the function returns the marginals of all individual variables. To specify which marginal variables to query, set the mars field when constructing a TensorNetworkModel. Note that the choice of marginal variables will affect the contraction order of the tensor network.\n\nArguments\n\ntn: The TensorNetworkModel to query.\nusecuda: Specifies whether to use CUDA for tensor contraction.\nrescale: Specifies whether to rescale the tensors during contraction.\n\nExample\n\nThe following example is taken from examples/asia-network/main.jl.\n\njulia> model = read_model_file(pkgdir(TensorInference, \"examples\", \"asia-network\", \"model.uai\"));\n\njulia> tn = TensorNetworkModel(model; evidence=Dict(1=>0))\nTensorNetworkModel{Int64, DynamicNestedEinsum{Int64}, Array{Float64}}\nvariables: 1 (evidence → 0), 2, 3, 4, 5, 6, 7, 8\ncontraction time = 2^6.022, space = 2^2.0, read-write = 2^7.077\n\njulia> marginals(tn)\nDict{Vector{Int64}, Vector{Float64}} with 8 entries:\n [8] => [0.450138, 0.549863]\n [3] => [0.5, 0.5]\n [1] => [1.0]\n [5] => [0.45, 0.55]\n [4] => [0.055, 0.945]\n [6] => [0.10225, 0.89775]\n [7] => [0.145092, 0.854908]\n [2] => [0.05, 0.95]\n\njulia> tn2 = TensorNetworkModel(model; evidence=Dict(1=>0), mars=[[2, 3], [3, 4]])\nTensorNetworkModel{Int64, DynamicNestedEinsum{Int64}, Array{Float64}}\nvariables: 1 (evidence → 0), 2, 3, 4, 5, 6, 7, 8\ncontraction time = 2^7.781, space = 2^5.0, read-write = 2^8.443\n\njulia> marginals(tn2)\nDict{Vector{Int64}, Matrix{Float64}} with 2 entries:\n [2, 3] => [0.025 0.025; 0.475 0.475]\n [3, 4] => [0.05 0.45; 0.005 0.495]\n\nIn this example, we first set the evidence for variable 1 to 0 and then query the marginals of all individual variables. The returned dictionary has keys that correspond to the queried variables and values that represent their marginals. These marginals are vectors, with each entry corresponding to the probability of the variable taking a specific value. In this example, the possible values are 0 or 1. For the evidence variable 1, the marginal is always [1.0] since its value is fixed at 0.\n\nNext, we specify the marginal variables to query as variables 2 and 3, and variables 3 and 4, respectively. The joint marginals may or may not affect the contraction time and space. In this example, the contraction space complexity increases from 2^{2.0} to 2^{5.0}, and the contraction time complexity increases from 2^{5.977} to 2^{7.781}. The output marginals are the joint probabilities of the queried variables, represented by tensors.\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.maximum_logp","page":"Public","title":"TensorInference.maximum_logp","text":"maximum_logp(\n tn::TensorNetworkModel;\n usecuda\n) -> AbstractArray{<:Real}\n\n\nReturns an output array containing largest log-probabilities.\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.most_probable_config","page":"Public","title":"TensorInference.most_probable_config","text":"most_probable_config(\n tn::TensorNetworkModel;\n usecuda\n) -> Tuple{Real, Vector}\n\n\nReturns the largest log-probability and the most probable configuration.\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.probability","page":"Public","title":"TensorInference.probability","text":"probability(\n tn::TensorNetworkModel;\n usecuda,\n rescale\n) -> AbstractArray\n\n\nContract the tensor network and return an array of probability of evidence. Precisely speaking, the return value is the partition function, which may not be l1-normalized.\n\nIf the openvars of the input tensor networks is zero, the array rank is zero. Otherwise, the return values corresponds to marginal probabilities.\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.dataset_from_artifact","page":"Public","title":"TensorInference.dataset_from_artifact","text":"dataset_from_artifact(\n artifact_name::AbstractString\n) -> Dict{String, Dict{String, Dict{Int64, ArtifactProblemSpec}}}\n\n\nHelper function that captures the problem names that belong to problem_set for the given task.\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.problem_from_artifact","page":"Public","title":"TensorInference.problem_from_artifact","text":"problem_from_artifact(\n artifact_name::String,\n task::String,\n problem_set::String,\n problem_id::Int64\n) -> ArtifactProblemSpec\n\n\nGet artifact from artifact name, task name, problem set name and problem id.\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.read_model","page":"Public","title":"TensorInference.read_model","text":"read_model(problem::ArtifactProblemSpec; eltype) -> UAIModel\n\n\nRead an UAI model from an artifact.\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.read_evidence","page":"Public","title":"TensorInference.read_evidence","text":"read_evidence(\n problem::ArtifactProblemSpec\n) -> Dict{Int64, Int64}\n\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.read_solution","page":"Public","title":"TensorInference.read_solution","text":"read_solution(\n problem::ArtifactProblemSpec;\n factor_eltype\n) -> Any\n\n\nReturn the solution in the artifact.\n\nThe UAI file formats are defined in: https://uaicompetition.github.io/uci-2022/file-formats/\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.read_queryvars","page":"Public","title":"TensorInference.read_queryvars","text":"read_queryvars(\n problem::ArtifactProblemSpec\n) -> Vector{Int64}\n\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.read_model_file","page":"Public","title":"TensorInference.read_model_file","text":"read_model_file(\n model_filepath::AbstractString;\n factor_eltype\n) -> UAIModel\n\n\nParse the problem instance found in model_filepath defined in the UAI model format. If the provided file path is empty, return nothing.\n\nThe UAI file formats are defined in: https://uaicompetition.github.io/uci-2022/file-formats/\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.read_evidence_file","page":"Public","title":"TensorInference.read_evidence_file","text":"read_evidence_file(\n evidence_filepath::AbstractString\n) -> Tuple{Vector{Int64}, Vector{Int64}}\n\n\nReturn the observed variables and values in evidence_filepath. If the passed file path is an empty string, return empty vectors.\n\nThe UAI file formats are defined in: https://uaicompetition.github.io/uci-2022/file-formats/\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.read_td_file","page":"Public","title":"TensorInference.read_td_file","text":"read_td_file(\n td_filepath::AbstractString\n) -> Tuple{Int64, Int64, Int64, Vector{Vector{Int64}}, Vector{Vector{Int64}}}\n\n\nParse a tree decomposition instance described the PACE format.\n\nThe PACE file format is defined in: https://pacechallenge.org/2017/treewidth/\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.sample","page":"Public","title":"TensorInference.sample","text":"sample(\n tn::TensorNetworkModel,\n n::Int64;\n usecuda\n) -> TensorInference.Samples\n\n\nGenerate samples from a tensor network based probabilistic model. Returns a vector of vector, each element being a configurations defined on get_vars(tn).\n\nArguments\n\ntn is the tensor network model.\nn is the number of samples to be returned.\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.update_evidence!","page":"Public","title":"TensorInference.update_evidence!","text":"update_evidence!(\n tnet::TensorNetworkModel,\n evidence::Dict\n) -> TensorNetworkModel\n\n\nUpdate the evidence of a tensor network model, without changing the set of observed variables!\n\nArguments\n\ntnet is the TensorNetworkModel instance.\nevidence is the new evidence, the keys must be a subset of existing evidence.\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.update_temperature","page":"Public","title":"TensorInference.update_temperature","text":"update_temperature(\n tnet::TensorNetworkModel,\n problem::GenericTensorNetworks.GraphProblem,\n β::Real\n) -> TensorNetworkModel\n\n\nUpdate the temperature of a tensor network model. The program will regenerate tensors from the problem, without repeated optimizing the contraction order.\n\nArguments\n\ntnet is the TensorNetworkModel instance.\nproblem is the target constraint satisfiability problem.\nβ is the inverse temperature.\n\n\n\n\n\n","category":"function"},{"location":"uai-file-formats/#UAI-file-formats","page":"UAI file formats","title":"UAI file formats","text":"","category":"section"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"The UAI Format consists of four potential parts (each associated with a file):","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"Model file format (.uai).\nEvidence file format (.evid).\nQuery file format (.query).\nResults file format (.MAR, .MAP, .MMAP .PR).","category":"page"},{"location":"uai-file-formats/#Model-file-format-(.uai)","page":"UAI file formats","title":"Model file format (.uai)","text":"","category":"section"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"We use the simple text file format specified below to describe problem instances (Markov networks). The format is a generalization of the Ergo file format initially developed by Noetic Systems Inc. for their Ergo software. We use the .uai suffix for the evaluation benchmark network files.","category":"page"},{"location":"uai-file-formats/#Structure","page":"UAI file formats","title":"Structure","text":"","category":"section"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"A file in the UAI format consists of the following two parts, in that order:","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"\n\n","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"The contents of each section (denoted <...> above) are described in the following:","category":"page"},{"location":"uai-file-formats/#Preamble","page":"UAI file formats","title":"Preamble","text":"","category":"section"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"Our description of the format will follow a simple Markov network with three variables and two functions. A sample preamble for such a network is:","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"MARKOV\n3\n2 2 3\n2\n2 0 1\n2 1 2","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"The preamble starts with one line denoting the type of network. Generally, this can be either BAYES (if the network is a Bayesian network) or MARKOV (in case of a Markov network). However, note that this year all networks will be given in a Markov networks (i.e. Bayesian networks will be moralized).","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"The second line contains the number of variables. The next line specifies the cardinalities of each variable, one at a time, separated by a whitespace (note that this implies an order on the variables which will be used throughout the file). The fourth line contains only one integer, denoting the number of cliques in the problem. Then, one clique per line, the scope of each clique is given as follows: The first integer in each line specifies the number of variables in the clique, followed by the actual indexes of the variables. The order of this list is not restricted. Note that the ordering of variables within a factor will follow the order provided here.","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"Referring to the example above, the first line denotes the Markov network, the second line tells us the problem consists of three variables, let's refer to them as X, Y, and Z. Their cardinalities are 2, 2, and 3 respectively (from the third line). Line four specifies that there are 2 cliques. The first clique is X,Y, while the second clique is Y,Z. Note that variables are indexed starting with 0.","category":"page"},{"location":"uai-file-formats/#Function-tables","page":"UAI file formats","title":"Function tables","text":"","category":"section"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"In this section each factor is specified by giving its full table (i.e, specifying value for each assignment). The order of the factor is identical to the one in which they were introduced in the preamble, the first variable have the role of the 'most significant' digit. For each factor table, first the number of entries is given (this should be equal to the product of the domain sizes of the variables in the scope). Then, one by one, separated by whitespace, the values for each assignment to the variables in the function's scope are enumerated. Tuples are implicitly assumed in ascending order, with the last variable in the scope as the 'least significant'. To illustrate, we continue with our Markov network example from above, let's assume the following conditional probability tables:","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"X P(X)\n0 0.436\n1 0.564\n\nX Y P(Y,X)\n0 0 0.128\n0 1 0.872\n1 0 0.920\n1 1 0.080\n\nY Z P(Z,Y)\n0 0 0.210\n0 1 0.333\n0 2 0.457\n1 0 0.811\n1 1 0.000\n1 2 0.189","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"The corresponding function tables in the file would then look like this:","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"2\n0.436 0.564\n\n4\n0.128 0.872\n0.920 0.080\n\n6\n0.210 0.333 0.457\n0.811 0.000 0.189","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"(Note that line breaks and empty lines are effectively just a whitespace, exactly like plain spaces \" \". They are used here to improve readability.)","category":"page"},{"location":"uai-file-formats/#Summary","page":"UAI file formats","title":"Summary","text":"","category":"section"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"To sum up, a problem file consists of 2 sections: the preamble and the full the function tables, the names and the labels. For our Markov network example above, the full file will look like:","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"MARKOV\n3\n2 2 3\n3\n1 0\n2 0 1\n2 1 2\n\n2\n0.436 0.564\n\n4\n0.128 0.872\n0.920 0.080\n\n6\n0.210 0.333 0.457\n0.811 0.000 0.189","category":"page"},{"location":"uai-file-formats/#Evidence-file-format-(.evid)","page":"UAI file formats","title":"Evidence file format (.evid)","text":"","category":"section"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"Evidence is specified in a separate file. This file has the same name as the original network file but with an added .evid suffix. For instance, problem.uai will have evidence in problem.uai.evid. The file starts with a line specifying the number of evidences samples. The evidence in each sample, will be written in a new line. Each line will begin with the number of observed variables in the sample, followed by pairs of variable and its observed value. The indexes correspond to the ones implied by the original problem file. If, for our above example, we want to provide a single sample where the variable Y has been observed as having its first value and Z with its second value, the file example.uai.evid would contain the following:","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"1\n2 1 0 2 1","category":"page"},{"location":"uai-file-formats/#Query-file-format-(.query)","page":"UAI file formats","title":"Query file format (.query)","text":"","category":"section"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"Query variables for marginal MAP (MMAP) inference are specified in a separate file. This file has the same name as the original network file but with an added .query suffix. For instance with respect to the UAI model format, problem.uai will have evidence in problem.uai.query.","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"The query file consists of a single line. The line will begin with the number of query variables, followed by the indexes of the query variables. The indexes correspond to the ones implied by the original problem file.","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"For our example Markov network given Model Format, if we wanted to use Y as the query variable the file example.uai.query would contain the following:","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"1 1","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"As a second example, if variables with indices 0, 4, 8 and 17 are query variables, the query file would contain the following:","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"4 0 4 8 17","category":"page"},{"location":"uai-file-formats/#Results-file-format-(.MAR,-.MAP,-.MMAP-.PR)","page":"UAI file formats","title":"Results file format (.MAR, .MAP, .MMAP .PR)","text":"","category":"section"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"The rest of the file will contain the solution for the task. The first line must contain one of the tasks (PR, MPE, MAR, MMAP, or MLC) solved.","category":"page"},{"location":"uai-file-formats/#Marginals,-MAR","page":"UAI file formats","title":"Marginals, MAR","text":"","category":"section"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"A space separated line that includes:","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"The number of variables in the model.\nA list of marginal approximations of all the variables. For each variable its cardinality is first stated, then the probability of each state is stated. The order of the variables is the same as in the model, all data is space separated.\nFor example, a model with 3 variables, with cardinalities of 2, 2, 3 respectively. The solution might look like this:\n 3 2 0.1 0.9 2 0.3 0.7 3 0.2 0.2 0.6","category":"page"},{"location":"uai-file-formats/#Marginal-MAP,-MMAP","page":"UAI file formats","title":"Marginal MAP, MMAP","text":"","category":"section"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"A space separated line that includes:","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"The number of query variables.\nthe most probable instantiation, a list of variable value pairs for all bmQ variables.\nFor example, if the solution is an assignment of 0, 1 and 0 to three query variables indexed by 2 3 and 4 respectively, the solution will look as follows:\n 3 2 0 3 1 4 0","category":"page"},{"location":"uai-file-formats/#Partition-function,-PR","page":"UAI file formats","title":"Partition function, PR","text":"","category":"section"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"Line with the value of the log_10 of the partition function.","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"For example, an approximation log_10 Pr(bme) = -02008, which is known to be an upper bound may have a solution line:","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"-0.2008","category":"page"},{"location":"probabilistic-inference/#Probabilistic-inference","page":"Probabilistic Inference","title":"Probabilistic inference","text":"","category":"section"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"TensorInference implements efficient methods to perform Bayesian inference in probabilistic graphical models, such as Bayesian Networks or Markov random fields. This page introduces probabilistic graphical models, provides an example using a Bayesian network, and explains what probabilistic inference is, including the different tasks it can involve.","category":"page"},{"location":"probabilistic-inference/#Probabilistic-graphical-models","page":"Probabilistic Inference","title":"Probabilistic graphical models","text":"","category":"section"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"A probabilistic graphical model (PGM) is a mathematical framework that uses graphs to compactly represent complex multivariate statistical distributions. They are used to reason in the presence of uncertainty. This reasoning process is known as probabilistic inference and will be defined and discussed in detail later on.","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"Bayesian networks and Markov random fields are popular types of PGMs. The following PGM is an example of a Bayesian network called the ASIA network. It was introduced by Lauritzen in 1988 [lauritzen1988local].","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"using TikzPictures\n\ntp = TikzPicture(\n L\"\"\"\n % The various elements are conveniently placed using a matrix:\n \\matrix[row sep=0.5cm,column sep=0.5cm] {\n % First line\n \\node (a) [myvar] {$A$}; &\n &\n &\n \\node (s) [myvar] {$S$}; &\n \\\\\n % Second line\n \\node (t) [myvar] {$T$}; &\n &\n \\node (l) [myvar] {$L$}; &\n &\n \\node (b) [myvar] {$B$}; \\\\\n % Third line\n &\n \\node (e) [myvar] {$E$}; &\n &\n &\n \\\\\n % Forth line\n \\node (x) [myvar] {$X$}; &\n &\n &\n \\node (d) [myvar] {$D$}; &\n \\\\\n };\n\n \\draw [myarrow] (a) edge (t);\n \\draw [myarrow] (s) edge (l);\n \\draw [myarrow] (s) edge (b);\n \\draw [myarrow] (t) edge (e);\n \\draw [myarrow] (l) edge (e);\n \\draw [myarrow] (e) edge (x);\n \\draw [myarrow] (e) edge (d);\n \\draw [myarrow] (b) edge (d);\n \"\"\",\n options=\"every node/.style={scale=1.5}\",\n preamble=\"\\\\input{\" * joinpath(@__DIR__, \"assets\", \"preambles\", \"asia-network\") * \"}\",\n)\nsave(SVG(joinpath(@__DIR__, \"asia-bayesian-network\")), tp)","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"(Image: )","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"Random variable Meaning\nA Recent trip to Asia\nT Patient has tuberculosis\nS Patient is a smoker\nL Patient has lung cancer\nB Patient has bronchitis\nE Patient hast T and/or L\nX Chest X-Ray is positive\nD Patient has dyspnoea","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"This network represents a simplified example from the realm of medical diagnosis, illustrating the probabilistic relationships between various random variables that correspond to potential diseases, symptoms, risk factors, and test results. It comprises a graph G = (bmVmathcalE) and a probability distribution P(bmV), where G is a directed acyclic graph, bmV represents the set of variables, and mathcalE is the set of edges connecting these variables. We assume all variables are discrete. Each variable V is quantified by a conditional probability distribution (CPD) P(V mid pa(V)), where pa(V) denotes the parent variables of V. Collectively, these conditional probability distributions, together with the graph G, induce a joint probability distribution over P(bmV), given by","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"P(bmV) = prod_VinbmV P(V mid pa(V))","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"A factor, denoted as phi_bmV, is defined over a set of variables bmV. It's a function that maps each instantiation bmV = bmv to a non-negative number. It's important to note that a probability distribution is a specific case of a factor. The product of two factors, phi_bmX and phi_bmY, is another factor, phi_bmZ, where bmZ = bmX cup bmY, and phi_bmZ(bmz) = phi_bmX(bmx)phi_bmY(bmy) for the instantiations bmx and bmy that align with the instantiation bmz. The marginalization of a factor phi_bmY into bmX subseteq bmY results in a new factor phi_bmX, where each phi_bmX(bmx) is calculated by summing the values of phi_bmY(bmy) for all bmy that are consistent with bmx. Importantly, factor marginalization and product operations form the fundamental basis for conducting probabilistic inference in PGMs.","category":"page"},{"location":"probabilistic-inference/#The-inference-tasks","page":"Probabilistic Inference","title":"The inference tasks","text":"","category":"section"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"Probabilistic inference is the process of determining the probability distribution of a set of unknown variables, given the values of known variables in a PGM. It encompasses several tasks that will be explained next.","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"Each task is performed with respect to a graphical model, denoted as G = bmV bmD bmphi, where:","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"bmV = V_1 V_2 dots V_N is the set of the model’s variables","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"bmD = D_V_1 D_V_2 dots D_V_N is the set of discrete domains for each variable, and","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"bmphi = phi_1 phi_2 dots phi_N is the set of factors that define the joint probability distribution of the model.","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"The variable set bmV can be further partitioned into two subsets: the evidence variables bmE and the remaining variables bmV^prime = bmV setminus bmE. Furthermore, within the set bmV^prime, the subset bmQ denotes the query variables. These are the variables for which we aim to estimate or infer values.","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"using TikzPictures\n\ntp = TikzPicture(\n L\"\"\"\n %\\draw[help lines] (0,0) grid (10,-7);\n\n % mrv: the \"node distances\" refer to the distance between the edge of a shape\n % to the edge of the other shape. That is why I use \"ie_aux\" and \"tasks_aux\"\n % below: to have equal distances between nodes with respect to the center of\n % the shapes.\n\n % row 1\n \\node[myroundbox] (rv) {Random Variables\\\\$\\bm{V}$};\n \\node[right=of rv](aux1) {};\n \\node[right=of aux1,myroundbox] (jd) {Joint Distribution\\\\$P(\\bm{V})$};\n \\node[right=of jd](aux2) {};\n \\node[right=of aux2,myroundbox] (e) {Evidence\\\\$\\bm{E=e}$};\n \\node[right=of e](aux3) {};\n \\node[right=of aux3,myroundbox] (qv) {Query Variables\\\\$\\bm{Q}$};\n % row 2\n \\node[below=of aux2,myrectbox] (ie) {Inference Engine};\n \\node[below=of aux2] (ie_aux) {};\n % row 3\n \\node[below=of ie_aux] (tasks_aux) {};\n \\node[left=of tasks_aux,myroundbox] (mar) {MAR};\n \\node[left=of mar] (aux4) {};\n \\node[left=of aux4,myroundbox] (pr) {PR};\n \\node[right=of tasks_aux,myroundbox] (map) {MAP};\n \\node[right=of map] (aux5) {};\n \\node[right=of aux5,myroundbox] (mmap) {MMAP};\n % row 0\n \\node[above=of aux2,yshift=-12mm,text=gray] (in) {\\textbf{Input}};\n % row 4\n \\node[below=of tasks_aux,yshift=7mm,text=gray] (out) {\\textbf{Output}};\n\n %% edges\n \\draw[myarrow] (rv) -- (ie);\n \\draw[myarrow] (jd) -- (ie);\n \\draw[myarrow] (e) -- (ie);\n \\draw[myarrow] (qv) -- (ie);\n \\draw[myarrow] (ie) -- (pr);\n \\draw[myarrow] (ie) -- (mar);\n \\draw[myarrow] (ie) -- (map);\n \\draw[myarrow] (ie) -- (mmap);\n \"\"\",\n options=\"transform shape, scale=1.8\",\n preamble=\"\\\\input{\" * joinpath(@__DIR__, \"assets\", \"preambles\", \"the-inference-tasks\") * \"}\",\n)\nsave(SVG(\"the-inference-tasks\"), tp)","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"(Image: )","category":"page"},{"location":"probabilistic-inference/#Probability-of-evidence-(PR)","page":"Probabilistic Inference","title":"Probability of evidence (PR)","text":"","category":"section"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"Computing the partition function (ie. normalizing constant) or probability of evidence:","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"PR(bmV^prime mid bmE=bme) = sum_V^prime in bmV^prime prod_phi in bmphi phi(V^primebme)","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"This task involves calculating the probability of the observed evidence, which can be useful for model comparison or anomaly detection. This involves summing the joint probability over all possible states of the unobserved variables in the model, given some observed variables. This is a fundamental task in Bayesian statistics and is often used as a stepping stone for other types of inference.","category":"page"},{"location":"probabilistic-inference/#Marginal-inference-(MAR):","page":"Probabilistic Inference","title":"Marginal inference (MAR):","text":"","category":"section"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"Computing the marginal probability distribution over all variables given evidence:","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"MAR(V_i mid bmE=bme) = frac sum_V^primeprime in bmV^prime\nsetminus V_i prod_phi in bmphi phi(V^primeprimebme) \n PR(bmV^prime mid bmE=bme) ","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"This task involves computing the marginal probability of a subset of variables, integrating out the others. In other words, it computes the probability distribution of some variables of interest regardless of the states of all other variables. This is useful when we're interested in the probabilities of some specific variables in the model, but not the entire model.","category":"page"},{"location":"probabilistic-inference/#Maximum-a-Posteriori-Probability-estimation-(MAP)","page":"Probabilistic Inference","title":"Maximum a Posteriori Probability estimation (MAP)","text":"","category":"section"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"Computing the most likely assignment to all variables given evidence:","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"MAP(V_i mid bmE=bme) = arg max_V^prime in bmV^prime\nprod_phi in bmphi phi(V^primebme)","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"In the MAP task, given some observed variables, the goal is to find the most probable assignment of values to some subset of the unobserved variables. It provides the states of variables that maximize the posterior probability given some observed evidence. This is often used when we want the most likely explanation or prediction according to the model.","category":"page"},{"location":"probabilistic-inference/#Marginal-Maximum-a-Posteriori-(MMAP)","page":"Probabilistic Inference","title":"Marginal Maximum a Posteriori (MMAP)","text":"","category":"section"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"Computing the most likely assignment to the query variables, bmQ subset bmV^prime after marginalizing out the remaining variables bmZ = bmV^prime setminus bmQ, also known as hidden or latent variables:","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"MMAP(V_i mid bmE=e) = arg max_Q in bmQ sum_Z in bmZ\nprod_phi in bmphi phi(Q Z e)","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"This task is essentially a combination of the MAR and MAP tasks. The MMAP task involves finding the most probable assignment (the MAP estimate) for a subset of the variables, while marginalizing over (summing out) the remaining variables. This task is useful when we want to know the most likely state of some variables, but there's some uncertainty over others that we need to average out.","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"[lauritzen1988local]: Steffen L Lauritzen and David J Spiegelhalter. Local computations with probabilities on graphical structures and their application to expert systems. Journal of the Royal Statistical Society: Series B (Methodological), 50(2):157–194, 1988.","category":"page"},{"location":"examples-overview/#Examples","page":"Overview","title":"Examples","text":"","category":"section"},{"location":"examples-overview/","page":"Overview","title":"Overview","text":"Pages = [\n \"generated/asia-network/main.md\",\n \"generated/hard-core-lattice-gas/main.md\",\n]\nDepth = 1","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"EditURL = \"performance-tips.jl\"","category":"page"},{"location":"generated/performance-tips/#Performance-Tips","page":"Performance tips","title":"Performance Tips","text":"","category":"section"},{"location":"generated/performance-tips/#Optimize-contraction-orders","page":"Performance tips","title":"Optimize contraction orders","text":"","category":"section"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"Let us use a problem instance from the \"Promedus\" dataset of the UAI 2014 competition as an example.","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"using TensorInference\nproblem = problem_from_artifact(\"uai2014\", \"MAR\", \"Promedus\", 11)\nmodel, evidence = read_model(problem), read_evidence(problem);","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"Next, we select the tensor network contraction order optimizer.","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"optimizer = TreeSA(ntrials = 1, niters = 5, βs = 0.1:0.3:100)","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"TreeSA{Int64, StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, GreedyMethod{OMEinsumContractionOrders.MinSpaceOut}, Any}(20, 0.1:0.3:100.0, 1, 5, 1.0, 0.2, :greedy, 0, Any[], GreedyMethod{OMEinsumContractionOrders.MinSpaceOut}(OMEinsumContractionOrders.MinSpaceOut(), 1))","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"Here, we choose the local search based TreeSA algorithm, which often finds the smallest time/space complexity and supports slicing. One can type ?TreeSA in a Julia REPL for more information about how to configure the hyper-parameters of the TreeSA method, while the detailed algorithm explanation is in arXiv: 2108.05665. Alternative tensor network contraction order optimizers include","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"GreedyMethod (default, fastest in searching speed but worst in contraction complexity)\nKaHyParBipartite\nSABipartite","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"tn = TensorNetworkModel(model; optimizer, evidence);","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"The returned object tn contains a field code that specifies the tensor network with optimized contraction order. To check the contraction complexity, please type","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"contraction_complexity(tn)","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"Time complexity: 2^19.330221020205514\nSpace complexity: 2^14.0\nRead-write complexity: 2^17.78831687691557","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"The returned object contains log2 values of the number of multiplications, the number elements in the largest tensor during contraction and the number of read-write operations to tensor elements.","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"probability(tn)","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"exp(-19.322038772705977) * fill(1.0)","category":"page"},{"location":"generated/performance-tips/#Using-the-slicing-technique-to-reduce-the-memory-cost","page":"Performance tips","title":"Using the slicing technique to reduce the memory cost","text":"","category":"section"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"For large scale applications, it is also possible to slice over certain degrees of freedom to reduce the space complexity, i.e. loop and accumulate over certain degrees of freedom so that one can have a smaller tensor network inside the loop due to the removal of these degrees of freedom. In the TreeSA optimizer, one can set nslices to a value larger than zero to turn on this feature. As a comparison we slice over 5 degrees of freedom, which can reduce the space complexity by at most 5. In this application, the slicing achieves the largest possible space complexity reduction 5, while the time and read-write complexity are only increased by less than 1, i.e. the peak memory usage is reduced by a factor 32, while the (theoretical) computing time is increased by at a factor 2.","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"optimizer = TreeSA(ntrials = 1, niters = 5, βs = 0.1:0.3:100, nslices=5)\ntn = TensorNetworkModel(model; optimizer, evidence);\ncontraction_complexity(tn)","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"Time complexity: 2^22.58280353572975\nSpace complexity: 2^11.0\nRead-write complexity: 2^21.01447821617116","category":"page"},{"location":"generated/performance-tips/#Faster-Tropical-tensor-contraction-to-speed-up-MAP-and-MMAP","page":"Performance tips","title":"Faster Tropical tensor contraction to speed up MAP and MMAP","text":"","category":"section"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"One can enjoy the BLAS level speed provided by TropicalGEMM by importing the package with","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"using TropicalGEMM","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"The benchmark in the TropicalGEMM repo shows this performance is close to the theoretical optimal value. Its implementation on GPU is under development in Github repo CuTropicalGEMM.jl as a part of Open Source Promotion Plan summer program.","category":"page"},{"location":"generated/performance-tips/#Working-with-GPUs","page":"Performance tips","title":"Working with GPUs","text":"","category":"section"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"To upload the computation to GPU, you just add using CUDA before calling the solve function, and set the keyword argument usecuda to true.","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"julia> using CUDA\n[ Info: OMEinsum loaded the CUDA module successfully\n\njulia> marginals(tn; usecuda = true);","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"Functions support usecuda keyword argument includes","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"probability\nlog_probability\nmarginals\nmost_probable_config","category":"page"},{"location":"generated/performance-tips/#Benchmarks","page":"Performance tips","title":"Benchmarks","text":"","category":"section"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"For performance metrics, see the Performance evaluation section.","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"This page was generated using Literate.jl.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"EditURL = \"main.jl\"","category":"page"},{"location":"generated/asia-network/main/#The-ASIA-network","page":"Asia Network","title":"The ASIA network","text":"","category":"section"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"The graph below corresponds to the ASIA network, a simple Bayesian model used extensively in educational settings. It was introduced by Lauritzen in 1988 [lauritzen1988local].","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"┌───┐ ┌───┐\n│ A │ ┌─┤ S ├─┐\n└─┬─┘ │ └───┘ │\n │ │ │\n ▼ ▼ ▼\n┌───┐ ┌───┐ ┌───┐\n│ T │ │ L │ │ B │\n└─┬─┘ └─┬─┘ └─┬─┘\n │ ┌───┐ │ │\n └──►│ E │◄──┘ │\n └─┬─┘ │\n┌───┐ │ ┌───┐ │\n│ X │◄──┴──►│ D │◄────┘\n└───┘ └───┘","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"The table below explains the meanings of each random variable used in the ASIA network model.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Random variable Meaning\nA Recent trip to Asia\nT Patient has tuberculosis\nS Patient is a smoker\nL Patient has lung cancer\nB Patient has bronchitis\nE Patient hast T and/or L\nX Chest X-Ray is positive\nD Patient has dyspnoea","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"We now demonstrate how to use the TensorInference.jl package for conducting a variety of inference tasks on the Asia network.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Import the TensorInference package, which provides the functionality needed for working with tensor networks and probabilistic graphical models.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"using TensorInference","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Load the ASIA network model from the asia.uai file located in the examples directory. See Model file format (.uai) for a description of the format of this file.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"model = read_model_file(pkgdir(TensorInference, \"examples\", \"asia-network\", \"model.uai\"))","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"UAIModel(nvars = 8, nfactors = 8)\n cards : [2, 2, 2, 2, 2, 2, 2, 2]\n factors : \n Factor(1), size = (2,)\n Factor(1, 2), size = (2, 2)\n Factor(3), size = (2,)\n Factor(3, 4), size = (2, 2)\n Factor(3, 5), size = (2, 2)\n Factor(2, 4, 6), size = (2, 2, 2)\n Factor(6, 7), size = (2, 2)\n Factor(5, 6, 8), size = (2, 2, 2)","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Create a tensor network representation of the loaded model.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"tn = TensorNetworkModel(model)","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"TensorNetworkModel{Int64, OMEinsum.DynamicNestedEinsum{Int64}, Array{Float64}}\nvariables: 1, 2, 3, 4, 5, 6, 7, 8\ncontraction time = 2^6.044, space = 2^2.0, read-write = 2^7.098","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Calculate the partition function. Since the factors in this model are normalized, the partition function is the same as the total probability, 1.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"probability(tn) |> first","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"1.0","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Calculate the marginal probabilities of each random variable in the model.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"marginals(tn)","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Dict{Vector{Int64}, Vector{Float64}} with 8 entries:\n [8] => [0.435971, 0.564029]\n [3] => [0.5, 0.5]\n [1] => [0.01, 0.99]\n [5] => [0.45, 0.55]\n [4] => [0.055, 0.945]\n [6] => [0.064828, 0.935172]\n [7] => [0.11029, 0.88971]\n [2] => [0.0104, 0.9896]","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Retrieve all the variables in the model.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"get_vars(tn)","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"8-element Vector{Int64}:\n 1\n 2\n 3\n 4\n 5\n 6\n 7\n 8","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Set the evidence: Assume that the \"X-ray\" result (variable 7) is negative. Since setting the evidence may affect the contraction order of the tensor network, recompute it.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"tn = TensorNetworkModel(model, evidence = Dict(7 => 0))","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"TensorNetworkModel{Int64, OMEinsum.DynamicNestedEinsum{Int64}, Array{Float64}}\nvariables: 1, 2, 3, 4, 5, 6, 7 (evidence → 0), 8\ncontraction time = 2^6.0, space = 2^2.0, read-write = 2^7.066","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Calculate the maximum log-probability among all configurations.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"maximum_logp(tn)","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"0-dimensional Array{Float64, 0}:\n-3.65222179200233","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Generate 10 samples from the posterior distribution.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"sample(tn, 10)","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"10-element TensorInference.Samples{Int64}:\n [0, 1, 0, 0, 1, 0, 0, 0]\n [1, 1, 1, 1, 1, 1, 0, 1]\n [1, 1, 0, 0, 0, 0, 0, 0]\n [1, 1, 1, 1, 1, 1, 0, 0]\n [1, 1, 0, 0, 0, 0, 0, 0]\n [1, 1, 0, 0, 0, 0, 0, 0]\n [1, 1, 1, 1, 1, 1, 0, 1]\n [1, 1, 1, 0, 1, 0, 0, 0]\n [1, 1, 0, 1, 1, 1, 0, 1]\n [1, 1, 0, 0, 1, 0, 0, 1]","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Retrieve both the maximum log-probability and the most probable configuration.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"logp, cfg = most_probable_config(tn)","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"(-3.65222179200233, [1, 1, 0, 0, 0, 0, 0, 0])","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Compute the most probable values of certain variables (e.g., 4 and 7) while marginalizing over others. This is known as Maximum a Posteriori (MAP) estimation.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"mmap = MMAPModel(model, evidence=Dict(7=>0), queryvars=[4,7])","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"MMAPModel{Int64, Array{Float64}}\nvariables: 4, 7 (evidence → 0)\nquery variables: [[1, 2, 6, 5, 3, 8]]\ncontraction time = 2^5.977, space = 2^2.0, read-write = 2^6.989","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Get the most probable configurations for variables 4 and 7.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"most_probable_config(mmap)","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"(-2.8754627318176693, [1, 0])","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Compute the total log-probability of having lung cancer. The results suggest that the probability is roughly half.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"log_probability(mmap, [1, 0]), log_probability(mmap, [0, 0])","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"(-2.8754627318176693, -2.9206248010671856)","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"[lauritzen1988local]: Steffen L Lauritzen and David J Spiegelhalter. Local computations with probabilities on graphical structures and their application to expert systems. Journal of the Royal Statistical Society: Series B (Methodological), 50(2):157–194, 1988.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"This page was generated using Literate.jl.","category":"page"},{"location":"performance-evaluation/#Performance-evaluation","page":"Performance evaluation","title":"Performance evaluation","text":"","category":"section"},{"location":"performance-evaluation/","page":"Performance evaluation","title":"Performance evaluation","text":"The graph below illustrates a comparison of the runtime performance of TensorInference.jl against Merlin [marinescu2022merlin], libDAI [mooij2010libdai], and JunctionTrees.jl [roa2022partial] libraries, specifically for the task of computing the marginal probabilities of all variables. Both Merlin and libDAI have previously participated in UAI inference competitions [gal2010summary][gogate2014uai], achieving favorable results. Additionally, we compared against JunctionTrees.jl, the predecessor of TensorInference.jl. The experiments were conducted on an Intel Core i9–9900K CPU @3.60GHz with 64 GB of RAM. Performance comparisons for other tasks will be added in the near future.","category":"page"},{"location":"performance-evaluation/","page":"Performance evaluation","title":"Performance evaluation","text":"","category":"page"},{"location":"performance-evaluation/","page":"Performance evaluation","title":"Performance evaluation","text":"The benchmark problems are arranged along the x-axis in ascending order of complexity, measured by the induced tree width. On average, TensorInference.jl achieves a speedup of 20 times across all problems. Notably, for the 10 most complex problems, the average speedup increases to 148 times, highlighting its superior scalability. The graph features a fitted linear curve in log-space to underscore the exponential improvement in computation time achieved by TensorInference.jl in comparison to the other alternatives. This speedup is primarily due to our package's unique approach: while traditional solvers typically focus only on minimizing space complexity (as quantified by the induced tree width), TensorInference.jl is designed to optimize for both time and space complexity. ","category":"page"},{"location":"performance-evaluation/#References","page":"Performance evaluation","title":"References","text":"","category":"section"},{"location":"performance-evaluation/","page":"Performance evaluation","title":"Performance evaluation","text":"[gal2010summary]: Gal Elidan and Amir Globerson. Summary of the 2010 UAI approximate inference challenge. 2010. [Online]. Available: https://www.cs.huji.ac.il/project/UAI10/summary.php [Accessed: 11 September 2023].","category":"page"},{"location":"performance-evaluation/","page":"Performance evaluation","title":"Performance evaluation","text":"[gogate2014uai]: Vibhav Gogate. UAI 2014 Probabilistic Inference Competition. 2014. [Online]. Available: https://www.ics.uci.edu/~dechter/softwares/benchmarks/Uai14/UAI2014Inference_Competition.pdf [Accessed: 11 September 2023].","category":"page"},{"location":"performance-evaluation/","page":"Performance evaluation","title":"Performance evaluation","text":"[marinescu2022merlin]: Radu Marinescu. Merlin. 2022. [Online]. Available: https://www.ibm.com/opensource/open/projects/merlin/ [Accessed: 11 September 2023].","category":"page"},{"location":"performance-evaluation/","page":"Performance evaluation","title":"Performance evaluation","text":"[mooij2010libdai]: Joris M. Mooij. libDAI: A Free and Open Source C++ Library for Discrete Approximate Inference in Graphical Models. Journal of Machine Learning Research, 11:2169-2173, Aug 2010. [Online]. Available: http://www.jmlr.org/papers/volume11/mooij10a/mooij10a.pdf.","category":"page"},{"location":"performance-evaluation/","page":"Performance evaluation","title":"Performance evaluation","text":"[roa2022partial]: Martin Roa-Villescas, Patrick W.A. Wijnings, Sander Stuijk, Henk Corporaal. \"Partial Evaluation in Junction Trees\". In: 2022 25th Euromicro Conference on Digital System Design (DSD), pp. 429-437, 2022. doi: 10.1109/DSD57027.2022.00064","category":"page"},{"location":"api/internal/#Internal-API","page":"Internal","title":"Internal API","text":"","category":"section"},{"location":"api/internal/#Index","page":"Internal","title":"Index","text":"","category":"section"},{"location":"api/internal/","page":"Internal","title":"Internal","text":"Types","category":"page"},{"location":"api/internal/","page":"Internal","title":"Internal","text":"Pages = [\"internal.md\"]\nOrder = [:type]","category":"page"},{"location":"api/internal/","page":"Internal","title":"Internal","text":"Functions","category":"page"},{"location":"api/internal/","page":"Internal","title":"Internal","text":"Pages = [\"internal.md\"]\nOrder = [:function]","category":"page"},{"location":"api/internal/#internal_Types","page":"Internal","title":"Types","text":"","category":"section"},{"location":"api/internal/","page":"Internal","title":"Internal","text":"Modules = [TensorInference]\nOrder = [:type]\nPublic = false","category":"page"},{"location":"api/internal/#TensorInference.Factor","page":"Internal","title":"TensorInference.Factor","text":"struct Factor{T, N}\n\nFields\n\nvars\nvals\n\nEncodes a discrete function over the set of variables vars that maps each instantiation of vars into a nonnegative number in vals.\n\n\n\n\n\n","category":"type"},{"location":"api/internal/#TensorInference.Samples","page":"Internal","title":"TensorInference.Samples","text":"struct Samples{L} <: AbstractVector{SubArray{Float64, 1, Matrix{Float64}, Tuple{Base.Slice{Base.OneTo{Int64}}, Int64}, true}}\n\nFields\n\nsamples::Matrix{Int64}\nlabels::Vector\nsetmask::BitVector\n\nThe sampled configurations are stored in samples, which is a vector of vector. labels is a vector of variable names for labeling configurations. The setmask is an boolean indicator to denote whether the sampling process of a variable is complete.\n\n\n\n\n\n","category":"type"},{"location":"api/internal/#internal_Functions","page":"Internal","title":"Functions","text":"","category":"section"},{"location":"api/internal/","page":"Internal","title":"Internal","text":"Modules = [TensorInference]\nOrder = [:function]\nPublic = false","category":"page"},{"location":"api/internal/#TensorInference.backward_sampling!-Tuple{Any, Tuple, Any, Any, TensorInference.Samples, Any}","page":"Internal","title":"TensorInference.backward_sampling!","text":"backward_sampling!(\n ixs,\n xs::Tuple,\n iy,\n y,\n samples::TensorInference.Samples,\n size_dict\n) -> TensorInference.Samples\n\n\nThe backward process for sampling configurations.\n\nixs and xs are labels and tensor data for input tensors,\niy and y are labels and tensor data for the output tensor,\nsamples is the samples generated for eliminated variables,\nsize_dict is a key-value map from tensor label to dimension size.\n\n\n\n\n\n","category":"method"},{"location":"api/internal/#TensorInference.backward_tropical!-Tuple{Any, Tuple, Vararg{Any, 4}}","page":"Internal","title":"TensorInference.backward_tropical!","text":"backward_tropical!(\n ixs,\n xs::Tuple,\n iy,\n y,\n ymask,\n size_dict\n) -> Vector{Any}\n\n\nThe backward rule for tropical einsum.\n\nixs and xs are labels and tensor data for input tensors,\niy and y are labels and tensor data for the output tensor,\nymask is the boolean mask for gradients,\nsize_dict is a key-value map from tensor label to dimension size.\n\n\n\n\n\n","category":"method"},{"location":"api/internal/#TensorInference.parse_mar_solution_file-Tuple{Vector{String}}","page":"Internal","title":"TensorInference.parse_mar_solution_file","text":"parse_mar_solution_file(\n rawlines::Vector{String};\n factor_eltype\n) -> Vector{Vector{Float64}}\n\n\nParse the solution marginals of all variables from the UAI MAR solution file. The order of the variables is the same as in the model definition.\n\nThe UAI file formats are defined in: https://uaicompetition.github.io/uci-2022/file-formats/\n\n\n\n\n\n","category":"method"},{"location":"api/internal/#TensorInference.read_query_file-Tuple{AbstractString}","page":"Internal","title":"TensorInference.read_query_file","text":"read_query_file(\n query_filepath::AbstractString\n) -> Vector{Int64}\n\n\nReturn the query variables in query_filepath. If the passed file path is an empty string, return an empty vector.\n\nThe UAI file formats are defined in: https://uaicompetition.github.io/uci-2022/file-formats/\n\n\n\n\n\n","category":"method"},{"location":"api/internal/#TensorInference.rescale_array-Union{Tuple{AbstractArray{T}}, Tuple{T}} where T","page":"Internal","title":"TensorInference.rescale_array","text":"rescale_array(tensor::AbstractArray{T}) -> RescaledArray\n\n\nReturns a rescaled array that equivalent to the input tensor.\n\n\n\n\n\n","category":"method"},{"location":"","page":"Home","title":"Home","text":"CurrentModule = TensorInference","category":"page"},{"location":"#TensorInference.jl","page":"Home","title":"TensorInference.jl","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"TensorInference is a standalone solver written in Julia, that harnesses tensor-based technology to implement state-of-the-art algorithms for probabilistic inference in graphical models. ","category":"page"},{"location":"#Package-features","page":"Home","title":"Package features","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Solutions to the most common probabilistic inference tasks, including:","category":"page"},{"location":"","page":"Home","title":"Home","text":"Probability of evidence (PR): Calculates the total probability of the observed evidence across all possible states of the unobserved variables.\nMarginal inference (MAR): Computes the probability distribution of a subset of variables, ignoring the states of all other variables.\nMaximum a Posteriori Probability estimation (MAP): Finds the most probable state of a subset of unobserved variables given some observed evidence.\nMarginal Maximum a Posteriori (MMAP): Finds the most probable state of a subset of variables, averaging out the uncertainty over the remaining ones.","category":"page"},{"location":"#Why-TensorInference.jl","page":"Home","title":"Why TensorInference.jl","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"A major challenge in developing intelligent systems is the ability to reason under uncertainty, a challenge that appears in many real-world problems across various domains, including artificial intelligence, medical diagnosis, computer vision, computational biology, and natural language processing. Reasoning under uncertainty involves calculating the probabilities of relevant variables while taking into account any information that is acquired. This process, which can be thought of as drawing global insights from local observations, is known as probabilistic inference.","category":"page"},{"location":"","page":"Home","title":"Home","text":"Probabilistic graphical models (PGMs) provide a unified framework to perform probabilistic inference. These models use graphs to represent the joint probability distribution of complex systems in a concise manner by exploiting the conditional independence between variables in the model. Additionally, they form the foundation for various algorithms that enable efficient probabilistic inference.","category":"page"},{"location":"","page":"Home","title":"Home","text":"However, even with the representational aid of PGMs, performing probabilistic inference remains an intractable endeavor on many real-world models. The reason is that performing probabilistic inference involves complex combinatorial optimization problems in very high dimensional spaces. To tackle these challenges, more efficient and scalable inference algorithms are needed.","category":"page"},{"location":"","page":"Home","title":"Home","text":"As an attempt to tackle the aforementioned challenges, we present TensorInference.jl, a Julia package for probabilistic inference that combines the representational capabilities of PGMs with the computational power of tensor networks. By harnessing the best of both worlds, TensorInference.jl aims to enhance the performance of probabilistic inference, thereby expanding the tractability spectrum of exact inference for more complex, real-world models.","category":"page"},{"location":"#Outline","page":"Home","title":"Outline","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Pages = [\n \"probabilisticinference.md\",\n \"tensornetwork.md\",\n \"uai-file-formats.md\",\n \"examples-overview.md\",\n \"performance.md\",\n \"api/public.md\",\n \"api/internal.md\",\n]\nDepth = 1","category":"page"}] +[{"location":"contributing/#Contributing","page":"Contributing","title":"Contributing","text":"","category":"section"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Welcome and thank you for considering contributing to TensorInference.jl.","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"By following these guidelines, you make it easier for everyone to work together. It shows you value the time of the people who create and manage this open-source project. In return, we'll show you the same respect by quickly looking at your issues, reviewing your changes, and helping you with your pull requests.","category":"page"},{"location":"contributing/#Getting-Started","page":"Contributing","title":"Getting Started","text":"","category":"section"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Contributions are made to this repo via Issues and Pull Requests (PRs). A few general guidelines that cover both:","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Search for existing Issues and PRs before creating your own.\nWe do our best to solve problems quickly. Still, some issues take longer to understand. Posting a comment can help, especially if you need a quick fix.","category":"page"},{"location":"contributing/#Issues","page":"Contributing","title":"Issues","text":"","category":"section"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Issues should be used to report problems with the package, ask for a new feature, or discuss possible changes before creating a Pull Request (PR). When you open a new issue, please include all the details needed to look into it.","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"If you find an Issue that addresses the problem you're having, please add your own reproduction information to the existing issue rather than creating a new one. ","category":"page"},{"location":"contributing/#Pull-Requests","page":"Contributing","title":"Pull Requests","text":"","category":"section"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Pull Requests (PRs) to our package are always welcome. Submitting a PR is a quick way to have your changes considered for the next software release. Generally, your Pull Request should:","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Either fix or add the functionality in question OR address widespread whitespace/style issues, not both.\nAdd unit or integration tests for any fixed or changed features.\nMinimize the number of changed lines to address a single concern.\nInclude documentation in the repo, especially for implementations of new features.","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"For major changes that affect core functionality or would require a new major release, we recommend opening an Issue to discuss your ideas before creating a Pull Request. While this step is optional, it can save everyone time in both the creation and review phases.","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"In general, we follow the \"fork-and-pull\" Git workflow","category":"page"},{"location":"contributing/","page":"Contributing","title":"Contributing","text":"Fork the repository to your own GitHub account.\nClone the project to your machine.\nCreate a branch locally with a succinct but descriptive name.\nCommit changes to the branch.\nFollow any formatting and testing guidelines specific to this repo.\nPush changes to your fork.\nOpen a PR in our repository and complete the PR template to help us efficiently review the changes.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"EditURL = \"main.jl\"","category":"page"},{"location":"generated/hard-core-lattice-gas/main/#The-hard-core-lattice-gas","page":"Hard-core Lattice Gas","title":"The hard core lattice gas","text":"","category":"section"},{"location":"generated/hard-core-lattice-gas/main/#Hard-core-lattice-gas-problem","page":"Hard-core Lattice Gas","title":"Hard-core lattice gas problem","text":"","category":"section"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"Hard-core lattice gas refers to a model used in statistical physics to study the behavior of particles on a lattice, where the particles are subject to an exclusion principle known as the \"hard-core\" interaction that characterized by a blockade radius. Distances between two particles can not be smaller than this radius.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"Nath T, Rajesh R. Multiple phase transitions in extended hard-core lattice gas models in two dimensions[J]. Physical Review E, 2014, 90(1): 012120.\nFernandes H C M, Arenzon J J, Levin Y. Monte Carlo simulations of two-dimensional hard core lattice gases[J]. The Journal of chemical physics, 2007, 126(11).","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"Let define a 10 times 10 triangular lattice, with unit vectors","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"beginalign*\nvec a = left(beginmatrix1 0endmatrixright)\nvec b = left(beginmatrixfrac12 fracsqrt32endmatrixright)\nendalign*","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"a, b = (1, 0), (0.5, 0.5*sqrt(3))\nNa, Nb = 10, 10\nsites = vec([50 .* (a .* i .+ b .* j) for i=1:Na, j=1:Nb])","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"100-element Vector{Tuple{Float64, Float64}}:\n (75.0, 43.30127018922193)\n (125.0, 43.30127018922193)\n (175.0, 43.30127018922193)\n (225.0, 43.30127018922193)\n (275.0, 43.30127018922193)\n (325.0, 43.30127018922193)\n (375.0, 43.30127018922193)\n (425.0, 43.30127018922193)\n (475.0, 43.30127018922193)\n (525.0, 43.30127018922193)\n (100.0, 86.60254037844386)\n (150.0, 86.60254037844386)\n (200.0, 86.60254037844386)\n (250.0, 86.60254037844386)\n (300.0, 86.60254037844386)\n (350.0, 86.60254037844386)\n (400.0, 86.60254037844386)\n (450.0, 86.60254037844386)\n (500.0, 86.60254037844386)\n (550.0, 86.60254037844386)\n (125.0, 129.9038105676658)\n (175.0, 129.9038105676658)\n (225.0, 129.9038105676658)\n (275.0, 129.9038105676658)\n (325.0, 129.9038105676658)\n (375.0, 129.9038105676658)\n (425.0, 129.9038105676658)\n (475.0, 129.9038105676658)\n (525.0, 129.9038105676658)\n (575.0, 129.9038105676658)\n (150.0, 173.20508075688772)\n (200.0, 173.20508075688772)\n (250.0, 173.20508075688772)\n (300.0, 173.20508075688772)\n (350.0, 173.20508075688772)\n (400.0, 173.20508075688772)\n (450.0, 173.20508075688772)\n (500.0, 173.20508075688772)\n (550.0, 173.20508075688772)\n (600.0, 173.20508075688772)\n (175.0, 216.50635094610965)\n (225.0, 216.50635094610965)\n (275.0, 216.50635094610965)\n (325.0, 216.50635094610965)\n (375.0, 216.50635094610965)\n (425.0, 216.50635094610965)\n (475.0, 216.50635094610965)\n (525.0, 216.50635094610965)\n (575.0, 216.50635094610965)\n (625.0, 216.50635094610965)\n (200.0, 259.8076211353316)\n (250.0, 259.8076211353316)\n (300.0, 259.8076211353316)\n (350.0, 259.8076211353316)\n (400.0, 259.8076211353316)\n (450.0, 259.8076211353316)\n (500.0, 259.8076211353316)\n (550.0, 259.8076211353316)\n (600.0, 259.8076211353316)\n (650.0, 259.8076211353316)\n (225.0, 303.1088913245535)\n (275.0, 303.1088913245535)\n (325.0, 303.1088913245535)\n (375.0, 303.1088913245535)\n (425.0, 303.1088913245535)\n (475.0, 303.1088913245535)\n (525.0, 303.1088913245535)\n (575.0, 303.1088913245535)\n (625.0, 303.1088913245535)\n (675.0, 303.1088913245535)\n (250.0, 346.41016151377545)\n (300.0, 346.41016151377545)\n (350.0, 346.41016151377545)\n (400.0, 346.41016151377545)\n (450.0, 346.41016151377545)\n (500.0, 346.41016151377545)\n (550.0, 346.41016151377545)\n (600.0, 346.41016151377545)\n (650.0, 346.41016151377545)\n (700.0, 346.41016151377545)\n (275.0, 389.71143170299734)\n (325.0, 389.71143170299734)\n (375.0, 389.71143170299734)\n (425.0, 389.71143170299734)\n (475.0, 389.71143170299734)\n (525.0, 389.71143170299734)\n (575.0, 389.71143170299734)\n (625.0, 389.71143170299734)\n (675.0, 389.71143170299734)\n (725.0, 389.71143170299734)\n (300.0, 433.0127018922193)\n (350.0, 433.0127018922193)\n (400.0, 433.0127018922193)\n (450.0, 433.0127018922193)\n (500.0, 433.0127018922193)\n (550.0, 433.0127018922193)\n (600.0, 433.0127018922193)\n (650.0, 433.0127018922193)\n (700.0, 433.0127018922193)\n (750.0, 433.0127018922193)","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"There exists blockade interactions between hard-core particles. We connect two lattice sites within blockade radius by an edge. Two ends of an edge can not both be occupied by particles.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"blockade_radius = 55\nusing GenericTensorNetworks: show_graph, unit_disk_graph\nusing GenericTensorNetworks.Graphs: edges, nv\ngraph = unit_disk_graph(vec(sites), blockade_radius)\nshow_graph(graph, sites; texts=fill(\"\", length(sites)))","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"(Image: )","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"These constraints defines an independent set problem that characterized by the following energy based model. Let G = (V E) be a graph, where V is the set of vertices and E is the set of edges. The energy model for the hard-core lattice gas problem is","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"E(mathbfn) = -sum_i in Vw_i n_i + U sum_(i j) in E n_i n_j","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"where n_i in 0 1 is the number of particles at site i, and w_i is the weight associated with it. For unweighted graphs, the weights are uniform. U is the repulsive interaction strength between two particles. To represent the independence constraint, we let U = infty, i.e. coexitence of two particles at two sites connected by an edge is completely forbidden. The solution space hard-core lattice gas is equivalent to that of an independent set problem. The independent set problem involves finding a set of vertices in a graph such that no two vertices in the set are adjacent (i.e., there is no edge connecting them). One can create a tensor network based modeling of an independent set problem with package GenericTensorNetworks.jl.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"using GenericTensorNetworks\nproblem = IndependentSet(graph)","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"GenericTensorNetworks.IndependentSet{GenericTensorNetworks.UnitWeight}(Graphs.SimpleGraphs.SimpleGraph{Int64}(261, [[2, 11], [1, 3, 11, 12], [2, 4, 12, 13], [3, 5, 13, 14], [4, 6, 14, 15], [5, 7, 15, 16], [6, 8, 16, 17], [7, 9, 17, 18], [8, 10, 18, 19], [9, 19, 20], [1, 2, 12, 21], [2, 3, 11, 13, 21, 22], [3, 4, 12, 14, 22, 23], [4, 5, 13, 15, 23, 24], [5, 6, 14, 16, 24, 25], [6, 7, 15, 17, 25, 26], [7, 8, 16, 18, 26, 27], [8, 9, 17, 19, 27, 28], [9, 10, 18, 20, 28, 29], [10, 19, 29, 30], [11, 12, 22, 31], [12, 13, 21, 23, 31, 32], [13, 14, 22, 24, 32, 33], [14, 15, 23, 25, 33, 34], [15, 16, 24, 26, 34, 35], [16, 17, 25, 27, 35, 36], [17, 18, 26, 28, 36, 37], [18, 19, 27, 29, 37, 38], [19, 20, 28, 30, 38, 39], [20, 29, 39, 40], [21, 22, 32, 41], [22, 23, 31, 33, 41, 42], [23, 24, 32, 34, 42, 43], [24, 25, 33, 35, 43, 44], [25, 26, 34, 36, 44, 45], [26, 27, 35, 37, 45, 46], [27, 28, 36, 38, 46, 47], [28, 29, 37, 39, 47, 48], [29, 30, 38, 40, 48, 49], [30, 39, 49, 50], [31, 32, 42, 51], [32, 33, 41, 43, 51, 52], [33, 34, 42, 44, 52, 53], [34, 35, 43, 45, 53, 54], [35, 36, 44, 46, 54, 55], [36, 37, 45, 47, 55, 56], [37, 38, 46, 48, 56, 57], [38, 39, 47, 49, 57, 58], [39, 40, 48, 50, 58, 59], [40, 49, 59, 60], [41, 42, 52, 61], [42, 43, 51, 53, 61, 62], [43, 44, 52, 54, 62, 63], [44, 45, 53, 55, 63, 64], [45, 46, 54, 56, 64, 65], [46, 47, 55, 57, 65, 66], [47, 48, 56, 58, 66, 67], [48, 49, 57, 59, 67, 68], [49, 50, 58, 60, 68, 69], [50, 59, 69, 70], [51, 52, 62, 71], [52, 53, 61, 63, 71, 72], [53, 54, 62, 64, 72, 73], [54, 55, 63, 65, 73, 74], [55, 56, 64, 66, 74, 75], [56, 57, 65, 67, 75, 76], [57, 58, 66, 68, 76, 77], [58, 59, 67, 69, 77, 78], [59, 60, 68, 70, 78, 79], [60, 69, 79, 80], [61, 62, 72, 81], [62, 63, 71, 73, 81, 82], [63, 64, 72, 74, 82, 83], [64, 65, 73, 75, 83, 84], [65, 66, 74, 76, 84, 85], [66, 67, 75, 77, 85, 86], [67, 68, 76, 78, 86, 87], [68, 69, 77, 79, 87, 88], [69, 70, 78, 80, 88, 89], [70, 79, 89, 90], [71, 72, 82, 91], [72, 73, 81, 83, 91, 92], [73, 74, 82, 84, 92, 93], [74, 75, 83, 85, 93, 94], [75, 76, 84, 86, 94, 95], [76, 77, 85, 87, 95, 96], [77, 78, 86, 88, 96, 97], [78, 79, 87, 89, 97, 98], [79, 80, 88, 90, 98, 99], [80, 89, 99, 100], [81, 82, 92], [82, 83, 91, 93], [83, 84, 92, 94], [84, 85, 93, 95], [85, 86, 94, 96], [86, 87, 95, 97], [87, 88, 96, 98], [88, 89, 97, 99], [89, 90, 98, 100], [90, 99]]), GenericTensorNetworks.UnitWeight())","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"There are plenty of discussions related to solution space properties in the GenericTensorNetworks documentaion page. In this example, we show how to use TensorInference to use probabilistic inference for understand the finite temperature properties of this statistical model. We use TensorNetworkModel to convert a combinatorial optimization problem to a probabilistic model. Here, we let the inverse temperature be beta = 3.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/#Probabilistic-modeling-correlation-functions","page":"Hard-core Lattice Gas","title":"Probabilistic modeling correlation functions","text":"","category":"section"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"using TensorInference\nβ = 3.0\npmodel = TensorNetworkModel(problem, β)","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"TensorNetworkModel{Int64, OMEinsum.DynamicNestedEinsum{Int64}, Array{Float64}}\nvariables: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100\ncontraction time = 2^18.013, space = 2^12.0, read-write = 2^15.986","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"The partition function of this statistical model can be computed with the probability function.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"partition_func = probability(pmodel)","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"exp(107.42267890429403) * fill(1.0)","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"The default return value is a log-rescaled tensor. Use indexing to get the real value.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"partition_func[]","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"4.4985927541462117e46","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"The marginal probabilities can be computed with the marginals function, which measures how likely a site is occupied.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"mars = marginals(pmodel)\nshow_graph(graph, sites; vertex_colors=[(b = mars[[i]][2]; (1-b, 1-b, 1-b)) for i in 1:nv(graph)], texts=fill(\"\", nv(graph)))","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"(Image: )","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"The can see the sites at the corner is more likely to be occupied. To obtain two-site correlations, one can set the variables to query marginal probabilities manually.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"pmodel2 = TensorNetworkModel(problem, β; mars=[[e.src, e.dst] for e in edges(graph)])\nmars = marginals(pmodel2);","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"We show the probability that both sites on an edge are not occupied","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"show_graph(graph, sites; edge_colors=[(b = mars[[e.src, e.dst]][1, 1]; (1-b, 1-b, 1-b)) for e in edges(graph)], texts=fill(\"\", nv(graph)), config=GraphDisplayConfig(; edge_line_width=5))","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"(Image: )","category":"page"},{"location":"generated/hard-core-lattice-gas/main/#The-most-likely-configuration","page":"Hard-core Lattice Gas","title":"The most likely configuration","text":"","category":"section"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"The MAP and MMAP can be used to get the most likely configuration given an evidence. The relavant function is most_probable_config. If we fix the vertex configuration at one corner to be one, we get the most probably configuration as bellow.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"pmodel3 = TensorNetworkModel(problem, β; evidence=Dict(1=>1))\nmars = marginals(pmodel3)\nlogp, config = most_probable_config(pmodel3)","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"(102.0, [1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1])","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"The log probability is 102. Let us visualize the configuration.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"show_graph(graph, sites; vertex_colors=[(1-b, 1-b, 1-b) for b in config], texts=fill(\"\", nv(graph)))","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"(Image: )","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"The number of particles is","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"sum(config)","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"34","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"Otherwise, we will get a suboptimal configuration.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"pmodel3 = TensorNetworkModel(problem, β; evidence=Dict(1=>0))\nlogp2, config2 = most_probable_config(pmodel)","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"(102.0, [1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1])","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"The log probability is 99, which is much smaller.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"show_graph(graph, sites; vertex_colors=[(1-b, 1-b, 1-b) for b in config2], texts=fill(\"\", nv(graph)))","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"(Image: )","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"The number of particles is","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"sum(config2)\n\n# Sampling configurations","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"34","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"One can ue sample to generate samples from hard-core lattice gas at finite temperature. The return value is a matrix, with the columns correspond to different samples.","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"configs = sample(pmodel3, 1000)\nsizes = sum.(configs)\n[count(==(i), sizes) for i=0:34] # counting sizes","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"35-element Vector{Int64}:\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 0\n 9\n 21\n 45\n 124\n 202\n 216\n 176\n 128\n 60\n 19\n 0","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"","category":"page"},{"location":"generated/hard-core-lattice-gas/main/","page":"Hard-core Lattice Gas","title":"Hard-core Lattice Gas","text":"This page was generated using Literate.jl.","category":"page"},{"location":"tensor-networks/#Tensor-networks","page":"Tensor Networks","title":"Tensor networks","text":"","category":"section"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"We now introduce the core ideas of tensor networks, highlighting their connections with probabilistic graphical models (PGM) to align the terminology between them.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"For our purposes, a tensor is equivalent to the concept of a factor as defined in the PGM domain, which we detail more formally below.","category":"page"},{"location":"tensor-networks/#What-is-a-tensor?","page":"Tensor Networks","title":"What is a tensor?","text":"","category":"section"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"Definition: A tensor T is defined as:","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"T prod_V in bmV mathcalD_V rightarrow textttnumber","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"Here, the function T maps each possible instantiation of the random variables in its scope bmV to a generic number type. In the context of tensor networks, a minimum requirement is that the number type is a commutative semiring. To define a commutative semiring with the addition operation oplus and the multiplication operation odot on a set S, the following relations must hold for any arbitrary three elements a b c in S.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"newcommandmymathbb1mathbb1\nbeginalign*\n(a oplus b) oplus c = a oplus (b oplus c) hspace5emtexttriangleright commutative monoid oplus with identity mymathbb0\na oplus mymathbb0 = mymathbb0 oplus a = a \na oplus b = b oplus a \n\n(a odot b) odot c = a odot (b odot c) hspace5emtexttriangleright commutative monoid odot with identity mymathbb1\na odot mymathbb1 = mymathbb1 odot a = a \na odot b = b odot a \n\na odot (boplus c) = aodot b oplus aodot c hspace5emtexttriangleright left and right distributive\n(aoplus b) odot c = aodot c oplus bodot c \n\na odot mymathbb0 = mymathbb0 odot a = mymathbb0\nendalign*","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"Tensors are represented using multidimensional arrays of numbers with labeled dimensions. These labels correspond to the array's indices, which in turn represent the set of random variables that the tensor is a function of. Thus, in this context, the terms label, index, and variable are synonymous and hence used interchangeably.","category":"page"},{"location":"tensor-networks/#What-is-a-tensor-network?","page":"Tensor Networks","title":"What is a tensor network?","text":"","category":"section"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"We now turn our attention to defining a tensor network, a mathematical object used to represent a multilinear map between tensors. This concept is widely employed in fields like condensed matter physics [Orus2014][Pfeifer2014], quantum simulation [Markov2008][Pan2022], and even in solving combinatorial optimization problems [Liu2023]. It's worth noting that we use a generalized version of the conventional notation, most commonly known through the eisnum function, which is commonly used in high-performance computing. Packages that implement this conventional notation include","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"numpy\nOMEinsum.jl\nPyTorch\nTensorFlow","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"This approach allows us to represent a broader range of sum-product multilinear operations between tensors, thus meeting the requirements of the PGM field.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"Definition[Liu2023]: A tensor network is a multilinear map represented by the triple mathcalN = (Lambda mathcalT bmsigma_0), where:","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"Lambda is the set of variables present in the network mathcalN.\nmathcalT = T^(k)_bmsigma_k _k=1^M is the set of input tensors, where each tensor T^(k)_bmsigma_k is identified by a superscript (k) and has an associated scope bmsigma_k.\nbmsigma_0 specifies the scope of the output tensor.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"More specifically, each tensor T^(k)_bmsigma_k in mathcalT is labeled by a string bmsigma_k in Lambda^r left(T^(k) right), where r left(T^(k) right) is the rank of T^(k). The multilinear map, also known as the contraction, applied to this triple is defined as","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"textttcontract(Lambda mathcalT bmsigma_0) = sum_bmsigma_Lambda\nsetminus bmsigma_0 prod_k=1^M T^(k)_bmsigma_k","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"Notably, the summation extends over all instantiations of the variables that are not part of the output tensor.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"As an example, consider matrix multiplication, which can be specified as a tensor network contraction:","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":" (AB)_ik = textttcontractleft(ijk A_ij B_jk ikright)","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"Here, matrices A and B are input tensors labeled by strings ij jk in i j k^2. The output tensor is labeled by string ik. Summations run over indices Lambda setminus ik = j. The contraction corresponds to","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":" textttcontractleft(ijk A_ij B_jk ikright) = sum_j\n A_ijB_jk","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"In the einsum notation commonly used in various programming languages, this is equivalent to ij, jk -> ik.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"Diagrammatically, a tensor network can be represented as an open hypergraph. In this diagram, a tensor maps to a vertex, and a variable maps to a hyperedge. Tensors sharing the same variable are connected by the same hyperedge for that variable. The diagrammatic representation of matrix multiplication is:","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"using TikzPictures\n\ntp = TikzPicture(\n L\"\"\"\n \\matrix[row sep=0.8cm,column sep=0.8cm,ampersand replacement= \\& ] {\n \\node (1) {}; \\&\n \\node (a) [mytensor] {$A$}; \\&\n \\node (b) [mytensor] {$B$}; \\&\n \\node (2) {}; \\&\n \\\\\n };\n \\draw [myedge, color=c01] (1) edge node[below] {$i$} (a);\n \\draw [myedge, color=c02] (a) edge node[below] {$j$} (b);\n \\draw [myedge, color=c03] (b) edge node[below] {$k$} (2);\n \"\"\",\n options=\"every node/.style={scale=2.0}\",\n preamble=\"\\\\input{\" * joinpath(@__DIR__, \"assets\", \"preambles\", \"the-tensor-network\") * \"}\",\n)\nsave(SVG(\"the-tensor-network1\"), tp)","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"In this diagram, we use different colors to denote different hyperedges. Hyperedges for i and j are left open to denote variables in the output string bmsigma_0. The reason we use hyperedges rather than regular edges will become clear in the following star contraction example.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":" textttcontract(ijkl A_il B_jl C_kl ijk) = sum_lA_il\n B_jl C_kl","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"The equivalent einsum notation employed by many programming languages is il, jl, kl -> ijk.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"Since the variable l is shared across all three tensors, a simple graph can't capture the diagram's complexity. The more appropriate hypergraph representation is shown below.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"using TikzPictures\n\ntp = TikzPicture(\n L\"\"\"\n \\matrix[row sep=0.4cm,column sep=0.4cm,ampersand replacement= \\& ] {\n \\&\n \\&\n \\node[color=c01] (j) {$j$}; \\&\n \\&\n \\&\n \\\\\n \\&\n \\&\n \\node (b) [mytensor] {$B$}; \\&\n \\&\n \\&\n \\\\\n \\node[color=c03] (i) {$i$}; \\&\n \\node (a) [mytensor] {$A$}; \\&\n \\node[color=c02] (l) {$l$}; \\&\n \\node (c) [mytensor] {$C$}; \\&\n \\node[color=c04] (k) {$k$}; \\&\n \\\\\n };\n \\draw [myedge, color=c01] (j) edge (b);\n \\draw [myedge, color=c02] (b) edge (l);\n \\draw [myedge, color=c03] (i) edge (a);\n \\draw [myedge, color=c02] (a) edge (l);\n \\draw [myedge, color=c02] (l) edge (c);\n \\draw [myedge, color=c04] (c) edge (k);\n \"\"\",\n options=\"every node/.style={scale=2.0}\",\n preamble=\"\\\\input{\" * joinpath(@__DIR__, \"assets\", \"preambles\", \"the-tensor-network\") * \"}\",\n)\nsave(SVG(\"the-tensor-network2\"), tp)","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"As a final note, our definition of a tensor network allows for repeated indices within the same tensor, which translates to self-loops in their corresponding diagrams.","category":"page"},{"location":"tensor-networks/#Tensor-network-contraction-orders","page":"Tensor Networks","title":"Tensor network contraction orders","text":"","category":"section"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"The performance of a tensor network contraction depends on the order in which the tensors are contracted. The order of contraction is usually specified by binary trees, where the leaves are the input tensors and the internal nodes represent the order of contraction. The root of the tree is the output tensor.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"Numerous approaches have been proposed to determine efficient contraction orderings, which include:","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"Greedy algorithms\nBreadth-first search and Dynamic programming [Pfeifer2014]\nGraph bipartitioning [Gray2021]\nLocal search [Kalachev2021]","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"Some of these have been implemented in the OMEinsum package. Please check Performance Tips for more details.","category":"page"},{"location":"tensor-networks/#References","page":"Tensor Networks","title":"References","text":"","category":"section"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"[Orus2014]: Orús R. A practical introduction to tensor networks: Matrix product states and projected entangled pair states[J]. Annals of physics, 2014, 349: 117-158.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"[Markov2008]: Markov I L, Shi Y. Simulating quantum computation by contracting tensor networks[J]. SIAM Journal on Computing, 2008, 38(3): 963-981.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"[Pfeifer2014]: Pfeifer R N C, Haegeman J, Verstraete F. Faster identification of optimal contraction sequences for tensor networks[J]. Physical Review E, 2014, 90(3): 033315.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"[Gray2021]: Gray J, Kourtis S. Hyper-optimized tensor network contraction[J]. Quantum, 2021, 5: 410.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"[Kalachev2021]: Kalachev G, Panteleev P, Yung M H. Multi-tensor contraction for XEB verification of quantum circuits[J]. arXiv:2108.05665, 2021.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"[Pan2022]: Pan F, Chen K, Zhang P. Solving the sampling problem of the sycamore quantum circuits[J]. Physical Review Letters, 2022, 129(9): 090502.","category":"page"},{"location":"tensor-networks/","page":"Tensor Networks","title":"Tensor Networks","text":"[Liu2023]: Liu J G, Gao X, Cain M, et al. Computing solution space properties of combinatorial optimization problems via generic tensor networks[J]. SIAM Journal on Scientific Computing, 2023, 45(3): A1239-A1270.","category":"page"},{"location":"api/public/#Public-API","page":"Public","title":"Public API","text":"","category":"section"},{"location":"api/public/#Index","page":"Public","title":"Index","text":"","category":"section"},{"location":"api/public/","page":"Public","title":"Public","text":"Modules","category":"page"},{"location":"api/public/","page":"Public","title":"Public","text":"Pages = [\"public.md\"]\nOrder = [:module]","category":"page"},{"location":"api/public/","page":"Public","title":"Public","text":"Types","category":"page"},{"location":"api/public/","page":"Public","title":"Public","text":"Pages = [\"public.md\"]\nOrder = [:type]","category":"page"},{"location":"api/public/","page":"Public","title":"Public","text":"Functions","category":"page"},{"location":"api/public/","page":"Public","title":"Public","text":"Pages = [\"public.md\"]\nOrder = [:function]","category":"page"},{"location":"api/public/#Modules","page":"Public","title":"Modules","text":"","category":"section"},{"location":"api/public/","page":"Public","title":"Public","text":"TensorInference","category":"page"},{"location":"api/public/#TensorInference","page":"Public","title":"TensorInference","text":"Main module for TensorInference.jl – A toolbox for probabilistic inference using contraction of tensor networks.\n\nExports\n\nArtifactProblemSpec\nGreedyMethod\nKaHyParBipartite\nMMAPModel\nMergeGreedy\nMergeVectors\nRescaledArray\nSABipartite\nTensorNetworkModel\nTreeSA\nUAIModel\ncontraction_complexity\ndataset_from_artifact\nget_cards\nget_vars\nlog_probability\nmarginals\nmaximum_logp\nmost_probable_config\nprobability\nproblem_from_artifact\nread_evidence\nread_evidence_file\nread_model\nread_model_file\nread_queryvars\nread_solution\nread_td_file\nsample\nupdate_evidence!\nupdate_temperature\n\n\n\n\n\n","category":"module"},{"location":"api/public/#Types","page":"Public","title":"Types","text":"","category":"section"},{"location":"api/public/","page":"Public","title":"Public","text":"GreedyMethod\nKaHyParBipartite\nMergeGreedy\nMergeVectors\nSABipartite\nTreeSA\nMMAPModel\nRescaledArray\nTensorNetworkModel\nArtifactProblemSpec\nUAIModel","category":"page"},{"location":"api/public/#OMEinsumContractionOrders.GreedyMethod","page":"Public","title":"OMEinsumContractionOrders.GreedyMethod","text":"GreedyMethod{MT}\nGreedyMethod(; method=MinSpaceOut(), nrepeat=10)\n\nThe fast but poor greedy optimizer. Input arguments are\n\nmethod is MinSpaceDiff() or MinSpaceOut.\nMinSpaceOut choose one of the contraction that produces a minimum output tensor size,\nMinSpaceDiff choose one of the contraction that decrease the space most.\nnrepeat is the number of repeatition, returns the best contraction order.\n\n\n\n\n\n","category":"type"},{"location":"api/public/#OMEinsumContractionOrders.KaHyParBipartite","page":"Public","title":"OMEinsumContractionOrders.KaHyParBipartite","text":"KaHyParBipartite{RT,IT,GM}\nKaHyParBipartite(; sc_target, imbalances=collect(0.0:0.005:0.8),\n max_group_size=40, greedy_config=GreedyMethod())\n\nOptimize the einsum code contraction order using the KaHyPar + Greedy approach. This program first recursively cuts the tensors into several groups using KaHyPar, with maximum group size specifed by max_group_size and maximum space complexity specified by sc_target, Then finds the contraction order inside each group with the greedy search algorithm. Other arguments are\n\nsc_target is the target space complexity, defined as log2(number of elements in the largest tensor),\nimbalances is a KaHyPar parameter that controls the group sizes in hierarchical bipartition,\nmax_group_size is the maximum size that allowed to used greedy search,\ngreedy_config is a greedy optimizer.\n\nReferences\n\nHyper-optimized tensor network contraction\nSimulating the Sycamore quantum supremacy circuits\n\n\n\n\n\n","category":"type"},{"location":"api/public/#OMEinsumContractionOrders.MergeGreedy","page":"Public","title":"OMEinsumContractionOrders.MergeGreedy","text":"MergeGreedy <: CodeSimplifier\nMergeGreedy(; threshhold=-1e-12)\n\nContraction code simplifier (in order to reduce the time of calling optimizers) that merges tensors greedily if the space complexity of merged tensors is reduced (difference smaller than the threshhold).\n\n\n\n\n\n","category":"type"},{"location":"api/public/#OMEinsumContractionOrders.MergeVectors","page":"Public","title":"OMEinsumContractionOrders.MergeVectors","text":"MergeVectors <: CodeSimplifier\nMergeVectors()\n\nContraction code simplifier (in order to reduce the time of calling optimizers) that merges vectors to closest tensors.\n\n\n\n\n\n","category":"type"},{"location":"api/public/#OMEinsumContractionOrders.SABipartite","page":"Public","title":"OMEinsumContractionOrders.SABipartite","text":"SABipartite{RT,BT}\nSABipartite(; sc_target=25, ntrials=50, βs=0.1:0.2:15.0, niters=1000\n max_group_size=40, greedy_config=GreedyMethod(), initializer=:random)\n\nOptimize the einsum code contraction order using the Simulated Annealing bipartition + Greedy approach. This program first recursively cuts the tensors into several groups using simulated annealing, with maximum group size specifed by max_group_size and maximum space complexity specified by sc_target, Then finds the contraction order inside each group with the greedy search algorithm. Other arguments are\n\nsize_dict, a dictionary that specifies leg dimensions,\nsc_target is the target space complexity, defined as log2(number of elements in the largest tensor),\nmax_group_size is the maximum size that allowed to used greedy search,\nβs is a list of inverse temperature 1/T,\nniters is the number of iteration in each temperature,\nntrials is the number of repetition (with different random seeds),\ngreedy_config configures the greedy method,\ninitializer, the partition configuration initializer, one can choose :random or :greedy (slow but better).\n\nReferences\n\nHyper-optimized tensor network contraction\n\n\n\n\n\n","category":"type"},{"location":"api/public/#OMEinsumContractionOrders.TreeSA","page":"Public","title":"OMEinsumContractionOrders.TreeSA","text":"TreeSA{RT,IT,GM}\nTreeSA(; sc_target=20, βs=collect(0.01:0.05:15), ntrials=10, niters=50,\n sc_weight=1.0, rw_weight=0.2, initializer=:greedy, greedy_config=GreedyMethod(; nrepeat=1))\n\nOptimize the einsum contraction pattern using the simulated annealing on tensor expression tree.\n\nsc_target is the target space complexity,\nntrials, βs and niters are annealing parameters, doing ntrials indepedent annealings, each has inverse tempteratures specified by βs, in each temperature, do niters updates of the tree.\nsc_weight is the relative importance factor of space complexity in the loss compared with the time complexity.\nrw_weight is the relative importance factor of memory read and write in the loss compared with the time complexity.\ninitializer specifies how to determine the initial configuration, it can be :greedy or :random. If it is using :greedy method to generate the initial configuration, it also uses two extra arguments greedy_method and greedy_nrepeat.\nnslices is the number of sliced legs, default is 0.\nfixed_slices is a vector of sliced legs, default is [].\n\nReferences\n\nRecursive Multi-Tensor Contraction for XEB Verification of Quantum Circuits\n\n\n\n\n\n","category":"type"},{"location":"api/public/#TensorInference.MMAPModel","page":"Public","title":"TensorInference.MMAPModel","text":"struct MMAPModel{LT, AT<:AbstractArray}\n\nComputing the most likely assignment to the query variables, Xₘ ⊆ X after marginalizing out the remaining variables Xₛ = X \\ Xₘ.\n\nrm MMAP(X_iE=e) = arg max_X_M sum_X_S prod_F f(x_M x_S e)\n\nFields\n\nvars is the query variables in the tensor network.\ncode is the tropical tensor network contraction pattern.\ntensors is the tensors fed into the tensor network.\nclusters is the clusters, each element of this cluster is a TensorNetworkModel instance for marginalizing certain variables.\nevidence is a dictionary to specifiy degree of freedoms fixed to certain values, which should not have overlap with the query variables.\n\n\n\n\n\n","category":"type"},{"location":"api/public/#TensorInference.RescaledArray","page":"Public","title":"TensorInference.RescaledArray","text":"struct RescaledArray{T, N, AT<:AbstractArray{T, N}} <: AbstractArray{T, N}\n\nRescaledArray(α, T) -> RescaledArray\n\nAn array data type with a log-prefactor, and a l∞-normalized storage, i.e. the maximum element in a tensor is 1. This tensor type can avoid the potential underflow/overflow of numbers in a tensor network. The constructor RescaledArray(α, T) creates a rescaled array that equal to exp(α) * T.\n\n\n\n\n\n","category":"type"},{"location":"api/public/#TensorInference.TensorNetworkModel","page":"Public","title":"TensorInference.TensorNetworkModel","text":"struct TensorNetworkModel{LT, ET, MT<:AbstractArray}\n\nProbabilistic modeling with a tensor network.\n\nFields\n\nvars are the degrees of freedom in the tensor network.\ncode is the tensor network contraction pattern.\ntensors are the tensors fed into the tensor network, the leading tensors are unity tensors associated with mars.\nevidence is a dictionary used to specify degrees of freedom that are fixed to certain values.\nmars is a vector, each element is a vector of variables to compute marginal probabilities.\n\n\n\n\n\n","category":"type"},{"location":"api/public/#TensorInference.ArtifactProblemSpec","page":"Public","title":"TensorInference.ArtifactProblemSpec","text":"struct ArtifactProblemSpec\n\nSpecify the UAI models from the artifacts. It can be used as the input of read_model.\n\nFields\n\nartifact_path::String\ntask::String\nproblem_set::String\nproblem_id::Int64\n\n\n\n\n\n","category":"type"},{"location":"api/public/#TensorInference.UAIModel","page":"Public","title":"TensorInference.UAIModel","text":"struct UAIModel{ET, FT<:(TensorInference.Factor{ET})}\n\nFields\n\nnvars is the number of variables,\ncards is a vector of cardinalities for variables,\nfactors is a vector of factors,\n\n\n\n\n\n","category":"type"},{"location":"api/public/#Functions","page":"Public","title":"Functions","text":"","category":"section"},{"location":"api/public/","page":"Public","title":"Public","text":"contraction_complexity\nget_cards\nget_vars\nlog_probability\nmarginals\nmaximum_logp\nmost_probable_config\nprobability\ndataset_from_artifact\nproblem_from_artifact\nread_model\nread_evidence\nread_solution\nread_queryvars\nread_model_file\nread_evidence_file\nread_td_file\nsample\nupdate_evidence!\nupdate_temperature","category":"page"},{"location":"api/public/#OMEinsumContractionOrders.contraction_complexity","page":"Public","title":"OMEinsumContractionOrders.contraction_complexity","text":"contraction_complexity(tensor_network)\n\nReturns the contraction complexity of a tensor newtork model.\n\n\n\n\n\ncontraction_complexity(eincode, size_dict) -> ContractionComplexity\n\nReturns the time, space and read-write complexity of the einsum contraction. The returned object contains 3 fields:\n\ntime complexity tc defined as log2(number of element-wise multiplications).\nspace complexity sc defined as log2(size of the maximum intermediate tensor).\nread-write complexity rwc defined as log2(the number of read-write operations).\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.get_cards","page":"Public","title":"TensorInference.get_cards","text":"get_cards(tn::TensorNetworkModel; fixedisone) -> Vector\n\n\nGet the cardinalities of variables in this tensor network.\n\n\n\n\n\nget_cards(mmap::MMAPModel; fixedisone) -> Vector\n\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.get_vars","page":"Public","title":"TensorInference.get_vars","text":"get_vars(tn::TensorNetworkModel) -> Vector\n\n\nGet the variables in this tensor network, they are also known as legs, labels, or degree of freedoms.\n\n\n\n\n\nget_vars(mmap::MMAPModel) -> Vector\n\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.log_probability","page":"Public","title":"TensorInference.log_probability","text":"log_probability(\n tn::TensorNetworkModel,\n config::Union{Dict, AbstractVector}\n) -> Real\n\n\nEvaluate the log probability (or partition function) of config.\n\n\n\n\n\nlog_probability(\n tn::TensorNetworkModel;\n usecuda\n) -> AbstractArray\n\n\nEvaluate the log probability (or partition function). It is the logged version of probability, which is less likely to overflow.\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.marginals","page":"Public","title":"TensorInference.marginals","text":"marginals(\n tn::TensorNetworkModel;\n usecuda,\n rescale\n) -> Dict{Vector{Int64}}\n\n\nQueries the marginals of the variables in a TensorNetworkModel. The function returns a dictionary, where the keys are the variables and the values are their respective marginals. A marginal is a probability distribution over a subset of variables, obtained by integrating or summing over the remaining variables in the model. By default, the function returns the marginals of all individual variables. To specify which marginal variables to query, set the mars field when constructing a TensorNetworkModel. Note that the choice of marginal variables will affect the contraction order of the tensor network.\n\nArguments\n\ntn: The TensorNetworkModel to query.\nusecuda: Specifies whether to use CUDA for tensor contraction.\nrescale: Specifies whether to rescale the tensors during contraction.\n\nExample\n\nThe following example is taken from examples/asia-network/main.jl.\n\njulia> model = read_model_file(pkgdir(TensorInference, \"examples\", \"asia-network\", \"model.uai\"));\n\njulia> tn = TensorNetworkModel(model; evidence=Dict(1=>0))\nTensorNetworkModel{Int64, DynamicNestedEinsum{Int64}, Array{Float64}}\nvariables: 1 (evidence → 0), 2, 3, 4, 5, 6, 7, 8\ncontraction time = 2^6.022, space = 2^2.0, read-write = 2^7.077\n\njulia> marginals(tn)\nDict{Vector{Int64}, Vector{Float64}} with 8 entries:\n [8] => [0.450138, 0.549863]\n [3] => [0.5, 0.5]\n [1] => [1.0]\n [5] => [0.45, 0.55]\n [4] => [0.055, 0.945]\n [6] => [0.10225, 0.89775]\n [7] => [0.145092, 0.854908]\n [2] => [0.05, 0.95]\n\njulia> tn2 = TensorNetworkModel(model; evidence=Dict(1=>0), mars=[[2, 3], [3, 4]])\nTensorNetworkModel{Int64, DynamicNestedEinsum{Int64}, Array{Float64}}\nvariables: 1 (evidence → 0), 2, 3, 4, 5, 6, 7, 8\ncontraction time = 2^7.781, space = 2^5.0, read-write = 2^8.443\n\njulia> marginals(tn2)\nDict{Vector{Int64}, Matrix{Float64}} with 2 entries:\n [2, 3] => [0.025 0.025; 0.475 0.475]\n [3, 4] => [0.05 0.45; 0.005 0.495]\n\nIn this example, we first set the evidence for variable 1 to 0 and then query the marginals of all individual variables. The returned dictionary has keys that correspond to the queried variables and values that represent their marginals. These marginals are vectors, with each entry corresponding to the probability of the variable taking a specific value. In this example, the possible values are 0 or 1. For the evidence variable 1, the marginal is always [1.0] since its value is fixed at 0.\n\nNext, we specify the marginal variables to query as variables 2 and 3, and variables 3 and 4, respectively. The joint marginals may or may not affect the contraction time and space. In this example, the contraction space complexity increases from 2^{2.0} to 2^{5.0}, and the contraction time complexity increases from 2^{5.977} to 2^{7.781}. The output marginals are the joint probabilities of the queried variables, represented by tensors.\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.maximum_logp","page":"Public","title":"TensorInference.maximum_logp","text":"maximum_logp(\n tn::TensorNetworkModel;\n usecuda\n) -> AbstractArray{<:Real}\n\n\nReturns an output array containing largest log-probabilities.\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.most_probable_config","page":"Public","title":"TensorInference.most_probable_config","text":"most_probable_config(\n tn::TensorNetworkModel;\n usecuda\n) -> Tuple{Real, Vector}\n\n\nReturns the largest log-probability and the most probable configuration.\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.probability","page":"Public","title":"TensorInference.probability","text":"probability(\n tn::TensorNetworkModel;\n usecuda,\n rescale\n) -> AbstractArray\n\n\nContract the tensor network and return an array of probability of evidence. Precisely speaking, the return value is the partition function, which may not be l1-normalized.\n\nIf the openvars of the input tensor networks is zero, the array rank is zero. Otherwise, the return values corresponds to marginal probabilities.\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.dataset_from_artifact","page":"Public","title":"TensorInference.dataset_from_artifact","text":"dataset_from_artifact(\n artifact_name::AbstractString\n) -> Dict{String, Dict{String, Dict{Int64, ArtifactProblemSpec}}}\n\n\nHelper function that captures the problem names that belong to problem_set for the given task.\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.problem_from_artifact","page":"Public","title":"TensorInference.problem_from_artifact","text":"problem_from_artifact(\n artifact_name::String,\n task::String,\n problem_set::String,\n problem_id::Int64\n) -> ArtifactProblemSpec\n\n\nGet artifact from artifact name, task name, problem set name and problem id.\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.read_model","page":"Public","title":"TensorInference.read_model","text":"read_model(problem::ArtifactProblemSpec; eltype) -> UAIModel\n\n\nRead an UAI model from an artifact.\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.read_evidence","page":"Public","title":"TensorInference.read_evidence","text":"read_evidence(\n problem::ArtifactProblemSpec\n) -> Dict{Int64, Int64}\n\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.read_solution","page":"Public","title":"TensorInference.read_solution","text":"read_solution(\n problem::ArtifactProblemSpec;\n factor_eltype\n) -> Any\n\n\nReturn the solution in the artifact.\n\nThe UAI file formats are defined in: https://uaicompetition.github.io/uci-2022/file-formats/\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.read_queryvars","page":"Public","title":"TensorInference.read_queryvars","text":"read_queryvars(\n problem::ArtifactProblemSpec\n) -> Vector{Int64}\n\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.read_model_file","page":"Public","title":"TensorInference.read_model_file","text":"read_model_file(\n model_filepath::AbstractString;\n factor_eltype\n) -> UAIModel\n\n\nParse the problem instance found in model_filepath defined in the UAI model format. If the provided file path is empty, return nothing.\n\nThe UAI file formats are defined in: https://uaicompetition.github.io/uci-2022/file-formats/\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.read_evidence_file","page":"Public","title":"TensorInference.read_evidence_file","text":"read_evidence_file(\n evidence_filepath::AbstractString\n) -> Tuple{Vector{Int64}, Vector{Int64}}\n\n\nReturn the observed variables and values in evidence_filepath. If the passed file path is an empty string, return empty vectors.\n\nThe UAI file formats are defined in: https://uaicompetition.github.io/uci-2022/file-formats/\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.read_td_file","page":"Public","title":"TensorInference.read_td_file","text":"read_td_file(\n td_filepath::AbstractString\n) -> Tuple{Int64, Int64, Int64, Vector{Vector{Int64}}, Vector{Vector{Int64}}}\n\n\nParse a tree decomposition instance described the PACE format.\n\nThe PACE file format is defined in: https://pacechallenge.org/2017/treewidth/\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.sample","page":"Public","title":"TensorInference.sample","text":"sample(\n tn::TensorNetworkModel,\n n::Int64;\n usecuda\n) -> TensorInference.Samples\n\n\nGenerate samples from a tensor network based probabilistic model. Returns a vector of vector, each element being a configurations defined on get_vars(tn).\n\nArguments\n\ntn is the tensor network model.\nn is the number of samples to be returned.\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.update_evidence!","page":"Public","title":"TensorInference.update_evidence!","text":"update_evidence!(\n tnet::TensorNetworkModel,\n evidence::Dict\n) -> TensorNetworkModel\n\n\nUpdate the evidence of a tensor network model, without changing the set of observed variables!\n\nArguments\n\ntnet is the TensorNetworkModel instance.\nevidence is the new evidence, the keys must be a subset of existing evidence.\n\n\n\n\n\n","category":"function"},{"location":"api/public/#TensorInference.update_temperature","page":"Public","title":"TensorInference.update_temperature","text":"update_temperature(\n tnet::TensorNetworkModel,\n problem::GenericTensorNetworks.GraphProblem,\n β::Real\n) -> TensorNetworkModel\n\n\nUpdate the temperature of a tensor network model. The program will regenerate tensors from the problem, without repeated optimizing the contraction order.\n\nArguments\n\ntnet is the TensorNetworkModel instance.\nproblem is the target constraint satisfiability problem.\nβ is the inverse temperature.\n\n\n\n\n\n","category":"function"},{"location":"uai-file-formats/#UAI-file-formats","page":"UAI file formats","title":"UAI file formats","text":"","category":"section"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"The UAI Format consists of four potential parts (each associated with a file):","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"Model file format (.uai).\nEvidence file format (.evid).\nQuery file format (.query).\nResults file format (.MAR, .MAP, .MMAP .PR).","category":"page"},{"location":"uai-file-formats/#Model-file-format-(.uai)","page":"UAI file formats","title":"Model file format (.uai)","text":"","category":"section"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"We use the simple text file format specified below to describe problem instances (Markov networks). The format is a generalization of the Ergo file format initially developed by Noetic Systems Inc. for their Ergo software. We use the .uai suffix for the evaluation benchmark network files.","category":"page"},{"location":"uai-file-formats/#Structure","page":"UAI file formats","title":"Structure","text":"","category":"section"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"A file in the UAI format consists of the following two parts, in that order:","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"\n\n","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"The contents of each section (denoted <...> above) are described in the following:","category":"page"},{"location":"uai-file-formats/#Preamble","page":"UAI file formats","title":"Preamble","text":"","category":"section"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"Our description of the format will follow a simple Markov network with three variables and two functions. A sample preamble for such a network is:","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"MARKOV\n3\n2 2 3\n2\n2 0 1\n2 1 2","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"The preamble starts with one line denoting the type of network. Generally, this can be either BAYES (if the network is a Bayesian network) or MARKOV (in case of a Markov network). However, note that this year all networks will be given in a Markov networks (i.e. Bayesian networks will be moralized).","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"The second line contains the number of variables. The next line specifies the cardinalities of each variable, one at a time, separated by a whitespace (note that this implies an order on the variables which will be used throughout the file). The fourth line contains only one integer, denoting the number of cliques in the problem. Then, one clique per line, the scope of each clique is given as follows: The first integer in each line specifies the number of variables in the clique, followed by the actual indexes of the variables. The order of this list is not restricted. Note that the ordering of variables within a factor will follow the order provided here.","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"Referring to the example above, the first line denotes the Markov network, the second line tells us the problem consists of three variables, let's refer to them as X, Y, and Z. Their cardinalities are 2, 2, and 3 respectively (from the third line). Line four specifies that there are 2 cliques. The first clique is X,Y, while the second clique is Y,Z. Note that variables are indexed starting with 0.","category":"page"},{"location":"uai-file-formats/#Function-tables","page":"UAI file formats","title":"Function tables","text":"","category":"section"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"In this section each factor is specified by giving its full table (i.e, specifying value for each assignment). The order of the factor is identical to the one in which they were introduced in the preamble, the first variable have the role of the 'most significant' digit. For each factor table, first the number of entries is given (this should be equal to the product of the domain sizes of the variables in the scope). Then, one by one, separated by whitespace, the values for each assignment to the variables in the function's scope are enumerated. Tuples are implicitly assumed in ascending order, with the last variable in the scope as the 'least significant'. To illustrate, we continue with our Markov network example from above, let's assume the following conditional probability tables:","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"X P(X)\n0 0.436\n1 0.564\n\nX Y P(Y,X)\n0 0 0.128\n0 1 0.872\n1 0 0.920\n1 1 0.080\n\nY Z P(Z,Y)\n0 0 0.210\n0 1 0.333\n0 2 0.457\n1 0 0.811\n1 1 0.000\n1 2 0.189","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"The corresponding function tables in the file would then look like this:","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"2\n0.436 0.564\n\n4\n0.128 0.872\n0.920 0.080\n\n6\n0.210 0.333 0.457\n0.811 0.000 0.189","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"(Note that line breaks and empty lines are effectively just a whitespace, exactly like plain spaces \" \". They are used here to improve readability.)","category":"page"},{"location":"uai-file-formats/#Summary","page":"UAI file formats","title":"Summary","text":"","category":"section"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"To sum up, a problem file consists of 2 sections: the preamble and the full the function tables, the names and the labels. For our Markov network example above, the full file will look like:","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"MARKOV\n3\n2 2 3\n3\n1 0\n2 0 1\n2 1 2\n\n2\n0.436 0.564\n\n4\n0.128 0.872\n0.920 0.080\n\n6\n0.210 0.333 0.457\n0.811 0.000 0.189","category":"page"},{"location":"uai-file-formats/#Evidence-file-format-(.evid)","page":"UAI file formats","title":"Evidence file format (.evid)","text":"","category":"section"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"Evidence is specified in a separate file. This file has the same name as the original network file but with an added .evid suffix. For instance, problem.uai will have evidence in problem.uai.evid. The file starts with a line specifying the number of evidences samples. The evidence in each sample, will be written in a new line. Each line will begin with the number of observed variables in the sample, followed by pairs of variable and its observed value. The indexes correspond to the ones implied by the original problem file. If, for our above example, we want to provide a single sample where the variable Y has been observed as having its first value and Z with its second value, the file example.uai.evid would contain the following:","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"1\n2 1 0 2 1","category":"page"},{"location":"uai-file-formats/#Query-file-format-(.query)","page":"UAI file formats","title":"Query file format (.query)","text":"","category":"section"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"Query variables for marginal MAP (MMAP) inference are specified in a separate file. This file has the same name as the original network file but with an added .query suffix. For instance with respect to the UAI model format, problem.uai will have evidence in problem.uai.query.","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"The query file consists of a single line. The line will begin with the number of query variables, followed by the indexes of the query variables. The indexes correspond to the ones implied by the original problem file.","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"For our example Markov network given Model Format, if we wanted to use Y as the query variable the file example.uai.query would contain the following:","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"1 1","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"As a second example, if variables with indices 0, 4, 8 and 17 are query variables, the query file would contain the following:","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"4 0 4 8 17","category":"page"},{"location":"uai-file-formats/#Results-file-format-(.MAR,-.MAP,-.MMAP-.PR)","page":"UAI file formats","title":"Results file format (.MAR, .MAP, .MMAP .PR)","text":"","category":"section"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"The rest of the file will contain the solution for the task. The first line must contain one of the tasks (PR, MPE, MAR, MMAP, or MLC) solved.","category":"page"},{"location":"uai-file-formats/#Marginals,-MAR","page":"UAI file formats","title":"Marginals, MAR","text":"","category":"section"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"A space separated line that includes:","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"The number of variables in the model.\nA list of marginal approximations of all the variables. For each variable its cardinality is first stated, then the probability of each state is stated. The order of the variables is the same as in the model, all data is space separated.\nFor example, a model with 3 variables, with cardinalities of 2, 2, 3 respectively. The solution might look like this:\n 3 2 0.1 0.9 2 0.3 0.7 3 0.2 0.2 0.6","category":"page"},{"location":"uai-file-formats/#Marginal-MAP,-MMAP","page":"UAI file formats","title":"Marginal MAP, MMAP","text":"","category":"section"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"A space separated line that includes:","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"The number of query variables.\nthe most probable instantiation, a list of variable value pairs for all bmQ variables.\nFor example, if the solution is an assignment of 0, 1 and 0 to three query variables indexed by 2 3 and 4 respectively, the solution will look as follows:\n 3 2 0 3 1 4 0","category":"page"},{"location":"uai-file-formats/#Partition-function,-PR","page":"UAI file formats","title":"Partition function, PR","text":"","category":"section"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"Line with the value of the log_10 of the partition function.","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"For example, an approximation log_10 Pr(bme) = -02008, which is known to be an upper bound may have a solution line:","category":"page"},{"location":"uai-file-formats/","page":"UAI file formats","title":"UAI file formats","text":"-0.2008","category":"page"},{"location":"probabilistic-inference/#Probabilistic-inference","page":"Probabilistic Inference","title":"Probabilistic inference","text":"","category":"section"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"TensorInference implements efficient methods to perform Bayesian inference in probabilistic graphical models, such as Bayesian Networks or Markov random fields. This page introduces probabilistic graphical models, provides an example using a Bayesian network, and explains what probabilistic inference is, including the different tasks it can involve.","category":"page"},{"location":"probabilistic-inference/#Probabilistic-graphical-models","page":"Probabilistic Inference","title":"Probabilistic graphical models","text":"","category":"section"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"A probabilistic graphical model (PGM) is a mathematical framework that uses graphs to compactly represent complex multivariate statistical distributions. They are used to reason in the presence of uncertainty. This reasoning process is known as probabilistic inference and will be defined and discussed in detail later on.","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"Bayesian networks and Markov random fields are popular types of PGMs. The following PGM is an example of a Bayesian network called the ASIA network. It was introduced by Lauritzen in 1988 [lauritzen1988local].","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"using TikzPictures\n\ntp = TikzPicture(\n L\"\"\"\n % The various elements are conveniently placed using a matrix:\n \\matrix[row sep=0.5cm,column sep=0.5cm] {\n % First line\n \\node (a) [myvar] {$A$}; &\n &\n &\n \\node (s) [myvar] {$S$}; &\n \\\\\n % Second line\n \\node (t) [myvar] {$T$}; &\n &\n \\node (l) [myvar] {$L$}; &\n &\n \\node (b) [myvar] {$B$}; \\\\\n % Third line\n &\n \\node (e) [myvar] {$E$}; &\n &\n &\n \\\\\n % Forth line\n \\node (x) [myvar] {$X$}; &\n &\n &\n \\node (d) [myvar] {$D$}; &\n \\\\\n };\n\n \\draw [myarrow] (a) edge (t);\n \\draw [myarrow] (s) edge (l);\n \\draw [myarrow] (s) edge (b);\n \\draw [myarrow] (t) edge (e);\n \\draw [myarrow] (l) edge (e);\n \\draw [myarrow] (e) edge (x);\n \\draw [myarrow] (e) edge (d);\n \\draw [myarrow] (b) edge (d);\n \"\"\",\n options=\"every node/.style={scale=1.5}\",\n preamble=\"\\\\input{\" * joinpath(@__DIR__, \"assets\", \"preambles\", \"asia-network\") * \"}\",\n)\nsave(SVG(joinpath(@__DIR__, \"asia-bayesian-network\")), tp)","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"(Image: )","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"Random variable Meaning\nA Recent trip to Asia\nT Patient has tuberculosis\nS Patient is a smoker\nL Patient has lung cancer\nB Patient has bronchitis\nE Patient hast T and/or L\nX Chest X-Ray is positive\nD Patient has dyspnoea","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"This network represents a simplified example from the realm of medical diagnosis, illustrating the probabilistic relationships between various random variables that correspond to potential diseases, symptoms, risk factors, and test results. It comprises a graph G = (bmVmathcalE) and a probability distribution P(bmV), where G is a directed acyclic graph, bmV represents the set of variables, and mathcalE is the set of edges connecting these variables. We assume all variables are discrete. Each variable V is quantified by a conditional probability distribution (CPD) P(V mid pa(V)), where pa(V) denotes the parent variables of V. Collectively, these conditional probability distributions, together with the graph G, induce a joint probability distribution over P(bmV), given by","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"P(bmV) = prod_VinbmV P(V mid pa(V))","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"A factor, denoted as phi_bmV, is defined over a set of variables bmV. It's a function that maps each instantiation bmV = bmv to a non-negative number. It's important to note that a probability distribution is a specific case of a factor. The product of two factors, phi_bmX and phi_bmY, is another factor, phi_bmZ, where bmZ = bmX cup bmY, and phi_bmZ(bmz) = phi_bmX(bmx)phi_bmY(bmy) for the instantiations bmx and bmy that align with the instantiation bmz. The marginalization of a factor phi_bmY into bmX subseteq bmY results in a new factor phi_bmX, where each phi_bmX(bmx) is calculated by summing the values of phi_bmY(bmy) for all bmy that are consistent with bmx. Importantly, factor marginalization and product operations form the fundamental basis for conducting probabilistic inference in PGMs.","category":"page"},{"location":"probabilistic-inference/#The-inference-tasks","page":"Probabilistic Inference","title":"The inference tasks","text":"","category":"section"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"Probabilistic inference is the process of determining the probability distribution of a set of unknown variables, given the values of known variables in a PGM. It encompasses several tasks that will be explained next.","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"Each task is performed with respect to a graphical model, denoted as G = bmV bmD bmphi, where:","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"bmV = V_1 V_2 dots V_N is the set of the model’s variables","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"bmD = D_V_1 D_V_2 dots D_V_N is the set of discrete domains for each variable, and","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"bmphi = phi_1 phi_2 dots phi_N is the set of factors that define the joint probability distribution of the model.","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"The variable set bmV can be further partitioned into two subsets: the evidence variables bmE and the remaining variables bmV^prime = bmV setminus bmE. Furthermore, within the set bmV^prime, the subset bmQ denotes the query variables. These are the variables for which we aim to estimate or infer values.","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"using TikzPictures\n\ntp = TikzPicture(\n L\"\"\"\n %\\draw[help lines] (0,0) grid (10,-7);\n\n % mrv: the \"node distances\" refer to the distance between the edge of a shape\n % to the edge of the other shape. That is why I use \"ie_aux\" and \"tasks_aux\"\n % below: to have equal distances between nodes with respect to the center of\n % the shapes.\n\n % row 1\n \\node[myroundbox] (rv) {Random Variables\\\\$\\bm{V}$};\n \\node[right=of rv](aux1) {};\n \\node[right=of aux1,myroundbox] (jd) {Joint Distribution\\\\$P(\\bm{V})$};\n \\node[right=of jd](aux2) {};\n \\node[right=of aux2,myroundbox] (e) {Evidence\\\\$\\bm{E=e}$};\n \\node[right=of e](aux3) {};\n \\node[right=of aux3,myroundbox] (qv) {Query Variables\\\\$\\bm{Q}$};\n % row 2\n \\node[below=of aux2,myrectbox] (ie) {Inference Engine};\n \\node[below=of aux2] (ie_aux) {};\n % row 3\n \\node[below=of ie_aux] (tasks_aux) {};\n \\node[left=of tasks_aux,myroundbox] (mar) {MAR};\n \\node[left=of mar] (aux4) {};\n \\node[left=of aux4,myroundbox] (pr) {PR};\n \\node[right=of tasks_aux,myroundbox] (map) {MAP};\n \\node[right=of map] (aux5) {};\n \\node[right=of aux5,myroundbox] (mmap) {MMAP};\n % row 0\n \\node[above=of aux2,yshift=-12mm,text=gray] (in) {\\textbf{Input}};\n % row 4\n \\node[below=of tasks_aux,yshift=7mm,text=gray] (out) {\\textbf{Output}};\n\n %% edges\n \\draw[myarrow] (rv) -- (ie);\n \\draw[myarrow] (jd) -- (ie);\n \\draw[myarrow] (e) -- (ie);\n \\draw[myarrow] (qv) -- (ie);\n \\draw[myarrow] (ie) -- (pr);\n \\draw[myarrow] (ie) -- (mar);\n \\draw[myarrow] (ie) -- (map);\n \\draw[myarrow] (ie) -- (mmap);\n \"\"\",\n options=\"transform shape, scale=1.8\",\n preamble=\"\\\\input{\" * joinpath(@__DIR__, \"assets\", \"preambles\", \"the-inference-tasks\") * \"}\",\n)\nsave(SVG(\"the-inference-tasks\"), tp)","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"(Image: )","category":"page"},{"location":"probabilistic-inference/#Probability-of-evidence-(PR)","page":"Probabilistic Inference","title":"Probability of evidence (PR)","text":"","category":"section"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"Computing the partition function (ie. normalizing constant) or probability of evidence:","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"PR(bmV^prime mid bmE=bme) = sum_V^prime in bmV^prime prod_phi in bmphi phi(V^primebme)","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"This task involves calculating the probability of the observed evidence, which can be useful for model comparison or anomaly detection. This involves summing the joint probability over all possible states of the unobserved variables in the model, given some observed variables. This is a fundamental task in Bayesian statistics and is often used as a stepping stone for other types of inference.","category":"page"},{"location":"probabilistic-inference/#Marginal-inference-(MAR):","page":"Probabilistic Inference","title":"Marginal inference (MAR):","text":"","category":"section"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"Computing the marginal probability distribution over all variables given evidence:","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"MAR(V_i mid bmE=bme) = frac sum_V^primeprime in bmV^prime\nsetminus V_i prod_phi in bmphi phi(V^primeprimebme) \n PR(bmV^prime mid bmE=bme) ","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"This task involves computing the marginal probability of a subset of variables, integrating out the others. In other words, it computes the probability distribution of some variables of interest regardless of the states of all other variables. This is useful when we're interested in the probabilities of some specific variables in the model, but not the entire model.","category":"page"},{"location":"probabilistic-inference/#Maximum-a-Posteriori-Probability-estimation-(MAP)","page":"Probabilistic Inference","title":"Maximum a Posteriori Probability estimation (MAP)","text":"","category":"section"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"Computing the most likely assignment to all variables given evidence:","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"MAP(V_i mid bmE=bme) = arg max_V^prime in bmV^prime\nprod_phi in bmphi phi(V^primebme)","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"In the MAP task, given some observed variables, the goal is to find the most probable assignment of values to some subset of the unobserved variables. It provides the states of variables that maximize the posterior probability given some observed evidence. This is often used when we want the most likely explanation or prediction according to the model.","category":"page"},{"location":"probabilistic-inference/#Marginal-Maximum-a-Posteriori-(MMAP)","page":"Probabilistic Inference","title":"Marginal Maximum a Posteriori (MMAP)","text":"","category":"section"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"Computing the most likely assignment to the query variables, bmQ subset bmV^prime after marginalizing out the remaining variables bmZ = bmV^prime setminus bmQ, also known as hidden or latent variables:","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"MMAP(V_i mid bmE=e) = arg max_Q in bmQ sum_Z in bmZ\nprod_phi in bmphi phi(Q Z e)","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"This task is essentially a combination of the MAR and MAP tasks. The MMAP task involves finding the most probable assignment (the MAP estimate) for a subset of the variables, while marginalizing over (summing out) the remaining variables. This task is useful when we want to know the most likely state of some variables, but there's some uncertainty over others that we need to average out.","category":"page"},{"location":"probabilistic-inference/","page":"Probabilistic Inference","title":"Probabilistic Inference","text":"[lauritzen1988local]: Steffen L Lauritzen and David J Spiegelhalter. Local computations with probabilities on graphical structures and their application to expert systems. Journal of the Royal Statistical Society: Series B (Methodological), 50(2):157–194, 1988.","category":"page"},{"location":"examples-overview/#Examples","page":"Overview","title":"Examples","text":"","category":"section"},{"location":"examples-overview/","page":"Overview","title":"Overview","text":"Pages = [\n \"generated/asia-network/main.md\",\n \"generated/hard-core-lattice-gas/main.md\",\n]\nDepth = 1","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"EditURL = \"performance-tips.jl\"","category":"page"},{"location":"generated/performance-tips/#Performance-Tips","page":"Performance tips","title":"Performance Tips","text":"","category":"section"},{"location":"generated/performance-tips/#Optimize-contraction-orders","page":"Performance tips","title":"Optimize contraction orders","text":"","category":"section"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"Let us use a problem instance from the \"Promedus\" dataset of the UAI 2014 competition as an example.","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"using TensorInference\nproblem = problem_from_artifact(\"uai2014\", \"MAR\", \"Promedus\", 11)\nmodel, evidence = read_model(problem), read_evidence(problem);","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"Next, we select the tensor network contraction order optimizer.","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"optimizer = TreeSA(ntrials = 1, niters = 5, βs = 0.1:0.3:100)","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"TreeSA{Int64, StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, GreedyMethod{OMEinsumContractionOrders.MinSpaceOut}, Any}(20, 0.1:0.3:100.0, 1, 5, 1.0, 0.2, :greedy, 0, Any[], GreedyMethod{OMEinsumContractionOrders.MinSpaceOut}(OMEinsumContractionOrders.MinSpaceOut(), 1))","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"Here, we choose the local search based TreeSA algorithm, which often finds the smallest time/space complexity and supports slicing. One can type ?TreeSA in a Julia REPL for more information about how to configure the hyper-parameters of the TreeSA method, while the detailed algorithm explanation is in arXiv: 2108.05665. Alternative tensor network contraction order optimizers include","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"GreedyMethod (default, fastest in searching speed but worst in contraction complexity)\nKaHyParBipartite\nSABipartite","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"tn = TensorNetworkModel(model; optimizer, evidence);","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"The returned object tn contains a field code that specifies the tensor network with optimized contraction order. To check the contraction complexity, please type","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"contraction_complexity(tn)","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"Time complexity: 2^21.503163838633586\nSpace complexity: 2^16.0\nRead-write complexity: 2^18.649049128960407","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"The returned object contains log2 values of the number of multiplications, the number elements in the largest tensor during contraction and the number of read-write operations to tensor elements.","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"probability(tn)","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"exp(-19.322038772705984) * fill(1.0)","category":"page"},{"location":"generated/performance-tips/#Using-the-slicing-technique-to-reduce-the-memory-cost","page":"Performance tips","title":"Using the slicing technique to reduce the memory cost","text":"","category":"section"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"For large scale applications, it is also possible to slice over certain degrees of freedom to reduce the space complexity, i.e. loop and accumulate over certain degrees of freedom so that one can have a smaller tensor network inside the loop due to the removal of these degrees of freedom. In the TreeSA optimizer, one can set nslices to a value larger than zero to turn on this feature. As a comparison we slice over 5 degrees of freedom, which can reduce the space complexity by at most 5. In this application, the slicing achieves the largest possible space complexity reduction 5, while the time and read-write complexity are only increased by less than 1, i.e. the peak memory usage is reduced by a factor 32, while the (theoretical) computing time is increased by at a factor 2.","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"optimizer = TreeSA(ntrials = 1, niters = 5, βs = 0.1:0.3:100, nslices=5)\ntn = TensorNetworkModel(model; optimizer, evidence);\ncontraction_complexity(tn)","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"Time complexity: 2^22.021695728226906\nSpace complexity: 2^11.0\nRead-write complexity: 2^20.849576142049628","category":"page"},{"location":"generated/performance-tips/#Faster-Tropical-tensor-contraction-to-speed-up-MAP-and-MMAP","page":"Performance tips","title":"Faster Tropical tensor contraction to speed up MAP and MMAP","text":"","category":"section"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"One can enjoy the BLAS level speed provided by TropicalGEMM by importing the package with","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"using TropicalGEMM","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"The benchmark in the TropicalGEMM repo shows this performance is close to the theoretical optimal value. Its implementation on GPU is under development in Github repo CuTropicalGEMM.jl as a part of Open Source Promotion Plan summer program.","category":"page"},{"location":"generated/performance-tips/#Working-with-GPUs","page":"Performance tips","title":"Working with GPUs","text":"","category":"section"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"To upload the computation to GPU, you just add using CUDA before calling the solve function, and set the keyword argument usecuda to true.","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"julia> using CUDA\n[ Info: OMEinsum loaded the CUDA module successfully\n\njulia> marginals(tn; usecuda = true);","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"Functions support usecuda keyword argument includes","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"probability\nlog_probability\nmarginals\nmost_probable_config","category":"page"},{"location":"generated/performance-tips/#Benchmarks","page":"Performance tips","title":"Benchmarks","text":"","category":"section"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"For performance metrics, see the Performance evaluation section.","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"","category":"page"},{"location":"generated/performance-tips/","page":"Performance tips","title":"Performance tips","text":"This page was generated using Literate.jl.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"EditURL = \"main.jl\"","category":"page"},{"location":"generated/asia-network/main/#The-ASIA-network","page":"Asia Network","title":"The ASIA network","text":"","category":"section"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"The graph below corresponds to the ASIA network, a simple Bayesian model used extensively in educational settings. It was introduced by Lauritzen in 1988 [lauritzen1988local].","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"┌───┐ ┌───┐\n│ A │ ┌─┤ S ├─┐\n└─┬─┘ │ └───┘ │\n │ │ │\n ▼ ▼ ▼\n┌───┐ ┌───┐ ┌───┐\n│ T │ │ L │ │ B │\n└─┬─┘ └─┬─┘ └─┬─┘\n │ ┌───┐ │ │\n └──►│ E │◄──┘ │\n └─┬─┘ │\n┌───┐ │ ┌───┐ │\n│ X │◄──┴──►│ D │◄────┘\n└───┘ └───┘","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"The table below explains the meanings of each random variable used in the ASIA network model.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Random variable Meaning\nA Recent trip to Asia\nT Patient has tuberculosis\nS Patient is a smoker\nL Patient has lung cancer\nB Patient has bronchitis\nE Patient hast T and/or L\nX Chest X-Ray is positive\nD Patient has dyspnoea","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"We now demonstrate how to use the TensorInference.jl package for conducting a variety of inference tasks on the Asia network.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Import the TensorInference package, which provides the functionality needed for working with tensor networks and probabilistic graphical models.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"using TensorInference","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Load the ASIA network model from the asia.uai file located in the examples directory. See Model file format (.uai) for a description of the format of this file.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"model = read_model_file(pkgdir(TensorInference, \"examples\", \"asia-network\", \"model.uai\"))","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"UAIModel(nvars = 8, nfactors = 8)\n cards : [2, 2, 2, 2, 2, 2, 2, 2]\n factors : \n Factor(1), size = (2,)\n Factor(1, 2), size = (2, 2)\n Factor(3), size = (2,)\n Factor(3, 4), size = (2, 2)\n Factor(3, 5), size = (2, 2)\n Factor(2, 4, 6), size = (2, 2, 2)\n Factor(6, 7), size = (2, 2)\n Factor(5, 6, 8), size = (2, 2, 2)","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Create a tensor network representation of the loaded model.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"tn = TensorNetworkModel(model)","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"TensorNetworkModel{Int64, OMEinsum.DynamicNestedEinsum{Int64}, Array{Float64}}\nvariables: 1, 2, 3, 4, 5, 6, 7, 8\ncontraction time = 2^6.044, space = 2^2.0, read-write = 2^7.098","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Calculate the partition function. Since the factors in this model are normalized, the partition function is the same as the total probability, 1.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"probability(tn) |> first","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"1.0","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Calculate the marginal probabilities of each random variable in the model.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"marginals(tn)","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Dict{Vector{Int64}, Vector{Float64}} with 8 entries:\n [8] => [0.435971, 0.564029]\n [3] => [0.5, 0.5]\n [1] => [0.01, 0.99]\n [5] => [0.45, 0.55]\n [4] => [0.055, 0.945]\n [6] => [0.064828, 0.935172]\n [7] => [0.11029, 0.88971]\n [2] => [0.0104, 0.9896]","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Retrieve all the variables in the model.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"get_vars(tn)","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"8-element Vector{Int64}:\n 1\n 2\n 3\n 4\n 5\n 6\n 7\n 8","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Set the evidence: Assume that the \"X-ray\" result (variable 7) is negative. Since setting the evidence may affect the contraction order of the tensor network, recompute it.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"tn = TensorNetworkModel(model, evidence = Dict(7 => 0))","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"TensorNetworkModel{Int64, OMEinsum.DynamicNestedEinsum{Int64}, Array{Float64}}\nvariables: 1, 2, 3, 4, 5, 6, 7 (evidence → 0), 8\ncontraction time = 2^6.044, space = 2^2.0, read-write = 2^7.109","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Calculate the maximum log-probability among all configurations.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"maximum_logp(tn)","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"0-dimensional Array{Float64, 0}:\n-3.6522217920023303","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Generate 10 samples from the posterior distribution.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"sample(tn, 10)","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"10-element TensorInference.Samples{Int64}:\n [1, 1, 0, 1, 1, 1, 0, 1]\n [1, 1, 0, 1, 1, 1, 0, 0]\n [1, 1, 1, 1, 1, 1, 0, 1]\n [1, 1, 0, 0, 0, 0, 0, 0]\n [1, 0, 0, 1, 1, 0, 0, 1]\n [1, 1, 0, 0, 0, 0, 0, 0]\n [1, 1, 0, 1, 0, 1, 0, 0]\n [1, 1, 0, 0, 0, 0, 0, 0]\n [1, 1, 1, 1, 1, 1, 0, 1]\n [1, 1, 0, 0, 0, 0, 0, 0]","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Retrieve both the maximum log-probability and the most probable configuration.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"logp, cfg = most_probable_config(tn)","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"(-3.6522217920023303, [1, 1, 0, 0, 0, 0, 0, 0])","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Compute the most probable values of certain variables (e.g., 4 and 7) while marginalizing over others. This is known as Maximum a Posteriori (MAP) estimation.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"mmap = MMAPModel(model, evidence=Dict(7=>0), queryvars=[4,7])","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"MMAPModel{Int64, Array{Float64}}\nvariables: 4, 7 (evidence → 0)\nquery variables: [[1, 2, 6, 5, 3, 8]]\ncontraction time = 2^6.0, space = 2^2.0, read-write = 2^7.0","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Get the most probable configurations for variables 4 and 7.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"most_probable_config(mmap)","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"(-2.8754627318176693, [1, 0])","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"Compute the total log-probability of having lung cancer. The results suggest that the probability is roughly half.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"log_probability(mmap, [1, 0]), log_probability(mmap, [0, 0])","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"(-2.8754627318176693, -2.9206248010671856)","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"[lauritzen1988local]: Steffen L Lauritzen and David J Spiegelhalter. Local computations with probabilities on graphical structures and their application to expert systems. Journal of the Royal Statistical Society: Series B (Methodological), 50(2):157–194, 1988.","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"","category":"page"},{"location":"generated/asia-network/main/","page":"Asia Network","title":"Asia Network","text":"This page was generated using Literate.jl.","category":"page"},{"location":"performance-evaluation/#Performance-evaluation","page":"Performance evaluation","title":"Performance evaluation","text":"","category":"section"},{"location":"performance-evaluation/","page":"Performance evaluation","title":"Performance evaluation","text":"The graph below illustrates a comparison of the runtime performance of TensorInference.jl against Merlin [marinescu2022merlin], libDAI [mooij2010libdai], and JunctionTrees.jl [roa2022partial] libraries, specifically for the task of computing the marginal probabilities of all variables. Both Merlin and libDAI have previously participated in UAI inference competitions [gal2010summary][gogate2014uai], achieving favorable results. Additionally, we compared against JunctionTrees.jl, the predecessor of TensorInference.jl. The experiments were conducted on an Intel Core i9–9900K CPU @3.60GHz with 64 GB of RAM. Performance comparisons for other tasks will be added in the near future.","category":"page"},{"location":"performance-evaluation/","page":"Performance evaluation","title":"Performance evaluation","text":"","category":"page"},{"location":"performance-evaluation/","page":"Performance evaluation","title":"Performance evaluation","text":"The benchmark problems are arranged along the x-axis in ascending order of complexity, measured by the induced tree width. On average, TensorInference.jl achieves a speedup of 20 times across all problems. Notably, for the 10 most complex problems, the average speedup increases to 148 times, highlighting its superior scalability. The graph features a fitted linear curve in log-space to underscore the exponential improvement in computation time achieved by TensorInference.jl in comparison to the other alternatives. This speedup is primarily due to our package's unique approach: while traditional solvers typically focus only on minimizing space complexity (as quantified by the induced tree width), TensorInference.jl is designed to optimize for both time and space complexity. ","category":"page"},{"location":"performance-evaluation/#References","page":"Performance evaluation","title":"References","text":"","category":"section"},{"location":"performance-evaluation/","page":"Performance evaluation","title":"Performance evaluation","text":"[gal2010summary]: Gal Elidan and Amir Globerson. Summary of the 2010 UAI approximate inference challenge. 2010. [Online]. Available: https://www.cs.huji.ac.il/project/UAI10/summary.php [Accessed: 11 September 2023].","category":"page"},{"location":"performance-evaluation/","page":"Performance evaluation","title":"Performance evaluation","text":"[gogate2014uai]: Vibhav Gogate. UAI 2014 Probabilistic Inference Competition. 2014. [Online]. Available: https://www.ics.uci.edu/~dechter/softwares/benchmarks/Uai14/UAI2014Inference_Competition.pdf [Accessed: 11 September 2023].","category":"page"},{"location":"performance-evaluation/","page":"Performance evaluation","title":"Performance evaluation","text":"[marinescu2022merlin]: Radu Marinescu. Merlin. 2022. [Online]. Available: https://www.ibm.com/opensource/open/projects/merlin/ [Accessed: 11 September 2023].","category":"page"},{"location":"performance-evaluation/","page":"Performance evaluation","title":"Performance evaluation","text":"[mooij2010libdai]: Joris M. Mooij. libDAI: A Free and Open Source C++ Library for Discrete Approximate Inference in Graphical Models. Journal of Machine Learning Research, 11:2169-2173, Aug 2010. [Online]. Available: http://www.jmlr.org/papers/volume11/mooij10a/mooij10a.pdf.","category":"page"},{"location":"performance-evaluation/","page":"Performance evaluation","title":"Performance evaluation","text":"[roa2022partial]: Martin Roa-Villescas, Patrick W.A. Wijnings, Sander Stuijk, Henk Corporaal. \"Partial Evaluation in Junction Trees\". In: 2022 25th Euromicro Conference on Digital System Design (DSD), pp. 429-437, 2022. doi: 10.1109/DSD57027.2022.00064","category":"page"},{"location":"api/internal/#Internal-API","page":"Internal","title":"Internal API","text":"","category":"section"},{"location":"api/internal/#Index","page":"Internal","title":"Index","text":"","category":"section"},{"location":"api/internal/","page":"Internal","title":"Internal","text":"Types","category":"page"},{"location":"api/internal/","page":"Internal","title":"Internal","text":"Pages = [\"internal.md\"]\nOrder = [:type]","category":"page"},{"location":"api/internal/","page":"Internal","title":"Internal","text":"Functions","category":"page"},{"location":"api/internal/","page":"Internal","title":"Internal","text":"Pages = [\"internal.md\"]\nOrder = [:function]","category":"page"},{"location":"api/internal/#internal_Types","page":"Internal","title":"Types","text":"","category":"section"},{"location":"api/internal/","page":"Internal","title":"Internal","text":"Modules = [TensorInference]\nOrder = [:type]\nPublic = false","category":"page"},{"location":"api/internal/#TensorInference.Factor","page":"Internal","title":"TensorInference.Factor","text":"struct Factor{T, N}\n\nFields\n\nvars\nvals\n\nEncodes a discrete function over the set of variables vars that maps each instantiation of vars into a nonnegative number in vals.\n\n\n\n\n\n","category":"type"},{"location":"api/internal/#TensorInference.Samples","page":"Internal","title":"TensorInference.Samples","text":"struct Samples{L} <: AbstractVector{SubArray{Float64, 1, Matrix{Float64}, Tuple{Base.Slice{Base.OneTo{Int64}}, Int64}, true}}\n\nFields\n\nsamples::Matrix{Int64}\nlabels::Vector\nsetmask::BitVector\n\nThe sampled configurations are stored in samples, which is a vector of vector. labels is a vector of variable names for labeling configurations. The setmask is an boolean indicator to denote whether the sampling process of a variable is complete.\n\n\n\n\n\n","category":"type"},{"location":"api/internal/#internal_Functions","page":"Internal","title":"Functions","text":"","category":"section"},{"location":"api/internal/","page":"Internal","title":"Internal","text":"Modules = [TensorInference]\nOrder = [:function]\nPublic = false","category":"page"},{"location":"api/internal/#TensorInference.backward_sampling!-Tuple{Any, Tuple, Any, Any, TensorInference.Samples, Any}","page":"Internal","title":"TensorInference.backward_sampling!","text":"backward_sampling!(\n ixs,\n xs::Tuple,\n iy,\n y,\n samples::TensorInference.Samples,\n size_dict\n) -> TensorInference.Samples\n\n\nThe backward process for sampling configurations.\n\nixs and xs are labels and tensor data for input tensors,\niy and y are labels and tensor data for the output tensor,\nsamples is the samples generated for eliminated variables,\nsize_dict is a key-value map from tensor label to dimension size.\n\n\n\n\n\n","category":"method"},{"location":"api/internal/#TensorInference.backward_tropical!-Tuple{Any, Tuple, Vararg{Any, 4}}","page":"Internal","title":"TensorInference.backward_tropical!","text":"backward_tropical!(\n ixs,\n xs::Tuple,\n iy,\n y,\n ymask,\n size_dict\n) -> Vector{Any}\n\n\nThe backward rule for tropical einsum.\n\nixs and xs are labels and tensor data for input tensors,\niy and y are labels and tensor data for the output tensor,\nymask is the boolean mask for gradients,\nsize_dict is a key-value map from tensor label to dimension size.\n\n\n\n\n\n","category":"method"},{"location":"api/internal/#TensorInference.parse_mar_solution_file-Tuple{Vector{String}}","page":"Internal","title":"TensorInference.parse_mar_solution_file","text":"parse_mar_solution_file(\n rawlines::Vector{String};\n factor_eltype\n) -> Vector{Vector{Float64}}\n\n\nParse the solution marginals of all variables from the UAI MAR solution file. The order of the variables is the same as in the model definition.\n\nThe UAI file formats are defined in: https://uaicompetition.github.io/uci-2022/file-formats/\n\n\n\n\n\n","category":"method"},{"location":"api/internal/#TensorInference.read_query_file-Tuple{AbstractString}","page":"Internal","title":"TensorInference.read_query_file","text":"read_query_file(\n query_filepath::AbstractString\n) -> Vector{Int64}\n\n\nReturn the query variables in query_filepath. If the passed file path is an empty string, return an empty vector.\n\nThe UAI file formats are defined in: https://uaicompetition.github.io/uci-2022/file-formats/\n\n\n\n\n\n","category":"method"},{"location":"api/internal/#TensorInference.rescale_array-Union{Tuple{AbstractArray{T}}, Tuple{T}} where T","page":"Internal","title":"TensorInference.rescale_array","text":"rescale_array(tensor::AbstractArray{T}) -> RescaledArray\n\n\nReturns a rescaled array that equivalent to the input tensor.\n\n\n\n\n\n","category":"method"},{"location":"","page":"Home","title":"Home","text":"CurrentModule = TensorInference","category":"page"},{"location":"#TensorInference.jl","page":"Home","title":"TensorInference.jl","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"TensorInference is a standalone solver written in Julia, that harnesses tensor-based technology to implement state-of-the-art algorithms for probabilistic inference in graphical models. ","category":"page"},{"location":"#Package-features","page":"Home","title":"Package features","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Solutions to the most common probabilistic inference tasks, including:","category":"page"},{"location":"","page":"Home","title":"Home","text":"Probability of evidence (PR): Calculates the total probability of the observed evidence across all possible states of the unobserved variables.\nMarginal inference (MAR): Computes the probability distribution of a subset of variables, ignoring the states of all other variables.\nMaximum a Posteriori Probability estimation (MAP): Finds the most probable state of a subset of unobserved variables given some observed evidence.\nMarginal Maximum a Posteriori (MMAP): Finds the most probable state of a subset of variables, averaging out the uncertainty over the remaining ones.","category":"page"},{"location":"#Why-TensorInference.jl","page":"Home","title":"Why TensorInference.jl","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"A major challenge in developing intelligent systems is the ability to reason under uncertainty, a challenge that appears in many real-world problems across various domains, including artificial intelligence, medical diagnosis, computer vision, computational biology, and natural language processing. Reasoning under uncertainty involves calculating the probabilities of relevant variables while taking into account any information that is acquired. This process, which can be thought of as drawing global insights from local observations, is known as probabilistic inference.","category":"page"},{"location":"","page":"Home","title":"Home","text":"Probabilistic graphical models (PGMs) provide a unified framework to perform probabilistic inference. These models use graphs to represent the joint probability distribution of complex systems in a concise manner by exploiting the conditional independence between variables in the model. Additionally, they form the foundation for various algorithms that enable efficient probabilistic inference.","category":"page"},{"location":"","page":"Home","title":"Home","text":"However, even with the representational aid of PGMs, performing probabilistic inference remains an intractable endeavor on many real-world models. The reason is that performing probabilistic inference involves complex combinatorial optimization problems in very high dimensional spaces. To tackle these challenges, more efficient and scalable inference algorithms are needed.","category":"page"},{"location":"","page":"Home","title":"Home","text":"As an attempt to tackle the aforementioned challenges, we present TensorInference.jl, a Julia package for probabilistic inference that combines the representational capabilities of PGMs with the computational power of tensor networks. By harnessing the best of both worlds, TensorInference.jl aims to enhance the performance of probabilistic inference, thereby expanding the tractability spectrum of exact inference for more complex, real-world models.","category":"page"},{"location":"#Outline","page":"Home","title":"Outline","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Pages = [\n \"probabilisticinference.md\",\n \"tensornetwork.md\",\n \"uai-file-formats.md\",\n \"examples-overview.md\",\n \"performance.md\",\n \"api/public.md\",\n \"api/internal.md\",\n]\nDepth = 1","category":"page"}] } diff --git a/dev/tensor-networks/index.html b/dev/tensor-networks/index.html index 8b12860..c34b231 100644 --- a/dev/tensor-networks/index.html +++ b/dev/tensor-networks/index.html @@ -16,4 +16,4 @@ \end{align*}\]

Tensors are represented using multidimensional arrays of numbers with labeled dimensions. These labels correspond to the array's indices, which in turn represent the set of random variables that the tensor is a function of. Thus, in this context, the terms label, index, and variable are synonymous and hence used interchangeably.

What is a tensor network?

We now turn our attention to defining a tensor network, a mathematical object used to represent a multilinear map between tensors. This concept is widely employed in fields like condensed matter physics [Orus2014][Pfeifer2014], quantum simulation [Markov2008][Pan2022], and even in solving combinatorial optimization problems [Liu2023]. It's worth noting that we use a generalized version of the conventional notation, most commonly known through the eisnum function, which is commonly used in high-performance computing. Packages that implement this conventional notation include

This approach allows us to represent a broader range of sum-product multilinear operations between tensors, thus meeting the requirements of the PGM field.

Definition[Liu2023]: A tensor network is a multilinear map represented by the triple $\mathcal{N} = (\Lambda, \mathcal{T}, \bm{\sigma}_0)$, where:

  • $\Lambda$ is the set of variables present in the network $\mathcal{N}$.
  • $\mathcal{T} = \{ T^{(k)}_{\bm{\sigma}_k} \}_{k=1}^{M}$ is the set of input tensors, where each tensor $T^{(k)}_{\bm{\sigma}_k}$ is identified by a superscript $(k)$ and has an associated scope $\bm{\sigma}_k$.
  • $\bm{\sigma}_0$ specifies the scope of the output tensor.

More specifically, each tensor $T^{(k)}_{\bm{\sigma}_k} \in \mathcal{T}$ is labeled by a string $\bm{\sigma}_k \in \Lambda^{r \left(T^{(k)} \right)}$, where $r \left(T^{(k)} \right)$ is the rank of $T^{(k)}$. The multilinear map, also known as the contraction, applied to this triple is defined as

\[\texttt{contract}(\Lambda, \mathcal{T}, \bm{\sigma}_0) = \sum_{\bm{\sigma}_{\Lambda \setminus [\bm{\sigma}_0]}} \prod_{k=1}^{M} T^{(k)}_{\bm{\sigma}_k},\]

Notably, the summation extends over all instantiations of the variables that are not part of the output tensor.

As an example, consider matrix multiplication, which can be specified as a tensor network contraction:

\[ (AB)_{ik} = \texttt{contract}\left(\{i,j,k\}, \{A_{ij}, B_{jk}\}, ik\right),\]

Here, matrices $A$ and $B$ are input tensors labeled by strings $ij, jk \in \{i, j, k\}^2$. The output tensor is labeled by string $ik$. Summations run over indices $\Lambda \setminus [ik] = \{j\}$. The contraction corresponds to

\[ \texttt{contract}\left(\{i,j,k\}, \{A_{ij}, B_{jk}\}, ik\right) = \sum_j A_{ij}B_{jk},\]

In the einsum notation commonly used in various programming languages, this is equivalent to ij, jk -> ik.

Diagrammatically, a tensor network can be represented as an open hypergraph. In this diagram, a tensor maps to a vertex, and a variable maps to a hyperedge. Tensors sharing the same variable are connected by the same hyperedge for that variable. The diagrammatic representation of matrix multiplication is:

In this diagram, we use different colors to denote different hyperedges. Hyperedges for $i$ and $j$ are left open to denote variables in the output string $\bm{\sigma}_0$. The reason we use hyperedges rather than regular edges will become clear in the following star contraction example.

\[ \texttt{contract}(\{i,j,k,l\}, \{A_{il}, B_{jl}, C_{kl}\}, ijk) = \sum_{l}A_{il} - B_{jl} C_{kl}\]

The equivalent einsum notation employed by many programming languages is il, jl, kl -> ijk.

Since the variable $l$ is shared across all three tensors, a simple graph can't capture the diagram's complexity. The more appropriate hypergraph representation is shown below.

As a final note, our definition of a tensor network allows for repeated indices within the same tensor, which translates to self-loops in their corresponding diagrams.

Tensor network contraction orders

The performance of a tensor network contraction depends on the order in which the tensors are contracted. The order of contraction is usually specified by binary trees, where the leaves are the input tensors and the internal nodes represent the order of contraction. The root of the tree is the output tensor.

Numerous approaches have been proposed to determine efficient contraction orderings, which include:

Some of these have been implemented in the OMEinsum package. Please check Performance Tips for more details.

References

  • Orus2014Orús R. A practical introduction to tensor networks: Matrix product states and projected entangled pair states[J]. Annals of physics, 2014, 349: 117-158.
  • Markov2008Markov I L, Shi Y. Simulating quantum computation by contracting tensor networks[J]. SIAM Journal on Computing, 2008, 38(3): 963-981.
  • Pfeifer2014Pfeifer R N C, Haegeman J, Verstraete F. Faster identification of optimal contraction sequences for tensor networks[J]. Physical Review E, 2014, 90(3): 033315.
  • Gray2021Gray J, Kourtis S. Hyper-optimized tensor network contraction[J]. Quantum, 2021, 5: 410.
  • Kalachev2021Kalachev G, Panteleev P, Yung M H. Multi-tensor contraction for XEB verification of quantum circuits[J]. arXiv:2108.05665, 2021.
  • Pan2022Pan F, Chen K, Zhang P. Solving the sampling problem of the sycamore quantum circuits[J]. Physical Review Letters, 2022, 129(9): 090502.
  • Liu2023Liu J G, Gao X, Cain M, et al. Computing solution space properties of combinatorial optimization problems via generic tensor networks[J]. SIAM Journal on Scientific Computing, 2023, 45(3): A1239-A1270.
+ B_{jl} C_{kl}\]

The equivalent einsum notation employed by many programming languages is il, jl, kl -> ijk.

Since the variable $l$ is shared across all three tensors, a simple graph can't capture the diagram's complexity. The more appropriate hypergraph representation is shown below.

As a final note, our definition of a tensor network allows for repeated indices within the same tensor, which translates to self-loops in their corresponding diagrams.

Tensor network contraction orders

The performance of a tensor network contraction depends on the order in which the tensors are contracted. The order of contraction is usually specified by binary trees, where the leaves are the input tensors and the internal nodes represent the order of contraction. The root of the tree is the output tensor.

Numerous approaches have been proposed to determine efficient contraction orderings, which include:

Some of these have been implemented in the OMEinsum package. Please check Performance Tips for more details.

References

  • Orus2014Orús R. A practical introduction to tensor networks: Matrix product states and projected entangled pair states[J]. Annals of physics, 2014, 349: 117-158.
  • Markov2008Markov I L, Shi Y. Simulating quantum computation by contracting tensor networks[J]. SIAM Journal on Computing, 2008, 38(3): 963-981.
  • Pfeifer2014Pfeifer R N C, Haegeman J, Verstraete F. Faster identification of optimal contraction sequences for tensor networks[J]. Physical Review E, 2014, 90(3): 033315.
  • Gray2021Gray J, Kourtis S. Hyper-optimized tensor network contraction[J]. Quantum, 2021, 5: 410.
  • Kalachev2021Kalachev G, Panteleev P, Yung M H. Multi-tensor contraction for XEB verification of quantum circuits[J]. arXiv:2108.05665, 2021.
  • Pan2022Pan F, Chen K, Zhang P. Solving the sampling problem of the sycamore quantum circuits[J]. Physical Review Letters, 2022, 129(9): 090502.
  • Liu2023Liu J G, Gao X, Cain M, et al. Computing solution space properties of combinatorial optimization problems via generic tensor networks[J]. SIAM Journal on Scientific Computing, 2023, 45(3): A1239-A1270.
diff --git a/dev/uai-file-formats/index.html b/dev/uai-file-formats/index.html index beaf951..506c49e 100644 --- a/dev/uai-file-formats/index.html +++ b/dev/uai-file-formats/index.html @@ -49,4 +49,4 @@ 6 0.210 0.333 0.457 0.811 0.000 0.189

Evidence file format (.evid)

Evidence is specified in a separate file. This file has the same name as the original network file but with an added .evid suffix. For instance, problem.uai will have evidence in problem.uai.evid. The file starts with a line specifying the number of evidences samples. The evidence in each sample, will be written in a new line. Each line will begin with the number of observed variables in the sample, followed by pairs of variable and its observed value. The indexes correspond to the ones implied by the original problem file. If, for our above example, we want to provide a single sample where the variable Y has been observed as having its first value and Z with its second value, the file example.uai.evid would contain the following:

1
-2 1 0 2 1

Query file format (.query)

Query variables for marginal MAP (MMAP) inference are specified in a separate file. This file has the same name as the original network file but with an added .query suffix. For instance with respect to the UAI model format, problem.uai will have evidence in problem.uai.query.

The query file consists of a single line. The line will begin with the number of query variables, followed by the indexes of the query variables. The indexes correspond to the ones implied by the original problem file.

For our example Markov network given Model Format, if we wanted to use Y as the query variable the file example.uai.query would contain the following:

1 1

As a second example, if variables with indices 0, 4, 8 and 17 are query variables, the query file would contain the following:

4 0 4 8 17

Results file format (.MAR, .MAP, .MMAP .PR)

The rest of the file will contain the solution for the task. The first line must contain one of the tasks (PR, MPE, MAR, MMAP, or MLC) solved.

Marginals, MAR

A space separated line that includes:

  • The number of variables in the model.

  • A list of marginal approximations of all the variables. For each variable its cardinality is first stated, then the probability of each state is stated. The order of the variables is the same as in the model, all data is space separated.

  • For example, a model with 3 variables, with cardinalities of 2, 2, 3 respectively. The solution might look like this:

      3 2 0.1 0.9 2 0.3 0.7 3 0.2 0.2 0.6

Marginal MAP, MMAP

A space separated line that includes:

  • The number of query variables.

  • the most probable instantiation, a list of variable value pairs for all $\bm{Q}$ variables.

  • For example, if the solution is an assignment of 0, 1 and 0 to three query variables indexed by 2 3 and 4 respectively, the solution will look as follows:

      3 2 0 3 1 4 0

Partition function, PR

Line with the value of the $\log_{10}$ of the partition function.

For example, an approximation $\log_{10} Pr(\bm{e}) = -0.2008$, which is known to be an upper bound may have a solution line:

-0.2008
+2 1 0 2 1

Query file format (.query)

Query variables for marginal MAP (MMAP) inference are specified in a separate file. This file has the same name as the original network file but with an added .query suffix. For instance with respect to the UAI model format, problem.uai will have evidence in problem.uai.query.

The query file consists of a single line. The line will begin with the number of query variables, followed by the indexes of the query variables. The indexes correspond to the ones implied by the original problem file.

For our example Markov network given Model Format, if we wanted to use Y as the query variable the file example.uai.query would contain the following:

1 1

As a second example, if variables with indices 0, 4, 8 and 17 are query variables, the query file would contain the following:

4 0 4 8 17

Results file format (.MAR, .MAP, .MMAP .PR)

The rest of the file will contain the solution for the task. The first line must contain one of the tasks (PR, MPE, MAR, MMAP, or MLC) solved.

Marginals, MAR

A space separated line that includes:

  • The number of variables in the model.

  • A list of marginal approximations of all the variables. For each variable its cardinality is first stated, then the probability of each state is stated. The order of the variables is the same as in the model, all data is space separated.

  • For example, a model with 3 variables, with cardinalities of 2, 2, 3 respectively. The solution might look like this:

      3 2 0.1 0.9 2 0.3 0.7 3 0.2 0.2 0.6

Marginal MAP, MMAP

A space separated line that includes:

  • The number of query variables.

  • the most probable instantiation, a list of variable value pairs for all $\bm{Q}$ variables.

  • For example, if the solution is an assignment of 0, 1 and 0 to three query variables indexed by 2 3 and 4 respectively, the solution will look as follows:

      3 2 0 3 1 4 0

Partition function, PR

Line with the value of the $\log_{10}$ of the partition function.

For example, an approximation $\log_{10} Pr(\bm{e}) = -0.2008$, which is known to be an upper bound may have a solution line:

-0.2008