Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding tests for sparse exploit codegen #928

Merged
merged 1 commit into from
Dec 2, 2024

Conversation

AlexRTer
Copy link
Collaborator

This PR adds some tests and makes some minor changes to the sparsity exploit lowering pass src/compiler/lowering/SparsityExploitationPass.cpp and is complementary to #919. For clarification, the currently fixed operations for the SparsityExploitationPass implement a cross-entropy computation of a sparse CSRMatrix and a DenseMatrix

Changes:

  • added script level and filecheck tests
  • replaced Operation * type with cast to specific Operation (auto op = parentOp.getDefiningOp<daphne::OpName>())
    • allows to make use of specific get functions for better readability (e.g. .getLhs(), .getTransa())
  • replaced notifyMatchFailure with ErrorHandler::compilerError

Complementing the original PR's description:
The lower-sparse-exploit pass is part of the mlir-codegen pipeline. It is a proof of concept with the aim to fuse operators that have a sparse operand that enables optimizations in the computation of its intermediates. For example, in an expression like sparse * (dense @ dense) the full dense result of the right matrix multiplication is not needed. It suffices to compute only the entries where the lhs sparse matrix is not zero. Using this information, a fused operator can avoid needless computations as well as materializing potentially very large dense intermediates.
Right now, the pass does this for a hard-coded pattern sum(CSRMat * ln(denseLhs @ t(denseRhs))) (transpose on both sides are optional), which could be generalized to something like IntersectOp(CSRMat, OuterBinary(DenseMat, DenseMat)). It runs a canonicalizer pass directly after lowering this pattern to avoid the lowering/execution of any redundant operations that are part of the pattern (see test/codegen/sparseExploit.mlir). These are handled in this separate pass as their results could still be relevant for other computations and are not generally trivially dead.

Here is an example script to test the pass (--explain mlir_codegen is optional):

// RUN: ./bin/daphne --select-matrix-repr --mlir-codegen --explain mlir_codegen ./sparseExploit.daphne

seed = 1;
sparsity = 0.05;
sparseRows = 10_000;
sparseCols = 1_000;
hiddenDim = 200;

startGeneratingMatrices = now();
sparseLhs = rand(sparseRows, sparseCols, 0.0, 1.0, sparsity, seed);

DenseU = rand(sparseRows, hiddenDim, 0.0, 1.0, 1.0, seed + 1); // sparsity: 1.0
DenseV = rand(sparseCols, hiddenDim, 0.0, 1.0, 1.0, seed + 2); // sparsity: 1.0
endGeneratingMatrices = now();

startCalc = now();
res = sum(sparseLhs * ln(DenseU @ t(DenseV)));
endCalc = now();

print(res);
print("sparse dim: " + sparseRows + "x" + sparseCols + " (sparsity: " + sparsity + "), dense dim: " + sparseRows + "x" + hiddenDim + "@" + hiddenDim + "x" + sparseCols + "->" + sparseRows + "x" + sparseCols);
print("time to generate matrices: " + as.f64(endGeneratingMatrices - startGeneratingMatrices) * 1e-9 + ", comp. time: " + as.f64(endCalc - startCalc) * 1e-9);

which shows a significant speedup on my machine from 0.17 seconds to 0.09 seconds to compute the result.

Currently, the pass lowers to an affine ParallelOp for the outer loop, but in theory all three loops can be parallelized. Note that this is not yet lowered to an actual multi-threaded implementation in MLIR (which requires additional lowering and linking of the respective libraries) and still runs single-threaded.

- added script level and filecheck tests
- replaced Operation ptr with cast to specific op to use `get` functions for readability
- replaced notifyMatchFailure with ErrorHandler::compilerError
@philipportner philipportner merged commit f2ba9c3 into daphne-eu:main Dec 2, 2024
3 checks passed
@philipportner
Copy link
Collaborator

Looks good, thanks @AlexRTer !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants