Skip to content

Commit

Permalink
Update the NetVips benchmarks
Browse files Browse the repository at this point in the history
  • Loading branch information
@kleisauke
kleisauke committed Nov 4, 2024
1 parent 5b31a8f commit 7c3b3ef
Show file tree
Hide file tree
Showing 6 changed files with 34 additions and 37 deletions.
2 changes: 2 additions & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@
*.sln.docstates
download/
build/native/pack/
tests/NetVips.Benchmarks/images
!tests/NetVips.Benchmarks/images/sample2.v

# User-specific files (MonoDevelop/Xamarin Studio)
*.userprefs
Expand Down
2 changes: 1 addition & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ keep entire images in memory, it's light. For example, the `NetVips` benchmark:
[NetVips.Benchmarks](https://github.com/kleisauke/net-vips/tree/master/tests/NetVips.Benchmarks)

Loads a large image, shrinks by 10%, sharpens, and saves again. On this test
`NetVips` is around 20 times faster than Magick.NET and 3 times faster than ImageSharp.
`NetVips` is around 22 times faster than Magick.NET and 4 times faster than ImageSharp.

The [libvips documentation](https://www.libvips.org/API/current/)
has a [chapter explaining how libvips opens files](
Expand Down
2 changes: 1 addition & 1 deletion docs/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ _disableFooter: true
<section>
<h2>Runs quickly and uses little memory</h2>
<p class="lead">NetVips is fast and needs little memory. The <a href="https://github.com/kleisauke/net-vips/tree/master/tests/NetVips.Benchmarks">NetVips.Benchmarks</a> project tests NetVips against Magick.NET and ImageSharp.
NetVips is around 20 times faster than Magick.NET and 3 times faster than ImageSharp.</p>
NetVips is around 22 times faster than Magick.NET and 4 times faster than ImageSharp.</p>
</section>
</div>
</div>
Expand Down
27 changes: 11 additions & 16 deletions tests/NetVips.Benchmarks/NetVips.Benchmarks.csproj
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,13 @@
</PropertyGroup>

<ItemGroup>
<PackageReference Include="BenchmarkDotNet" Version="0.13.10" />
<PackageReference Include="Magick.NET-Q8-AnyCPU" Version="13.4.0" />
<PackageReference Include="NetVips.Native" Version="8.15.0" Condition="'$(UseGlobalLibvips)' == 'false'" />
<PackageReference Include="SixLabors.ImageSharp" Version="3.0.2" />
<PackageReference Include="SkiaSharp" Version="2.88.6" />
<PackageReference Include="SkiaSharp.NativeAssets.Linux" Version="2.88.6" Condition="$([MSBuild]::IsOSPlatform('Linux'))" />
<PackageReference Include="System.Drawing.Common" Version="8.0.0" Condition="$([MSBuild]::IsOSPlatform('Windows'))" />
<PackageReference Include="BenchmarkDotNet" Version="0.14.0" />
<PackageReference Include="Magick.NET-Q8-AnyCPU" Version="14.0.0" />
<PackageReference Include="NetVips.Native" Version="8.16.0" Condition="'$(UseGlobalLibvips)' == 'false'" />
<PackageReference Include="SixLabors.ImageSharp" Version="3.1.5" />
<PackageReference Include="SkiaSharp" Version="2.88.8" />
<PackageReference Include="SkiaSharp.NativeAssets.Linux" Version="2.88.8" Condition="$([MSBuild]::IsOSPlatform('Linux'))" />
<PackageReference Include="System.Drawing.Common" Version="8.0.10" Condition="$([MSBuild]::IsOSPlatform('Windows'))" />
</ItemGroup>

<ItemGroup>
Expand All @@ -35,16 +35,11 @@
</ItemGroup>

<ItemGroup>
<ContentWithTargetPath Include="images\sample2.v">
<None Include="images\t.*">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
<TargetPath>sample2.v</TargetPath>
</ContentWithTargetPath>

<None Include="$(OutDir)\t.*">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
<Link>%(Filename)%(Extension)</Link>
<Visible>False</Visible>
</None>
<Link>%(Filename)%(Extension)</Link>
<Visible>False</Visible>
</None>
</ItemGroup>

</Project>
34 changes: 17 additions & 17 deletions tests/NetVips.Benchmarks/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,31 +10,31 @@ of VIPS library.

## Benchmarks

Run on 18/11/23 with libvips 8.15.0, Magick.NET 13.4.0, ImageSharp 3.0.2, SkiaSharp 2.88.6 and System.Drawing.Common 8.0.0.
Run on 04/11/24 with libvips 8.16.0, Magick.NET 14.0.0, ImageSharp 3.1.5, SkiaSharp 2.88.8 and System.Drawing.Common 8.0.10.

```
BenchmarkDotNet v0.13.10, Windows 11 (10.0.22631.2715/23H2/2023Update/SunValley3)
BenchmarkDotNet v0.14.0, Windows 11 (10.0.22631.4317/23H2/2023Update/SunValley3)
AMD Ryzen 9 7900, 1 CPU, 24 logical and 12 physical cores
.NET SDK 8.0.100
[Host] : .NET 8.0.0 (8.0.23.53103), X64 RyuJIT AVX2
.NET 8.0 CLI (NativeAOT) : .NET 8.0.0 (8.0.23.53103), X64 RyuJIT AVX2
.NET SDK 8.0.403
[Host] : .NET 8.0.10 (8.0.1024.46610), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
.NET 8.0 CLI (NativeAOT) : .NET 8.0.10 (8.0.1024.46610), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
Job=.NET 8.0 CLI (NativeAOT) Runtime=NativeAOT 8.0 Toolchain=.NET 8.0
```
| Method | input | output | Mean | Error | StdDev | Ratio | RatioSD |
|--------------------------- |------ |------- |------------:|---------:|---------:|------:|--------:|
| **NetVips** | **t.jpg** | **t2.jpg** | **105.43 ms** | **2.054 ms** | **2.522 ms** | **1.00** | **0.00** |
| Magick.NET | t.jpg | t2.jpg | 1,741.20 ms | 6.126 ms | 5.116 ms | 16.44 | 0.46 |
| ImageSharp | t.jpg | t2.jpg | 336.34 ms | 4.142 ms | 3.672 ms | 3.17 | 0.08 |
| SkiaSharp<sup>1</sup> | t.jpg | t2.jpg | 895.18 ms | 5.438 ms | 4.541 ms | 8.45 | 0.24 |
| System.Drawing<sup>2</sup> | t.jpg | t2.jpg | 1,435.57 ms | 2.944 ms | 2.754 ms | 13.55 | 0.37 |
| | | | | | | | |
| **NetVips** | **t.tif** | **t2.tif** | **65.26 ms** | **1.258 ms** | **1.235 ms** | **1.00** | **0.00** |
| Magick.NET | t.tif | t2.tif | 1,647.24 ms | 4.682 ms | 4.380 ms | 25.27 | 0.49 |
| ImageSharp | t.tif | t2.tif | 250.84 ms | 2.152 ms | 2.013 ms | 3.85 | 0.07 |
| System.Drawing<sup>2</sup> | t.tif | t2.tif | 1,212.72 ms | 4.588 ms | 4.292 ms | 18.61 | 0.36 |
| Method | input | output | Mean | Error | StdDev | Ratio | RatioSD |
|--------------------------- |------ |------- |------------:|----------:|----------:|------:|--------:|
| **NetVips** | **t.jpg** | **t2.jpg** | **92.70 ms** | **1.020 ms** | **0.904 ms** | **1.00** | **0.01** |
| Magick.NET | t.jpg | t2.jpg | 1,634.68 ms | 6.062 ms | 5.062 ms | 17.64 | 0.17 |
| ImageSharp | t.jpg | t2.jpg | 352.31 ms | 5.753 ms | 5.650 ms | 3.80 | 0.07 |
| SkiaSharp<sup>1</sup> | t.jpg | t2.jpg | 890.21 ms | 2.238 ms | 2.094 ms | 9.60 | 0.09 |
| System.Drawing<sup>2</sup> | t.jpg | t2.jpg | 3,220.71 ms | 9.438 ms | 8.367 ms | 34.75 | 0.34 |
| | | | | | | | |
| **NetVips** | **t.tif** | **t2.tif** | **54.46 ms** | **1.000 ms** | **1.228 ms** | **1.00** | **0.03** |
| Magick.NET | t.tif | t2.tif | 1,524.11 ms | 15.602 ms | 14.594 ms | 28.00 | 0.67 |
| ImageSharp | t.tif | t2.tif | 248.38 ms | 1.802 ms | 1.505 ms | 4.56 | 0.10 |
| System.Drawing<sup>2</sup> | t.tif | t2.tif | 3,000.09 ms | 13.126 ms | 10.961 ms | 55.12 | 1.23 |

<sup>1</sup> SkiaSharp does not have TIFF support, so I only tested with JPEG files.
<sup>2</sup> System.Drawing does not have a sharpening or convolution operation, so I skipped that part of the benchmark.
Expand Down
4 changes: 2 additions & 2 deletions tests/NetVips.Benchmarks/Runner.cs
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
using System;
using System.IO;
using BenchmarkDotNet.Running;

namespace NetVips.Benchmarks;
Expand All @@ -7,7 +7,7 @@ public class Runner
{
public static void Main(string[] args)
{
TestImage.BuildTestImages(AppDomain.CurrentDomain.BaseDirectory);
TestImage.BuildTestImages(Path.Combine(Directory.GetCurrentDirectory(), "images"));
BenchmarkRunner.Run<Benchmark>();
}
}

0 comments on commit 7c3b3ef

Please sign in to comment.