Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add UR estimation to the osu!mania ruleset #22613

Closed
wants to merge 98 commits into from
Closed
Show file tree
Hide file tree
Changes from 57 commits
Commits
Show all changes
98 commits
Select commit Hold shift + click to select a range
7882423
add deviation estimation to mania's perfcalc
Natelytle Oct 13, 2022
4af6c26
add UR attribute, change initial min to 300UR
Natelytle Oct 13, 2022
6ea6ba2
bound deviation estimate to numbers greater than 0
Natelytle Oct 13, 2022
3d80168
fix hitwindows
Natelytle Oct 13, 2022
f4c7e9d
account for converts, add note counts
Natelytle Oct 13, 2022
dfd386c
use onlineid for contingency
Natelytle Oct 13, 2022
5296fb5
fix convert hit windows
Natelytle Oct 15, 2022
562954a
account for HR/EZ in deviation calculation
Natelytle Oct 15, 2022
49495db
use erfc approximation
Natelytle Oct 24, 2022
dd1754f
Add rudimentary difficulty scaling curve
Natelytle Oct 27, 2022
79b3541
Change curve
Natelytle Oct 27, 2022
3a51c53
Don't give PP to jittered plays
Natelytle Oct 29, 2022
fb7cdc4
merge master and change hit window formulas
Natelytle Jan 23, 2023
c5233c7
change hit window formulas
Natelytle Jan 23, 2023
8e78adb
fix judgement
Natelytle Jan 23, 2023
4a5b14f
lazer deviation is finally accurate woo
Natelytle Jan 24, 2023
82bf3e7
increase totalhits by 1
Natelytle Jan 24, 2023
522271b
quick legacy LN idea
Natelytle Jan 24, 2023
3a78b9e
Fix formatting
Natelytle Jan 27, 2023
6f5e03c
change legacy LN calculation
Natelytle Jan 27, 2023
c331058
switch to totalvalue
Natelytle Jan 27, 2023
bc7fd8d
add comments
Natelytle Jan 27, 2023
80128d2
rewrite everything with erf for simplicity
Natelytle Jan 31, 2023
91fe6a5
change isLegacy toggle
Natelytle Jan 31, 2023
8fb84c2
whop
Natelytle Jan 31, 2023
68cea94
readability bump
Natelytle Feb 6, 2023
4535b29
rename method
Natelytle Feb 6, 2023
7196325
Merge remote-tracking branch 'master/master' into maniastatacc
Natelytle Feb 7, 2023
b0cd729
fix convert OD with lazer judgements
Natelytle Feb 12, 2023
2b5ab41
Revert unrelated files changed in formatting commit
Natelytle Feb 12, 2023
fde20e1
Add desmos links to likelihood functions
Natelytle Feb 12, 2023
5a0aa24
Code quality overhaul
Natelytle Feb 13, 2023
9ff4c1b
fix const name
Natelytle Feb 13, 2023
01dbce6
More formatting
Natelytle Feb 13, 2023
db0fda0
Remove pointless erf approximation, fix lazer return statement
Natelytle Feb 16, 2023
2d7c99b
Massively improve estimation accuracy using a folded distribution
Natelytle Feb 17, 2023
33a73e3
update desmos, remove redundant parenthesis
Natelytle Feb 18, 2023
97da8cc
Reformatting, increase precision a lot by rewriting in terms of the l…
Natelytle Feb 19, 2023
08ceb02
Rename "Judgements" to "HitWindows"
Natelytle Feb 19, 2023
35119bc
Address reviews
Natelytle Feb 19, 2023
06a2358
account for mania tests
Natelytle Feb 19, 2023
11f6fd0
Fix issue with high misscount plays returning 115 UR
Natelytle Feb 20, 2023
15ca535
Fix incorrect logDiff function, make lazer tail multiplier a const
Natelytle Feb 20, 2023
1e272f1
Address abraker's review, add approximation for legacy LN tails
Natelytle Feb 21, 2023
bafb8f6
Address abraker's review, add approximation for legacy LN tails
Natelytle Feb 21, 2023
1f76d48
Merge remote-tracking branch 'origin/maniastatacc' into maniastatacc
Natelytle Feb 21, 2023
4a8d679
Fix NaNs (whoops)
Natelytle Feb 21, 2023
3a1c479
Avoid repeated code using structs and methods
Natelytle Feb 24, 2023
66529f6
Add comments
Natelytle Feb 24, 2023
bfc642e
Change types, remove pointless duplicate likelihood function
Natelytle Feb 25, 2023
ab8d19e
Fix erfc approximation
Natelytle Feb 25, 2023
6399b9b
expose hit windows publicly for testing purposes
Natelytle Mar 6, 2023
32a3878
Add tests
Natelytle Apr 16, 2023
d0d0aae
Edit comments
Natelytle Apr 16, 2023
34c2cc0
Add comments and fail messages to UR estimation tests
Natelytle Apr 16, 2023
4affdc6
Change UR scaling curve
Natelytle Apr 16, 2023
a10dccf
Revert to live scaling
Natelytle Apr 16, 2023
d44d893
Rename to IsConvert, add handling for converts below od4
Natelytle Apr 16, 2023
9de2585
Move comment
Natelytle Apr 16, 2023
6ace77a
Revert back to previous curve, add more tests
Natelytle Apr 17, 2023
ec19983
Update desmos
Natelytle Apr 17, 2023
d8d4cc7
Change curve to clamp PP to <700ur
Natelytle Apr 17, 2023
18cef8b
Change hit window test to not require a beatmap, test multiple ODs
Natelytle Apr 18, 2023
69b99f6
Change curve (again) to fix jittering edge cases
Natelytle Apr 18, 2023
6e3e522
Merge remote-tracking branch 'osumaster/master' into maniastatacc
Natelytle Apr 22, 2023
0a72493
Band-aid math error on stable LNs, add tail deviation multiplier
Natelytle Jul 30, 2023
a58470d
Fix stable LN logic
Natelytle Jul 31, 2023
f60317c
Nest Ternary into Math.Log
Eve-ning Jul 31, 2023
951861a
Shorten property fetching code
Eve-ning Jul 31, 2023
20de476
Fix incorrect function name logPNote
Eve-ning Jul 31, 2023
7ed9141
Fix incorrect legacyLogPHold function name
Eve-ning Jul 31, 2023
c20e150
Merge master
Natelytle Aug 1, 2023
0ed0915
Collapse boilerplate code to parse judgements
Eve-ning Aug 1, 2023
e9b02c2
Remove resource dependency on test
Eve-ning Aug 1, 2023
b131ac8
Move test case code closer to test callables
Eve-ning Aug 1, 2023
1d7dd8b
Expose getHitWindows for tests
Eve-ning Aug 1, 2023
515ff55
Narrow scope of hitWindows fns
Eve-ning Aug 1, 2023
d00d7fd
Narrow scope of computeEstimatedUr
Eve-ning Aug 1, 2023
3dea273
Update changed values
Eve-ning Aug 2, 2023
b2ad8ad
Add remark on null & exception throwable
Eve-ning Aug 2, 2023
35f6ffe
Improve testing
Eve-ning Aug 2, 2023
5864d87
Add comment justifying args on TestEdge
Eve-ning Aug 2, 2023
104eea3
Delete ur-estimation-test.osu
Eve-ning Aug 2, 2023
da36c2a
Remove unresolvable cref
Eve-ning Aug 2, 2023
dd4a3ec
Add test for More Max Less UR
Eve-ning Aug 2, 2023
9cc6a3a
Switch to using deviation on notes and heads for multiplier scaling
Natelytle Aug 2, 2023
95c7237
Remove IsConvert difficulty attribute
smoogipoo Aug 7, 2023
a2197e2
Store difficulty attribute values
smoogipoo Aug 7, 2023
a05c3b8
Update ManiaUnstableRateEstimationTest.cs
Eve-ning Aug 14, 2023
4eb307b
Merge pull request #1 from Eve-ning/maniastatacc-improve-test
Natelytle Aug 14, 2023
1e80f79
Fix non-existent attribute call
Natelytle Aug 14, 2023
52799d5
Merge remote-tracking branch 'osumaster/master' into maniastatacc
Natelytle Sep 4, 2023
75c295c
Remove hit windows scaling by rate, adjust tests
Natelytle Sep 4, 2023
5de7e42
Merge branch 'master' into maniastatacc
smoogipoo Sep 24, 2023
aa8b9cd
Merge master
Natelytle Dec 13, 2023
ad818a4
Increase readability
Natelytle Jan 15, 2024
4f2496f
Merge branch 'ppy:master' into maniastatacc
Natelytle May 16, 2024
0f47f46
Merge branch 'master' into maniastatacc
smoogipoo May 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
117 changes: 117 additions & 0 deletions osu.Game.Rulesets.Mania.Tests/ManiaUnstableRateEstimationTest.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
// Copyright (c) ppy Pty Ltd <[email protected]>. Licensed under the MIT Licence.
// See the LICENCE file in the repository root for full licence text.

#nullable disable

using NUnit.Framework;
using osu.Game.Beatmaps;
using osu.Game.Rulesets.Difficulty;
using osu.Game.Rulesets.Mania.Difficulty;
using osu.Game.Rulesets.Mania.Mods;
using osu.Game.Scoring;
using osu.Game.Tests.Beatmaps;
using System.Collections.Generic;
using System.IO;
using System.Reflection;
using osu.Framework.Extensions.ObjectExtensions;
using osu.Game.Beatmaps.Formats;
using osu.Game.IO;
using osu.Game.Rulesets.Mania.Scoring;
using osu.Game.Rulesets.Mods;
using osu.Game.Rulesets.Scoring;

namespace osu.Game.Rulesets.Mania.Tests
{
public class ManiaUnstableRateEstimationTest
{
private const string resource_namespace = "Testing.Beatmaps";
protected string ResourceAssembly => "osu.Game.Rulesets.Mania";

// Test that both SS scores and near 0% scores are handled properly, within a margin of +-0.001 UR
[TestCase(42.978515625d, new[] { 11847, 0, 0, 0, 0, 0 }, "ur-estimation-test")]
[TestCase(9523485.0d, new[] { 0, 0, 0, 0, 1, 11846 }, "ur-estimation-test")]
public void Test1(double expectedEstimatedUnstableRate, int[] judgements, string name)
=> TestUnstableRate(expectedEstimatedUnstableRate, judgements, name);

// General test to make sure UR estimation isn't changed by anything, within a margin of +-0.001 UR. Tests rate changes as well.
[TestCase(309.990234375d, new[] { 5336, 3886, 1661, 445, 226, 293 }, "ur-estimation-test")]
public void Test1ClockRateAdjusted(double expectedEstimatedUnstableRate, int[] judgements, string name)
=> TestUnstableRate(expectedEstimatedUnstableRate, judgements, name, new ManiaModDoubleTime());

// Compares the true hit windows to the hit windows computed manually in perfcalc, within a margin of error of +-0.000001ms.
[TestCase(7.0d, "ur-estimation-test")]
public void Test2(double overallDifficulty, string name)
=> TestHitWindows(overallDifficulty, name);

protected void TestUnstableRate(double expectedEstimatedUnstableRate, int[] judgementCounts, string name, params Mod[] mods)
{
DifficultyAttributes attributes = new ManiaDifficultyCalculator(new ManiaRuleset().RulesetInfo, getBeatmap(name)).Calculate(mods);

var judgements = new Dictionary<HitResult, int>
{
{ HitResult.Perfect, judgementCounts[0] },
{ HitResult.Great, judgementCounts[1] },
{ HitResult.Good, judgementCounts[2] },
{ HitResult.Ok, judgementCounts[3] },
{ HitResult.Meh, judgementCounts[4] },
{ HitResult.Miss, judgementCounts[5] }
};

ManiaPerformanceAttributes perfAttributes = new ManiaPerformanceCalculator().Calculate(new ScoreInfo(getBeatmap(name).BeatmapInfo)
{
Mods = mods,
Statistics = judgements
}, attributes);

// Platform-dependent math functions (Pow, Cbrt, Exp, etc) and advanced math functions (Erf, FindMinimum) may result in slight differences.
Assert.That(perfAttributes.EstimatedUr, Is.EqualTo(expectedEstimatedUnstableRate).Within(0.001), "The estimated mania UR differed from the expected value.");
}

protected void TestHitWindows(double overallDifficulty, string name)
{
DifficultyAttributes attributes = new ManiaDifficultyCalculator(new ManiaRuleset().RulesetInfo, getBeatmap(name)).Calculate();

var hitWindows = new ManiaHitWindows();
hitWindows.SetDifficulty(overallDifficulty);

double[] trueHitWindows =
{
hitWindows.WindowFor(HitResult.Perfect),
hitWindows.WindowFor(HitResult.Great),
hitWindows.WindowFor(HitResult.Good),
hitWindows.WindowFor(HitResult.Ok),
hitWindows.WindowFor(HitResult.Meh)
};

ManiaPerformanceAttributes perfAttributes = new ManiaPerformanceCalculator().Calculate(new ScoreInfo(getBeatmap(name).BeatmapInfo), attributes);

// Platform-dependent math functions (Pow, Cbrt, Exp, etc) may result in minute differences.
Assert.That(perfAttributes.HitWindows, Is.EqualTo(trueHitWindows).Within(0.000001), "The true mania hit windows are different to the ones calculated in ManiaPerformanceCalculator.");
}

private WorkingBeatmap getBeatmap(string name)
{
using (var resStream = openResource($"{resource_namespace}.{name}.osu"))
using (var stream = new LineBufferedReader(resStream))
{
var decoder = Decoder.GetDecoder<Beatmap>(stream);

((LegacyBeatmapDecoder)decoder).ApplyOffsets = false;

return new TestWorkingBeatmap(decoder.Decode(stream))
{
BeatmapInfo =
{
Ruleset = new ManiaRuleset().RulesetInfo
}
};
}
}

private Stream openResource(string name)
{
string localPath = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location).AsNonNull();
return Assembly.LoadFrom(Path.Combine(localPath, $"{ResourceAssembly}.dll")).GetManifestResourceStream($@"{ResourceAssembly}.Resources.{name}");
}
}
}
21 changes: 21 additions & 0 deletions osu.Game.Rulesets.Mania/Difficulty/ManiaDifficultyAttributes.cs
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,27 @@ public class ManiaDifficultyAttributes : DifficultyAttributes
[JsonProperty("great_hit_window")]
public double GreatHitWindow { get; set; }

/// <summary>
/// The perceived overall difficulty of the map.
/// </summary>
[JsonProperty("overall_difficulty")]
public double OverallDifficulty { get; set; }

/// <summary>
/// The number of notes in the beatmap.
/// </summary>
public int NoteCount { get; set; }

/// <summary>
/// The number of hold notes in the beatmap.
/// </summary>
public int HoldNoteCount { get; set; }

/// <summary>
/// Conversion status from standard.
/// </summary>
public bool Convert { get; set; }
Natelytle marked this conversation as resolved.
Show resolved Hide resolved

public override IEnumerable<(int attributeId, object value)> ToDatabaseAttributes()
{
foreach (var v in base.ToDatabaseAttributes())
Expand Down
10 changes: 9 additions & 1 deletion osu.Game.Rulesets.Mania/Difficulty/ManiaDifficultyCalculator.cs
Original file line number Diff line number Diff line change
Expand Up @@ -48,14 +48,22 @@ protected override DifficultyAttributes CreateDifficultyAttributes(IBeatmap beat
HitWindows hitWindows = new ManiaHitWindows();
hitWindows.SetDifficulty(beatmap.Difficulty.OverallDifficulty);

int noteCount = beatmap.HitObjects.Count(h => h is Note);
int holdNoteCount = beatmap.HitObjects.Count(h => h is HoldNote);
bool convert = beatmap.BeatmapInfo.Ruleset.OnlineID != 3;

return new ManiaDifficultyAttributes
{
StarRating = skills[0].DifficultyValue() * star_scaling_factor,
Mods = mods,
// In osu-stable mania, rate-adjustment mods don't affect the hit window.
// This is done the way it is to introduce fractional differences in order to match osu-stable for the time being.
GreatHitWindow = Math.Ceiling((int)(getHitWindow300(mods) * clockRate) / clockRate),
MaxCombo = beatmap.HitObjects.Sum(maxComboForObject)
MaxCombo = beatmap.HitObjects.Sum(maxComboForObject),
OverallDifficulty = beatmap.Difficulty.OverallDifficulty,
NoteCount = noteCount,
HoldNoteCount = holdNoteCount,
Convert = convert
};
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,12 @@ public class ManiaPerformanceAttributes : PerformanceAttributes
[JsonProperty("difficulty")]
public double Difficulty { get; set; }

[JsonProperty("estimated_ur")]
public double? EstimatedUr { get; set; }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as #20963, would prefer this to be called EstimatedUnstableRate

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.


[JsonProperty("hit_windows")]
public double[] HitWindows { get; set; }

public override IEnumerable<PerformanceDisplayAttribute> GetAttributesForDisplay()
{
foreach (var attribute in base.GetAttributesForDisplay())
Expand Down
Loading