Skip to content

Commit

Permalink
docs: test counter (#187)
Browse files Browse the repository at this point in the history
docs: testing counter script

docs: moved helper functions from test_counter to utils file

docs: abstracted functions in test_counter

docs: added doc comments to test_counter script and utils file

docs: added info about how to run the script to testing read me

fix: moved analyzer package from dependencies to dev dependencies

fix: put dev dependencies in alphabetical order
  • Loading branch information
DE7924 authored Oct 15, 2024
1 parent 18ea9a2 commit 5987c21
Show file tree
Hide file tree
Showing 5 changed files with 575 additions and 9 deletions.
1 change: 1 addition & 0 deletions pubspec.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ dependencies:
web: ^1.0.0

dev_dependencies:
analyzer: ^6.7.0
build_runner: ^2.4.10
flutter_test:
sdk: flutter
Expand Down
84 changes: 84 additions & 0 deletions test/TESTING_README.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# Testing Conventions Flutter Components

### Helper Functions

As you are writing tests think about helper function you could write and add them to the `test_utils/utils.dart` file. This will help you and others write tests faster and more consistently.

- For golden tests
`goldenTest(GoldenFiles goldenFile, Widget widget, Type widgetType, String fileName, {bool darkMode = false})`
- For debugFillProperties tests
`debugFillPropertiesTest(Widget widget, Map<String, dynamic> debugFillProperties)`

### Groups

- Accessibility Tests
Semantic labels, touch areas, contrast ratios, etc.
- Content Tests
Finds the widget, parameter statuses, etc.
- Dimensions Tests
Size, padding, margin, alignment, etc.
- Styling Tests
Rendered colors, fonts, borders, radii etc.
- Interaction Tests
Gesture recognizers, taps, drags, etc.
- Golden Tests
Compares the rendered widget with the golden file. Use the `goldenTest()` function from test_utils/utils.dart.
- Performance Tests
Animation performance, rendering performance, data manupulation performance, etc.

### Testing File Template

```
import 'dart:ui';
import 'package:flutter/foundation.dart';
import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:zeta_flutter/zeta_flutter.dart';
import '../../../test_utils/test_app.dart';
import '../../../test_utils/tolerant_comparator.dart';
import '../../../test_utils/utils.dart';
void main() {
const String componentName = 'ENTER_COMPONENT_NAME (e.g. ZetaButton)';
const String parentFolder = 'ENTER_PARENT_FOLDER (e.g. button)';
const goldenFile = GoldenFiles(component: parentFolder);
setUpAll(() {
goldenFileComparator = TolerantComparator(goldenFile.uri);
});
group('$componentName Accessibility Tests', () {});
group('$componentName Content Tests', () {
final debugFillProperties = {
'': '',
};
debugFillPropertiesTest(
widget,
debugFillProperties,
);
});
group('$componentName Dimensions Tests', () {});
group('$componentName Styling Tests', () {});
group('$componentName Interaction Tests', () {});
group('$componentName Golden Tests', () {
goldenTest(goldenFile, widget, widgetType, 'PNG_FILE_NAME');
});
group('$componentName Performance Tests', () {});
}
```

### Test Visibility Table

You can find the test visibility table at the following path: 'test/scripts/output/test_table.mdx'

To generate the table run the following command from the root of the project:

```bash
dart test/scripts/test_counter.dart
```

#### Visibility Excel Sheet

https://zebra-my.sharepoint.com/:x:/p/de7924/Ea0l7BF7AzJJoBVPrg4cIswBZRyek6iNT3zzwDcLn-5ZGg?e=NTJIZU
241 changes: 241 additions & 0 deletions test/scripts/test_counter.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,241 @@
import 'dart:io';

import 'package:analyzer/dart/analysis/utilities.dart';
import 'package:analyzer/dart/ast/ast.dart';
import 'package:analyzer/dart/ast/visitor.dart';

import 'utils/utils.dart';

/// A visitor that recursively visits AST nodes to identify and process test groups.
///
/// This class extends `RecursiveAstVisitor<void>` and overrides necessary methods
/// to traverse the abstract syntax tree (AST) of Dart code. It is specifically
/// designed to locate and handle test groups within the code, which are typically
/// defined using the `group` function in test files.
///
/// By implementing this visitor, you can analyze the structure of your test files,
/// extract information about test groups, and perform any required operations on them.
/// This can be useful for generating reports, performing static analysis, or
/// automating certain tasks related to your test suite.
class TestGroupVisitor extends RecursiveAstVisitor<void> {
final List<Map<String, dynamic>> groups = [];

/// Visits a method invocation node in the abstract syntax tree (AST).
///
/// This method is typically used in the context of traversing or analyzing
/// Dart code. It processes a [MethodInvocation] node, which represents
/// a method call in the source code.
///
/// [node] - The [MethodInvocation] node to visit.
/// The method checks if the method invocation is one of the following:
/// - `group`
/// - `testWidgets`
/// - `test`
/// - `goldenTest`
/// - `debugFillPropertiesTest`
/// Then it extracts the group name and test names from the method invocation.
///
/// - Parameter node: The [MethodInvocation] node to visit.
@override
void visitMethodInvocation(MethodInvocation node) {
if (node.hasNullParentNode()) {
if (node.methodIsOneOf(['group'])) {
final groupName = node.getGroupName();
final groupBody = node.argumentList.arguments.last;

final tests = <Map<String, dynamic>>[];

if (groupBody is FunctionExpression) {
final body = groupBody.body;
if (body is BlockFunctionBody) {
body.block.visitChildren(TestVisitor(tests));
}
}

groups.add({
'group': groupName,
'tests': tests,
});
} else if (node.methodIsOneOf(['testWidgets', 'test', 'goldenTest', 'debugFillPropertiesTest'])) {
final testName = node.getTestName();

if (groups.any((el) => el['group'] == 'unorganised')) {
final unorganisedGroup = groups.firstWhere((el) => el['group'] == 'unorganised');
(unorganisedGroup['tests'] as List).add({
'name': testName,
});
} else {
groups.add({
'group': 'unorganised',
'tests': [
{
'name': testName,
},
],
});
}
}
}
super.visitMethodInvocation(node);
}
}

/// A visitor class that extends `RecursiveAstVisitor<void>` to traverse
/// the Abstract Syntax Tree (AST) of Dart code. This class is specifically
/// designed to extract test names from test files.
///
/// The `TestVisitor` class overrides necessary methods to visit nodes
/// in the AST and identify test definitions. It collects the names of
/// the tests, which can then be used for various purposes such as
/// generating test reports or running specific tests.
///
class TestVisitor extends RecursiveAstVisitor<void> {
TestVisitor(this.tests);

final List<Map<String, dynamic>> tests;

/// Visits a method invocation node in the abstract syntax tree (AST).
/// This method checks if the method invocation is one of the following:
/// - `testWidgets`
/// - `test`
/// - `goldenTest`
/// - `debugFillPropertiesTest`
/// Then it extracts the test name from the method invocation.
///
/// [node] - The [MethodInvocation] node to visit.
@override
void visitMethodInvocation(MethodInvocation node) {
if (node.methodIsOneOf(['testWidgets', 'test'])) {
final testName = node.getTestName();
tests.add({
'name': testName,
});
} else if (node.methodIsOneOf(['debugFillPropertiesTest'])) {
tests.add({
'name': node.getMethodName(),
});
} else if (node.methodIsOneOf(['goldenTest'])) {
tests.add({
'name': node.toString(),
});
}

super.visitMethodInvocation(node);
}
}

/// Generates an MDX (Markdown Extended) table representation of the test counts.
///
/// The function takes a nested map where the outer map's keys are test group names,
/// and the inner map's keys are test names with their corresponding integer counts.
///
/// Example input:
/// ```dart
/// {
/// "test/src/components\\banner\\banner_test.dart": {
/// "Accessibility": 3,
/// },
/// }
/// ```
///
/// Example output:
/// ```mdx
/// | Component | Accessibility | Content | Dimensions | Styling | Interaction | Golden | Performance | Unorganised | Total Tests |
/// | ----------- | ------------- | ------- | ---------- | ------- | ----------- | ------ | ----------- | ----------- | ----------- |
/// | Banner | 3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 3 |
/// | Total Tests | 3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 3 |
/// ```
///
/// Parameters:
/// - `testCount`: A map where the keys are test group names and the values are maps
/// of test names with their corresponding counts.
///
/// Returns:
/// - A string in MDX format representing the test counts in a table with totals.
String generateMDX(Map<String, Map<String, int>> testCount) {
final Map<String, int> groupTotals = {
'Accessibility': 0,
'Content': 0,
'Dimensions': 0,
'Styling': 0,
'Interaction': 0,
'Golden': 0,
'Performance': 0,
'unorganised': 0,
};

final List<String> data = [
'| Component | Accessibility | Content | Dimensions | Styling | Interaction | Golden | Performance | Unorganised | Total Tests |',
'| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |',
]
..addComponentRows(testCount, groupTotals)
..addCategoryTotalRow(testCount, groupTotals);

return data.join('\n');
}

/// Parses a collection of test files and returns a map where the keys are
/// strings and the values are lists of maps containing dynamic data.
///
/// The function takes an iterable of [FileSystemEntity] objects representing
/// the test files to be parsed. It processes these files asynchronously and
/// returns a [Future] that completes with a map. Each key in the map is a
/// string, and each value is a list of maps with string keys and dynamic values.
///
/// - Parameter testFiles: An iterable collection of [FileSystemEntity]
/// objects representing the test files to be parsed.
/// - Returns: A [Future] that completes with a map where the keys are strings
/// and the values are lists of maps containing dynamic data.
Future<TestGroups> parseTestFiles(Iterable<FileSystemEntity> testFiles) async {
final TestGroups testGroups = {};
for (final FileSystemEntity file in testFiles) {
final contents = await File(file.path).readAsString();
final parseResult = parseString(content: contents);
final visitor = TestGroupVisitor();
parseResult.unit.visitChildren(visitor);
testGroups[file.path] = visitor.groups;
}
return testGroups;
}

/// Counts the number of tests in each test group and returns a map with the counts.
///
/// - Parameters:
/// - testGroups: A map where the keys are group names and the values are lists of test maps.
/// - Returns: A map where the keys are component names and the values are maps containing the count of tests in each test group.
Map<String, Map<String, int>> countTests(TestGroups testGroups) {
final TestCount testCount = {};
testGroups.forEach((filePath, groups) {
final Map<String, int> groupCounts = {};
for (final group in groups) {
final groupName = group['group'] as String;
final tests = group['tests'] as List;
groupCounts[groupName] = tests.length;
}
testCount[filePath] = groupCounts;
});
return testCount;
}

void main() async {
// check for output directory and create if it doesn't exist
final Directory outputDirectory = await outputPath('test/scripts/output');

// get all test files
final Iterable<FileSystemEntity> testFiles = getTestFiles('test/src/components');

// parse each test file and extract test groups
final TestGroups testGroups = await parseTestFiles(testFiles);

// write test groups to file
await writeJSONToFile('${outputDirectory.path}/test_groups.json', testGroups);

// count the number of tests in each group
final TestCount testCount = countTests(testGroups);

// write test counts to file
await writeJSONToFile('${outputDirectory.path}/test_counts.json', testCount);

// generate MDX table
await writeMDXToFile('${outputDirectory.path}/test_table.mdx', generateMDX(testCount));
}
Loading

0 comments on commit 5987c21

Please sign in to comment.