Add "Ignore missing trailing EOL" option to Compare settings #2573

sdottaka · 2024-12-11T00:00:36Z

This PR adds a new option, Ignore missing trailing EOL, to the Compare/General category in the Options window.

When this option is enabled, if one file ends with a newline character (CR, LF, or CRLF) while the other file does not have a trailing newline, the difference in the trailing newline is ignored as long as the content of the last line is identical in both files.

refs #203, #989, #2317

Testing/GoogleTest/ByteCompare/ByteCompare_test.cpp

+			}
+
+			{// diff left: no EOL - right: CR+A
+				std::vector<char> buf_left(WMCMPBUFF * 2 - 2 + i);
+				std::vector<char> buf_right(WMCMPBUFF * 2 + i);
+
+				memset(buf_left.data(), 'A', buf_left.size());
+				memset(buf_right.data(), 'A', buf_right.size());
+
+				buf_right[buf_right.size() - 2] = '\r';
+				buf_right[buf_right.size() - 1] = 'A';
+
+				TempFile file_left(filename_left, buf_left.data(), buf_left.size());
+				TempFile file_right(filename_right, buf_right.data(), buf_right.size());
+
+				FilePair pair(filename_left, filename_right);
+
+				EXPECT_EQ(DIFFCODE::TEXT | DIFFCODE::DIFF, bc.CompareFiles(&pair.diffData));
+			}
+		}
+
+		for (int i = 0; i < 3; i++)
+		{
+			{// diff left: LF+LF - right: no EOL
+				std::vector<char> buf_left(WMCMPBUFF * 2 + i);
+				std::vector<char> buf_right(WMCMPBUFF * 2 - 2 + i);
+
+				memset(buf_left.data(), 'A', buf_left.size());
+				memset(buf_right.data(), 'A', buf_right.size());
+
+				buf_left[buf_left.size() - 2] = '\n';
+				buf_left[buf_left.size() - 1] = '\n';
+
+				TempFile file_left(filename_left, buf_left.data(), buf_left.size());
+				TempFile file_right(filename_right, buf_right.data(), buf_right.size());
+
+				FilePair pair(filename_left, filename_right);
+
+				EXPECT_EQ(DIFFCODE::TEXT | DIFFCODE::DIFF, bc.CompareFiles(&pair.diffData));
+			}
+
+			{// diff left: no EOL - right: LF+LF
+				std::vector<char> buf_left(WMCMPBUFF * 2 - 2 + i);
+				std::vector<char> buf_right(WMCMPBUFF * 2 + i);
+
+				memset(buf_left.data(), 'A', buf_left.size());
+				memset(buf_right.data(), 'A', buf_right.size());
+
+				buf_right[buf_right.size() - 2] = '\n';
+				buf_right[buf_right.size() - 1] = '\n';
+
+				TempFile file_left(filename_left, buf_left.data(), buf_left.size());
+				TempFile file_right(filename_right, buf_right.data(), buf_right.size());
+
+				FilePair pair(filename_left, filename_right);
+
+				EXPECT_EQ(DIFFCODE::TEXT | DIFFCODE::DIFF, bc.CompareFiles(&pair.diffData));
+			}
+		}
+
+		for (int i = 0; i < 3; i++)
+		{
+			{// diff left: CR+CR - right: no EOL
+				std::vector<char> buf_left(WMCMPBUFF * 2 + i);
+				std::vector<char> buf_right(WMCMPBUFF * 2 - 2 + i);
+
+				memset(buf_left.data(), 'A', buf_left.size());
+				memset(buf_right.data(), 'A', buf_right.size());
+
+				buf_left[buf_left.size() - 2] = '\r';
+				buf_left[buf_left.size() - 1] = '\r';
+
+				TempFile file_left(filename_left, buf_left.data(), buf_left.size());
+				TempFile file_right(filename_right, buf_right.data(), buf_right.size());
+
+				FilePair pair(filename_left, filename_right);
+
+				EXPECT_EQ(DIFFCODE::TEXT | DIFFCODE::DIFF, bc.CompareFiles(&pair.diffData));
+			}
+
+			{// diff left: no EOL - right: CR+CR
+				std::vector<char> buf_left(WMCMPBUFF * 2 - 2 + i);
+				std::vector<char> buf_right(WMCMPBUFF * 2 + i);
+
+				memset(buf_left.data(), 'A', buf_left.size());
+				memset(buf_right.data(), 'A', buf_right.size());
+
+				buf_right[buf_right.size() - 2] = '\r';
+				buf_right[buf_right.size() - 1] = '\r';
+
+				TempFile file_left(filename_left, buf_left.data(), buf_left.size());
+				TempFile file_right(filename_right, buf_right.data(), buf_right.size());
+
+				FilePair pair(filename_left, filename_right);
+
+				EXPECT_EQ(DIFFCODE::TEXT | DIFFCODE::DIFF, bc.CompareFiles(&pair.diffData));
+			}
+		}
+
+	}


Src/CompareEngines/ByteComparator.cpp

Testing/GoogleTest/ByteCompare/ByteCompare_test.cpp

+				EXPECT_EQ(DIFFCODE::TEXT | DIFFCODE::DIFF, bc.CompareFiles(&pair.diffData));
+			}
+
+			{// diff left: no EOL - right: CR+A
+				std::vector<char> buf_left(WMCMPBUFF * 2 - 2 + i);
+				std::vector<char> buf_right(WMCMPBUFF * 2 + i);
+
+				memset(buf_left.data(), 'A', buf_left.size());
+				memset(buf_right.data(), 'A', buf_right.size());
+
+				buf_right[buf_right.size() - 2] = '\r';
+				buf_right[buf_right.size() - 1] = 'A';
+
+				TempFile file_left(filename_left, buf_left.data(), buf_left.size());
+				TempFile file_right(filename_right, buf_right.data(), buf_right.size());
+
+				FilePair pair(filename_left, filename_right);
+
+				EXPECT_EQ(DIFFCODE::TEXT | DIFFCODE::DIFF, bc.CompareFiles(&pair.diffData));
+			}
+		}
+
+		for (int i = 0; i < 3; i++)
+		{
+			{// diff left: LF+LF - right: no EOL
+				std::vector<char> buf_left(WMCMPBUFF * 2 + i);
+				std::vector<char> buf_right(WMCMPBUFF * 2 - 2 + i);
+
+				memset(buf_left.data(), 'A', buf_left.size());
+				memset(buf_right.data(), 'A', buf_right.size());
+
+				buf_left[buf_left.size() - 2] = '\n';
+				buf_left[buf_left.size() - 1] = '\n';
+
+				TempFile file_left(filename_left, buf_left.data(), buf_left.size());
+				TempFile file_right(filename_right, buf_right.data(), buf_right.size());
+
+				FilePair pair(filename_left, filename_right);
+
+				EXPECT_EQ(DIFFCODE::TEXT | DIFFCODE::DIFF, bc.CompareFiles(&pair.diffData));
+			}
+
+			{// diff left: no EOL - right: LF+LF
+				std::vector<char> buf_left(WMCMPBUFF * 2 - 2 + i);
+				std::vector<char> buf_right(WMCMPBUFF * 2 + i);
+
+				memset(buf_left.data(), 'A', buf_left.size());
+				memset(buf_right.data(), 'A', buf_right.size());
+
+				buf_right[buf_right.size() - 2] = '\n';
+				buf_right[buf_right.size() - 1] = '\n';
+
+				TempFile file_left(filename_left, buf_left.data(), buf_left.size());
+				TempFile file_right(filename_right, buf_right.data(), buf_right.size());
+
+				FilePair pair(filename_left, filename_right);
+
+				EXPECT_EQ(DIFFCODE::TEXT | DIFFCODE::DIFF, bc.CompareFiles(&pair.diffData));
+			}
+		}
+
+		for (int i = 0; i < 3; i++)
+		{
+			{// diff left: CR+CR - right: no EOL
+				std::vector<char> buf_left(WMCMPBUFF * 2 + i);
+				std::vector<char> buf_right(WMCMPBUFF * 2 - 2 + i);
+
+				memset(buf_left.data(), 'A', buf_left.size());
+				memset(buf_right.data(), 'A', buf_right.size());
+
+				buf_left[buf_left.size() - 2] = '\r';
+				buf_left[buf_left.size() - 1] = '\r';
+
+				TempFile file_left(filename_left, buf_left.data(), buf_left.size());
+				TempFile file_right(filename_right, buf_right.data(), buf_right.size());
+
+				FilePair pair(filename_left, filename_right);
+
+				EXPECT_EQ(DIFFCODE::TEXT | DIFFCODE::DIFF, bc.CompareFiles(&pair.diffData));
+			}
+
+			{// diff left: no EOL - right: CR+CR
+				std::vector<char> buf_left(WMCMPBUFF * 2 - 2 + i);
+				std::vector<char> buf_right(WMCMPBUFF * 2 + i);
+
+				memset(buf_left.data(), 'A', buf_left.size());
+				memset(buf_right.data(), 'A', buf_right.size());
+
+				buf_right[buf_right.size() - 2] = '\r';
+				buf_right[buf_right.size() - 1] = '\r';
+
+				TempFile file_left(filename_left, buf_left.data(), buf_left.size());
+				TempFile file_right(filename_right, buf_right.data(), buf_right.size());
+
+				FilePair pair(filename_left, filename_right);
+
+				EXPECT_EQ(DIFFCODE::TEXT | DIFFCODE::DIFF, bc.CompareFiles(&pair.diffData));
+			}
+		}
+	}


Src/CompareEngines/ByteComparator.cpp

@@ -353,6 +355,8 @@
 					if ((!m_eol0 || !m_eol1) && (orig0 == end0 || orig1 == end1))
 					{
 						// one side had an end-of-line, but the other didn't
+						ptr0 = ptr0b;


Src/CompareEngines/ByteComparator.cpp

@@ -353,6 +355,8 @@
 					if ((!m_eol0 || !m_eol1) && (orig0 == end0 || orig1 == end1))
 					{
 						// one side had an end-of-line, but the other didn't
+						ptr0 = ptr0b;
+						ptr1 = ptr1b;


Src/CompareEngines/ByteCompare.cpp

+					if ((eof[0] || eof[1]) &&
+						((end0 - ptr0 <= 1 && (lasteol[0] == "\r" || lasteol[0] == "\n" || lasteol[0] == "\r\n") && (end1 == ptr1))) ||
+						((end0 - ptr0 == 2 && (lasteol[0] == "\r\n") && (end1 == ptr1))) ||
+						((end1 - ptr1 <= 1 && (lasteol[1] == "\r" || lasteol[1] == "\n" || lasteol[1] == "\r\n") && (end0 == ptr0))) ||
+						((end1 - ptr1 == 2 && (lasteol[1] == "\r\n") && (end0 == ptr0))))


Testing/GoogleTest/ByteCompare/ByteCompare_test.cpp

+
+				FilePair pair(filename_left, filename_right);
+
+				EXPECT_EQ(DIFFCODE::TEXT | DIFFCODE::DIFF, bc.CompareFiles(&pair.diffData));
+			}
+
+			{// diff left: no EOL - right: CR+CR
+				std::vector<char> buf_left(WMCMPBUFF * 2 - 2 + i);
+				std::vector<char> buf_right(WMCMPBUFF * 2 - 2 + i);
+
+				memset(buf_left.data(), 'A', buf_left.size());
+				memset(buf_right.data(), 'A', buf_right.size());
+
+				buf_right[10] = '\n';
+				buf_left[10] = '\n';
+				buf_left[11] = '\n';
+
+				buf_right[19] = '\n';
+				buf_left[20] = '\n';
+				buf_left[21] = '\n';
+
+				buf_right[buf_right.size() - 2] = '\r';
+				buf_right[buf_right.size() - 1] = '\r';
+
+				TempFile file_left(filename_left, buf_left.data(), buf_left.size());
+				TempFile file_right(filename_right, buf_right.data(), buf_right.size());
+
+				FilePair pair(filename_left, filename_right);
+
+				EXPECT_EQ(DIFFCODE::TEXT | DIFFCODE::DIFF, bc.CompareFiles(&pair.diffData));
+			}
+		}
+
+		for (int i = 0; i < 4; i++)
+		{
+			{// diff left: CRLF+CRLF - right: no EOL
+				std::vector<char> buf_left(WMCMPBUFF * 2 - 4 + i);
+				std::vector<char> buf_right(WMCMPBUFF * 2 - 4 + i);
+
+				memset(buf_left.data(), 'A', buf_left.size());
+				memset(buf_right.data(), 'A', buf_right.size());
+
+				buf_left[10] = '\n';
+				buf_right[10] = '\n';
+				buf_right[11] = '\n';
+
+				buf_left[19] = '\n';
+				buf_right[20] = '\n';
+				buf_right[21] = '\n';
+
+				buf_left[28] = '\n';
+				buf_right[30] = '\n';
+				buf_right[31] = '\n';
+
+				buf_left[buf_left.size() - 4] = '\r';
+				buf_left[buf_left.size() - 3] = '\n';
+				buf_left[buf_left.size() - 2] = '\r';
+				buf_left[buf_left.size() - 1] = '\n';
+
+				TempFile file_left(filename_left, buf_left.data(), buf_left.size());
+				TempFile file_right(filename_right, buf_right.data(), buf_right.size());
+
+				FilePair pair(filename_left, filename_right);
+
+				EXPECT_EQ(DIFFCODE::TEXT | DIFFCODE::DIFF, bc.CompareFiles(&pair.diffData));
+			}
+
+			{// diff left: no EOL - right: CRLF+CRLF
+				std::vector<char> buf_left(WMCMPBUFF * 2 - 4 + i);
+				std::vector<char> buf_right(WMCMPBUFF * 2 - 4 + i);
+
+				memset(buf_left.data(), 'A', buf_left.size());
+				memset(buf_right.data(), 'A', buf_right.size());
+
+				buf_right[10] = '\n';
+				buf_left[10] = '\n';
+				buf_left[11] = '\n';
+
+				buf_right[19] = '\n';
+				buf_left[20] = '\n';
+				buf_left[21] = '\n';
+
+				buf_right[28] = '\n';
+				buf_left[30] = '\n';
+				buf_left[31] = '\n';
+
+				buf_right[buf_right.size() - 4] = '\r';
+				buf_right[buf_right.size() - 3] = '\n';
+				buf_right[buf_right.size() - 2] = '\r';
+				buf_right[buf_right.size() - 1] = '\n';
+
+				TempFile file_left(filename_left, buf_left.data(), buf_left.size());
+				TempFile file_right(filename_right, buf_right.data(), buf_right.size());
+
+				FilePair pair(filename_left, filename_right);
+
+				EXPECT_EQ(DIFFCODE::TEXT | DIFFCODE::DIFF, bc.CompareFiles(&pair.diffData));
+			}
+		}
+	}


Testing/GoogleTest/DiffWrapper/DiffWrapper_test.cpp

@@ -95,6 +95,124 @@
 	}
 }

+TEST(DiffWrapper, RunFileDiff_IgnoreMissingTrailingEol)


…2573) (3)

sdottaka added 2 commits December 11, 2024 08:58

WIP

e652b30

WIP

c1cd36b

github-advanced-security bot found potential problems Dec 13, 2024

View reviewed changes

WIP

2f011cf

github-advanced-security bot found potential problems Dec 15, 2024

View reviewed changes

sdottaka added 3 commits December 15, 2024 14:42

Merge branch 'master' into Ignore_eof_newline_presence

455e747

Merge branch 'master' into Ignore_eof_newline_presence

6e50319

WIP

76b5948

github-advanced-security bot found potential problems Dec 16, 2024

View reviewed changes

Src/CompareEngines/ByteComparator.cpp Fixed Show fixed Hide fixed

Src/CompareEngines/ByteComparator.cpp Fixed Show fixed Hide fixed

WIP

852a4b1

github-advanced-security bot found potential problems Dec 16, 2024

View reviewed changes

sdottaka added 2 commits December 17, 2024 08:07

WIP

d47f7ea

WIP

97d0d9c

github-advanced-security bot found potential problems Dec 17, 2024

View reviewed changes

sdottaka added 3 commits December 18, 2024 09:04

WIP

087f8a9

WIP

a2c5c43

WIP

1d46fa4

github-advanced-security bot found potential problems Dec 18, 2024

View reviewed changes

Testing/GoogleTest/DiffWrapper/DiffWrapper_test.cpp Fixed Show fixed Hide fixed

WIP

1155de3

github-advanced-security bot found potential problems Dec 20, 2024

View reviewed changes

Testing/GoogleTest/DiffWrapper/DiffWrapper_test.cpp

@@ -95,6 +95,124 @@

}

}

TEST(DiffWrapper, RunFileDiff_IgnoreMissingTrailingEol)

Check warning

Code scanning / CodeQL

Poorly documented large function Warning

Poorly documented function: fewer than 2% comments for a function of 117 lines.

WIP

d947129

sdottaka changed the title ~~WIP~~ Add "Ignore missing trailing EOL" option to Compare settings Dec 21, 2024

sdottaka added 8 commits December 21, 2024 14:33

WIP

f578628

WIP

a34d606

WIP

d29d133

WIP

6e95c48

WIP

ff4deec

WIP

930e0a4

WIP

d83a157

WIP

e70ef57

sdottaka added 3 commits December 22, 2024 21:24

WIP

bf91d28

WIP

947de2a

WIP

14bc5a9

sdottaka merged commit 31108d3 into master Dec 24, 2024
4 checks passed

sdottaka deleted the Ignore_eof_newline_presence branch December 24, 2024 13:38

sdottaka added this to the v2.16.45.1 milestone Dec 24, 2024

sdottaka added text compare/merge folder compare/merge labels Dec 24, 2024

This was referenced Dec 24, 2024

The last blank line is not ignored #203

Closed

Linebreak, 0d0a, as a last character is not ignored as a difference. #989

Closed

Invalid compare Result #2317

Closed

sdottaka added a commit that referenced this pull request Dec 25, 2024

Add "Ignore missing trailing EOL" option to Compare settings (#2573) (2)

0c4239b

sdottaka added a commit that referenced this pull request Dec 25, 2024

Update Add "Ignore missing trailing EOL" option to Compare settings (#…

d7bd7e2

…2573) (3)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add "Ignore missing trailing EOL" option to Compare settings #2573

Add "Ignore missing trailing EOL" option to Compare settings #2573

sdottaka commented Dec 11, 2024 •

edited

Loading

Add "Ignore missing trailing EOL" option to Compare settings #2573

Add "Ignore missing trailing EOL" option to Compare settings #2573

Conversation

sdottaka commented Dec 11, 2024 • edited Loading

sdottaka commented Dec 11, 2024 •

edited

Loading