Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(NODE-6616): short circuit EJSON stringifying #4360

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

refactor(NODE-6616): short circuit EJSON stringifying #4360

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 33 additions & 1 deletion src/mongo_logger.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { inspect, promisify } from 'util';
import { isUint8Array } from 'util/types';

import { type Document, EJSON, type EJSONOptions, type ObjectId } from './bson';
import type { CommandStartedEvent } from './cmap/command_monitoring_events';
Expand Down Expand Up @@ -421,13 +422,44 @@ export function stringifyWithMaxLen(
): string {
let strToTruncate = '';

let currentLength = 0;
const maxDocumentLengthEnsurer = function maxDocumentLengthEnsurer(key: string, value: any) {
if (currentLength >= maxDocumentLength) {
return undefined;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will essentially redact the keys entirely, right? I guess you've considered the option of using a placeholder and decided against it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't consider a placeholder, that may work just as well for performance too since it would be a hardcoded value (maybe an empty string?). True, it will redact keys but the thinking was that those would be lost anyway by the slicing that follows.

}

currentLength += key.length + 4;

if (typeof value === 'string') {
currentLength += value.length;
} else if (typeof value === 'number' || typeof value === 'bigint') {
currentLength += 20;
Copy link
Contributor

@baileympearson baileympearson Jan 3, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where is this lengths coming from? wouldn't the length of numbers be dependent on the actual number being stringified (ex: 0 would be 1, but 1.1234 would be 6?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes correct they would depend on the value. I can improve this to calculate the number of digits from the value.

Generally, the lengths here are approximations when we can't determine the length without actually stringifying the value in an attempt to avoid paying that potentially expensive cost. With a better implementation, we should always get a bit over maxLen returned to us and then slicing ensures we actually hit the target.

} else if (typeof value === 'boolean') {
currentLength += value ? 4 : 5;
} else if (value != null && typeof value === 'object' && '_bsontype' in value) {
if (isUint8Array(value.buffer)) {
currentLength += (value.buffer.byteLength + value.buffer.byteLength * 0.5) | 0;
} else if (value._bsontype === 'Binary') {
currentLength += (value.position + value.position * 0.3) | 0;
} else if (value._bsontype === 'Code') {
currentLength += value.code.length;
}
}

return value;
};

if (typeof value === 'string') {
strToTruncate = value;
} else if (typeof value === 'function') {
strToTruncate = value.name;
} else {
try {
strToTruncate = EJSON.stringify(value, options);
if (maxDocumentLength !== 0) {
strToTruncate = EJSON.stringify(value, maxDocumentLengthEnsurer, 0, options);
} else {
strToTruncate = EJSON.stringify(value, options);
}
} catch (e) {
strToTruncate = `Extended JSON serialization failed with: ${e.message}`;
}
Expand Down
Loading