Skip to content

Commit

Permalink
Introduce LockManager for Async Handling and Optimize PUML Diagrams (#…
Browse files Browse the repository at this point in the history
…2345)

Introduce LockManager for Async Handling and Optimize PUML Diagrams

Introduce the LockManager module to manage the lifecycle of asynchronous
functions, ensuring robust and consistent handling within synchronous
methods.

Utilize the createLock, deleteLock, and waitForLockRelease methods for
starting, stopping, and waiting for async functions respectively.

Code snippet:

```javascript
const LockManager = require('path_to/LockManager');
const lockId = LockManager.createLock();
LockManager.deleteLock(lockId);
await LockManager.waitForLockRelease();
```

Fix the issue where all distinct PUML diagrams were being reprocessed on
every hot reload by introducing a hashmap tracker.

- The hash key comes from the content and the file name of the PUML diagram.

By implementing these changes, the code is made more maintainable,
asynchronous functions are properly managed, and PUML diagram processing
is optimized.
  • Loading branch information
SPWwj authored Aug 27, 2023
1 parent cd5ba3d commit da9d3f7
Show file tree
Hide file tree
Showing 17 changed files with 184 additions and 42 deletions.
10 changes: 1 addition & 9 deletions docs/devGuide/development/workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -375,15 +375,7 @@ To update PlantUML to a newer version:

1. Download the JAR file from [PlantUML's website](https://plantuml.com/download).
1. Rename the file to `plantuml.jar` (if required), and replace the existing JAR file located in `packages/core/src/plugins/default`.
1. Generate the image files for the `.puml` files listed in `docs/userGuide/diagrams`.

<box type="tip" seamless header="Here are the recommended steps to generate the image files:">

1. Add a new `.md` file in `userGuide`, e.g. `plantuml.md`, containing `<puml>` tags of all diagrams to be generated.
1. Serve the documentation site using `markbind serve -d`.
1. Access the corresponding HTML page with the generated diagrams, i.e. `/userGuide/plantuml.html`.
1. Right-click on each image and save the image in `docs/userGuide/diagrams`.
</box>
1. Check the HTML pages that contain PlantUML diagrams, i.e. `/userGuide/components/imagesAndDiagrams.html`.

### Updating Bootstrap and Bootswatch

Expand Down
Binary file removed docs/userGuide/diagrams/activity.png
Binary file not shown.
Binary file removed docs/userGuide/diagrams/archimate.png
Binary file not shown.
Binary file removed docs/userGuide/diagrams/class.png
Binary file not shown.
Binary file removed docs/userGuide/diagrams/component.png
Binary file not shown.
Binary file removed docs/userGuide/diagrams/ditaa.png
Binary file not shown.
Binary file removed docs/userGuide/diagrams/entityrelation.png
Binary file not shown.
Binary file removed docs/userGuide/diagrams/gantt.png
Binary file not shown.
Binary file removed docs/userGuide/diagrams/object.png
Binary file not shown.
Binary file removed docs/userGuide/diagrams/sequence.png
Binary file not shown.
Binary file removed docs/userGuide/diagrams/state.png
Binary file not shown.
Binary file removed docs/userGuide/diagrams/usecase.png
Binary file not shown.
44 changes: 19 additions & 25 deletions docs/userGuide/syntax/diagrams.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,9 @@ See [Deploying via Github Actions](../deployingTheSite.html#deploying-via-github
</box>
<div id="main-example">
<include src="outputBox.md" boilerplate>
<include src="codeAndOutput.md" boilerplate>
<variable name="code">
```
<puml width="300">
@startuml
alice -> bob ++ : hello
Expand All @@ -52,11 +51,6 @@ bob -> george !! : delete
return success
@enduml
</puml>
```
</variable>

<variable name="output">
<pic src="../diagrams/sequence.png" width="300" />
</variable>
</include>
Expand Down Expand Up @@ -88,7 +82,7 @@ in another file:
</variable>

<variable id="output">
<pic src="../diagrams/sequence.png" width="300" />
<puml src="../diagrams/sequence.puml" width=300 />
</variable>

</include>
Expand All @@ -103,50 +97,50 @@ The full PlantUML syntax reference can be found at plantuml.com/guide
<div id="puml-examples">

**Sequence Diagram**:<br>
<pic src="../diagrams/sequence.png" />
<puml src="../diagrams/sequence.puml" />

**Use Case Diagram**:<br>
<pic src="../diagrams/usecase.png" />
<puml src="../diagrams/usecase.puml" />

**Class Diagram**:<br>
<pic src="../diagrams/class.png" />
<puml src="../diagrams/class.puml" />

**Activity Diagram**:<br>
<pic src="../diagrams/activity.png" />
<puml src="../diagrams/activity.puml" />

**Component Diagram**:<br>
<pic src="../diagrams/component.png" />
<puml src="../diagrams/component.puml" />

**State Diagram**:<br>
<pic src="../diagrams/state.png" />
<puml src="../diagrams/state.puml" />

**Object Diagram**:<br>
<pic src="../diagrams/object.png" />
<puml src="../diagrams/object.puml" />

**Gantt Diagram**:<br>
<pic src="../diagrams/gantt.png" />
<puml src="../diagrams/gantt.puml" />

**Entity Relation Diagram**:<br>
<pic src="../diagrams/entityrelation.png" />
<puml src="../diagrams/entityrelation.puml" />

**Ditaa Diagram**:<br>
<pic src="../diagrams/ditaa.png" />
<puml src="../diagrams/ditaa.puml" />

**Archimate Diagram**:<br>
<pic src="../diagrams/archimate.png" />
<puml src="../diagrams/archimate.puml" />

</div>
</panel>
<p/>

****Options****
Name | Type | Description
--- | --- | ---
alt | `string` | The alternative text of the diagram.
Name | Type | Description
-----|----------|-------------------------------------
alt | `string` | The alternative text of the diagram.
height | `string` | The height of the diagram in pixels.
name | `string` | The name of the output file.
src | `string` | The URL of the diagram if your diagram is in another `.puml` file.<br>The URL can be specified as absolute or relative references. More info in: _[Intra-Site Links]({{baseUrl}}/userGuide/formattingContents.html#intraSiteLinks)_
width | `string` | The width of the diagram in pixels.<br>If both width and height are specified, width takes priority over height. It is to maintain the diagram's aspect ratio.
name | `string` | The name of the output file.<br>Avoid using the same name for different diagrams to prevent overwriting.
src | `string` | The URL of the diagram if your diagram is in another `.puml` file.<br>The URL can be specified as absolute or relative references. More info in: _[Intra-Site Links]({{baseUrl}}/userGuide/formattingContents.html#intraSiteLinks)_
width | `string` | The width of the diagram in pixels.<br>If both width and height are specified, width takes priority over height. It is to maintain the diagram's aspect ratio.

<div id="short" class="d-none">

Expand Down
5 changes: 5 additions & 0 deletions packages/core/src/Page/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ require('../patches/htmlparser2');

const _ = { cloneDeep, isObject, isArray };

const LockManager = require('../utils/LockManager');

const PACKAGE_VERSION = require('../../package.json').version;

const {
Expand Down Expand Up @@ -534,6 +536,9 @@ export class Page {
// Each source path will only contain 1 copy of build/re-build page (the latest one)
pageVueServerRenderer.pageEntries[this.pageConfig.sourcePath] = builtPage;

// Wait for all pages resources to be generated before writing to disk
await LockManager.waitForLockRelease();

/*
* Server-side render Vue page app into actual html.
*
Expand Down
39 changes: 31 additions & 8 deletions packages/core/src/plugins/default/markbind-plugin-plantuml.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,24 @@ import * as urlUtil from '../../utils/urlUtil';
import { PluginContext } from '../Plugin';
import { NodeProcessorConfig } from '../../html/NodeProcessor';
import { MbNode } from '../../utils/node';
import LockManager from '../../utils/LockManager';

interface DiagramStatus {
hashKey: string;
}

const JAR_PATH = path.resolve(__dirname, 'plantuml.jar');

const processedDiagrams = new Set();
const PUML_EXT = '.png';

/**
* This Map maintains a record of processed diagrams. When a diagram is generated or regenerated,
* it's added to this map. Subsequently, if a PUML or non-PUML file is edited, leading to a hot reload,
* the generateDiagram function can avoid redundant regeneration by checking this map.
* If the diagram's identifier is present in the map,
* the generation process is bypassed, thus preventing duplicates.
*/
const processedDiagrams = new Map<string, DiagramStatus>();

let graphvizCheckCompleted = false;

Expand All @@ -28,15 +42,22 @@ let graphvizCheckCompleted = false;
* @param content puml dsl used to generate the puml diagram
*/
function generateDiagram(imageOutputPath: string, content: string) {
const hashKey = cryptoJS.MD5(imageOutputPath + content).toString();

// Avoid generating twice
if (processedDiagrams.has(imageOutputPath)) { return; }
processedDiagrams.add(imageOutputPath);
if (processedDiagrams.has(imageOutputPath) && processedDiagrams.get(imageOutputPath)?.hashKey === hashKey) {
return;
}

// Creates output dir if it doesn't exist
const outputDir = path.dirname(imageOutputPath);
if (!fs.existsSync(outputDir)) {
fs.mkdirSync(outputDir, { recursive: true });
}
const lockId = LockManager.createLock();

// Add new diagram to the map
processedDiagrams.set(imageOutputPath, { hashKey });

// Java command to launch PlantUML jar
const cmd = `java -jar "${JAR_PATH}" -nometadata -pipe > "${imageOutputPath}"`;
Expand All @@ -59,6 +80,7 @@ function generateDiagram(imageOutputPath: string, content: string) {
childProcess.on('error', (error) => {
logger.debug(error as unknown as string);
logger.error(`Error generating ${imageOutputPath}`);
LockManager.deleteLock(lockId);
});

childProcess.stderr?.on('data', (errorMsg) => {
Expand All @@ -68,6 +90,7 @@ function generateDiagram(imageOutputPath: string, content: string) {
childProcess.on('exit', () => {
// This goes to the log file, but not shown on the console
logger.debug(errorLog);
LockManager.deleteLock(lockId);
});
}

Expand All @@ -90,7 +113,6 @@ export = {
},

beforeSiteGenerate: () => {
processedDiagrams.clear();
graphvizCheckCompleted = false;
},

Expand All @@ -111,6 +133,7 @@ export = {

let pumlContent;
let pathFromRootToImage;

if (node.attribs.src) {
const srcWithoutBaseUrl = urlUtil.stripBaseUrl(node.attribs.src, config.baseUrl);
const srcWithoutLeadingSlash = srcWithoutBaseUrl.startsWith('/')
Expand All @@ -126,8 +149,8 @@ export = {
return;
}

pathFromRootToImage = fsUtil.setExtension(srcWithoutLeadingSlash, '.png');
node.attribs.src = fsUtil.ensurePosix(fsUtil.setExtension(node.attribs.src, '.png'));
pathFromRootToImage = fsUtil.setExtension(srcWithoutLeadingSlash, PUML_EXT);
node.attribs.src = fsUtil.ensurePosix(fsUtil.setExtension(node.attribs.src, PUML_EXT));
} else {
pumlContent = cheerio(node).text();

Expand All @@ -136,13 +159,13 @@ export = {
const nameWithoutLeadingSlash = nameWithoutBaseUrl.startsWith('/')
? nameWithoutBaseUrl.substring(1)
: nameWithoutBaseUrl;
pathFromRootToImage = fsUtil.ensurePosix(fsUtil.setExtension(nameWithoutLeadingSlash, '.png'));
pathFromRootToImage = fsUtil.ensurePosix(fsUtil.setExtension(nameWithoutLeadingSlash, PUML_EXT));

delete node.attribs.name;
} else {
const normalizedContent = pumlContent.replace(/\r\n/g, '\n');
const hashedContent = cryptoJS.MD5(normalizedContent).toString();
pathFromRootToImage = `${hashedContent}.png`;
pathFromRootToImage = `${hashedContent}${PUML_EXT}`;
}

node.attribs.src = `${config.baseUrl}/${pathFromRootToImage}`;
Expand Down
86 changes: 86 additions & 0 deletions packages/core/src/utils/LockManager.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
import { v4 as uuidv4 } from 'uuid';

/**
* The `LockManager` is a singleton class designed to help wait for required async
* promised operations to complete
* before the page is generated. It provides functionalities to create, delete, and wait
* for the release of locks.
* The locks are stored in a Map with a unique ID (either provided or auto-generated) as
* the key.
* The class provides an instance property to get the singleton instance of `LockManager`.
*/

class LockManager {
// Holds the single instance of LockManager.
private static _instance: LockManager;

// A Map to keep track of the active locks.
private locks: Map<string, boolean>;

/**
* Private constructor to prevent direct instantiation from outside.
* Initializes the locks Map.
*/
private constructor() {
this.locks = new Map();
}

/**
* Provides a way to access the single instance of the LockManager.
* If it doesn't exist, it creates one.
* @returns {LockManager} The single instance of LockManager.
*/
public static get instance() {
if (!LockManager._instance) {
LockManager._instance = new LockManager();
}

return LockManager._instance;
}

/**
* Creates a new lock.
* @param {string} [id] - An optional ID to use for the lock. If not provided, a UUID will be generated.
* @returns {string} The ID of the created lock.
*/
createLock(id?: string): string {
const lockId = id ?? uuidv4();
this.locks.set(lockId, true);
return lockId;
}

/**
* Deletes a lock by its ID.
* @param {string} lockId - The ID of the lock to be deleted.
*/
deleteLock(lockId: string): void {
this.locks.delete(lockId);
}

/**
* Deletes all locks, clearing the locks Map.
*/
deleteAllLocks(): void {
this.locks.clear();
}

/**
* Waits until all locks are released and then resolves.
* @returns {Promise<void>} A promise that resolves when all locks are released.
*/
waitForLockRelease(): Promise<void> {
return new Promise((resolve) => {
const checkLocks = () => {
if (this.locks.size === 0) {
resolve();
} else {
setTimeout(checkLocks, 100);
}
};
checkLocks();
});
}
}

// Export the singleton instance of LockManager.
export = LockManager.instance;
42 changes: 42 additions & 0 deletions packages/core/test/unit/utils/LockManager.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
import LockManager from '../../../src/utils/LockManager';

describe('LockManager', () => {
let lockManager: typeof LockManager;

beforeEach(() => {
lockManager = LockManager;
});

afterEach(() => {
lockManager.deleteAllLocks();
});

it('should create a new lock', () => {
const lockId = lockManager.createLock();
expect(lockId).toBeDefined();
});

it('should use the provided ID when creating a lock', () => {
const lockId = 'customId';
const createdLockId = lockManager.createLock(lockId);
expect(createdLockId).toEqual(lockId);
});

it('should delete all locks', () => {
lockManager.createLock();
lockManager.createLock();
lockManager.deleteAllLocks();
});

it('should wait until all locks are released and resolve', async () => {
const lockId1 = lockManager.createLock();
const lockId2 = lockManager.createLock();

const waitForLockReleasePromise = lockManager.waitForLockRelease();

setTimeout(() => lockManager.deleteLock(lockId1), 100);
setTimeout(() => lockManager.deleteLock(lockId2), 200);

await expect(waitForLockReleasePromise).resolves.toBeUndefined();
});
});

0 comments on commit da9d3f7

Please sign in to comment.