Investigate gh-pages deployment cache "path not file" after updating to latest version

[EltonGohJH]

Please confirm that you have searched existing issues in the repo

Yes, I have searched the existing issues

Any related issues?

#2538

Tell us about your environment

MarkBind version

5.5.2

Describe the bug and the steps to reproduce it

$ markbind deploy -n
  __  __                  _      ____    _               _
 |  \/  |   __ _   _ __  | | __ | __ )  (_)  _ __     __| |
 | |\/| |  / _` | | '__| | |/ / |  _ \  | | | '_ \   / _` |
 | |  | | | (_| | | |    |   <  | |_) | | | | | | | | (_| |
 |_|  |_|  \__,_| |_|    |_|\_\ |____/  |_| |_| |_|  \__,_|

 v5.5.2
error: The "path" argument must be of type string. Received undefined

To reproduce run markbind deploy locally. It worked for me on WSL2 but it seems like it may not be working for Windows. Note: once we set the cache_dir env variable it will work.

Expected behavior

Should work without the need to explicitly setting cache_dir

Anything else?

We updated gh-pages and it started to not work. Related to this update which introduced the cache folder. Related to the v3.1.0 update for github pages.

[damithc]

For ease of reference, I'm posting here the current remedy for Windows:

Run the following command in the terminal used for running the markbind deploy command:

set CACHE_DIR=cache

[Tim-Siu]

Can confirm this is also the issue for macOS environment.

[gerteck]

I seem to face this issue when trying to deploy on gh-pages via Travis CI:

This seems to be the case for appveyor, circleci as well.

Xubuntu, without setting Cache Directly:

Seems to be related to:

https://github.com/tschaub/gh-pages/blob/main/lib/index.js

[gerteck]

This bug seems to be introduced as mentioned previously (in related PR/issue) in gh-pages v3.1, where the find-cache-dir dependency was added into gh-pages. Subsequently, when the gh-pages was bumped for MarkBind in #2532 from v2.2.0 to v5.0.0, was probably when it sneaked in.

In the markbind deploy workflow, ghpages.publish is called:

markbind/packages/core/src/Site/index.ts

Lines 1357 to 1366 in 968a980

	/**
	* Helper function for deploy(). Returns the ghpages link where the repo will be hosted.
	*/
	async generateDepUrl(ciTokenVar: boolean | string, defaultDeployConfig: DeployOptions) {
	const publish = Bluebird.promisify(ghpages.publish);
	await this.readSiteConfig();
	const depOptions = await this.getDepOptions(ciTokenVar, defaultDeployConfig, publish);
	return Site.getDepUrl(depOptions);
	}

In the gh-pages module,

  return userPromise.then((user) =>
    getRepo(options)
      .then((repo) => {
        repoUrl = repo;
        const clone = getCacheDir(repo);
        log('Cloning %s into %s', repo, clone);
        return Git.clone(repo, clone, options.branch, options);
      })

source code

getCacheDir(repo) is being called. Subsequently,

function getCacheDir(optPath) {
  const dir = findCacheDir({name: 'gh-pages'});
  if (!optPath) {
    return dir;
  }


  return path.join(dir, filenamify(optPath));
}
exports.getCacheDir = getCacheDir;

source code
findCacheDir is then called. The issue seems to stem from dir being returned as undefined.

module.exports = (options = {}) => {
	if (env.CACHE_DIR && !['true', 'false', '1', '0'].includes(env.CACHE_DIR)) {
		return useDirectory(path.join(env.CACHE_DIR, options.name), options);
	}

	let {cwd: directory = cwd()} = options;

	if (options.files) {
		directory = commonDir(directory, options.files);
	}

        // searches for `package.json` in current and parent directories
	directory = pkgDir.sync(directory);

	if (!directory) {
                // returns undefined as `package.json` file not found
		return;
	}

	const nodeModules = getNodeModuleDirectory(directory);
	if (!nodeModules) {
		return undefined;
	}

	return useDirectory(path.join(directory, 'node_modules', '.cache', options.name), options);
};

source code (v3.3.2)

This is where setting process.env.CACHE_DIR directly seems to fix the problem. This is because this algorithm checks for the CACHE_DIR environmental variable, and uses it directly if it has been set.
Otherwise, where CACHE_DIR is undefined, it seems that the function also returns undefined, hence causing the error.

As mentioned in findCacheDirectory package

findCacheDirectory(options?)
Finds the cache directory using the given options.

The algorithm checks for the CACHE_DIR environmental variable and uses it if it is not set to true, false, 1 or 0. If one is not found, it tries to find a package.json file, searching every parent directory of the cwd specified (or implied from other options). It returns a string containing the absolute path to the cache directory, or undefined if package.json was never found or if the node_modules directory is unwritable.

My guess as to why this returns undefined is that as each markbind project (markbind init) does not have a package.json file in the project directory, the function returns undefined as it does not recognise it as the working directory, breaking the workflow. ** Also note that it recursively searches for package.json files in parent directories as well, I had a rogue package.json file in my home dir that suddenly caused the deploy to work causing some confusion. One can test this by:

• Setting CACHE_DIR = '', or "1", or "0" etc.
  • $env:CACHE_DIR = "" on powershell / set CACHE_DIR=cache / echo %CACHE_DIR%
• Verify that deploy does not work
• creating a 'package.json` file in the repo / working directory
• Verify that deploy suddenly works

I think that setting CACHE_DIR directly in the deployment workflow directly could fix the problem all things considered, but I would like to seek others' opinons on this. Wondering if there is a less hacky solution 🫤 .

Related:

• CS2103 website build fails due to missing bootstrap-icons.css #2538
• Specify CACHE_DIR when deploying with GH actions #2542

Related gh-pages issue