DevLog.at

Debugging mocha-parallel-tests thread issues in Node.js

I just discovered one of our tests has a race condition. It took a while to debug, but I've narrowed it down to the interaction between mocha-parallel-tests and our little stub utility:

const originals: Array<[any, any, any]> = [];

export function stub<T, K extends keyof T>(obj: T, key: K, value: T[K]): void {
  originals.push([obj, key, key in obj ? obj[key] : '#delete#']);
  obj[key] = value;
}
export function resetStubs() {
  for (let [obj, key, originalValue] of originals) {
    if (originalValue === '#delete#') {
      delete obj[key];
    }
    else {
      obj[key] = originalValue;
    }
  }
}

Example use: stub(MyModule, 'foo', () => 'myValue'). The nice part is how TypeScript ensures the return value of your function stub is the same type as the original. In other words, you can't accidentally stub an incorrectly shaped value.

mocha-parallel-tests forks your node process for each test file you run in your repo. Normally this is fine, but it seems we have two tests in separate files that are stubbing the same module and property. One overwrites the other and thereby causes the other to fail.

We definitely don't want to give up the speed benefits to parallel test running, and I want to keep the elegant nature of using the current stub function. Solving this will take a bit of thought.

2m

To avoid concurrency issues, there will need to be two stub registries:

  1. A global registry that queues access to stubbing
  2. A per-file registry so the global registry knows which stubs to clean up when a file is done.

An alternative would be to pass a key unique to the process – a filename would suffice in this case – to the two functions mentioned above. This might be more elegant, as each file already has access to __filename thanks to node.

2m

I might not understand the behavior of mocha-parallel-tests after all. Looking more into it.

2m

Ok, this is making less sense. Browsing through mocha-parallel-tests's source code, it uses node worker threads to run mocha for each test file. If that's the case, how is one file conflicting with another?

2m

Ahh wait, we're using an older version. Maybe upgrading will do the trick, and [hopefully] make my previous solution wholly unnecessary.

2m

Nope, upgrading only makes it hang indefinitely. But it looks like their docs say it only uses worker threads for node v12+ (we're on v10).

2m

It doesn't look like their v1 source is available to browse on GitHub. Time to dig into the node_modules folder.

2m

Looks like we're using 1.2.10, which is the latest 1.x version according to npm (see Versions tab). But the docs on this version tells a more concerning story:

If you're sure that running any of your test suites doesn't affect others, you should try to parallel them with mocha-parallel-tests

uh oh.

2m

Hmm, so one of our modules (the target) is only being loaded once, whereas the stub util file is being loaded multiple times.

This is definitely the cause of the issue, but now I need to figure out why it's happening.

2m

Ok, it looks like there's a point when mocha-parallel-tests decides to fork the process (I'm only guessing fork because I don't see any require('child_process') in the source). Before that point, modules are shared and only get required once. After that, modules are no longer shared.

The fix was to add a simple require() in the correct spot (with an explanatory comment) and update our stubs utility to handle concurrent access. The final gist for that is here. It may be incomplete for more complex use cases, but it's working well so far for us – no more non-deterministic test failures!

2m

Welcome to DevLog

The open thought platform for developers.

Share your work as you work on it. Easier than a blog, handier than Twitter.

Sign in and write down valuable thoughts that would otherwise be forgotten.