NodeJS Video Tutorial: Understanding the node_modules Folder

The folder node_modules is a somewhat mysterious file which is the source of quite a few problems. Today I suggest you dive into the heart of the matter and analyze its role in the resolution of dependencies within the framework of NodeJS.

How NodeJS solves

So let’s take a simple import example

import {hello} from 'hello'

hello()

In the case of an import that does not constitute a relative path (not prefixed by ./, ../ Where /) NodeJS has its own resolution system which can be summarized as follows:

• He is looking for a file node_modules/NOM_DU_MODULE adjacent to current file
• If the folder is not found it does the same search in the parent folder (and this recursively)

If the file requesting the module is itself in a folder node_modules in this case it looks directly in this folder (rather than looking for a folder node_modules adjacent).

If a folder with the name of the module is found it will then read the file package.json of this folder to know the file to import thanks to the keys main Where exports.

{
  "main": "src/index.js"
}

This allows it to resolve the file which will therefore be imported into our original file and the import will be translated like this:

import {hello} from './node_modules/hello/src/index.js'

hello()

The case of symbolic links

Symbolic links add a small layer of complexity in dependency resolution because NodeJS uses the actual file path in its resolution. Take this example:

/Projet1
  /node_modules
    /a -> /Projet2/node_modules/a
    /b
  /src
    index.js
  package.json
/Projet2
  /node_modules
    /a
    /b

The module a module dependent b and we have inside its file a import b from 'b'

If the file index.js try to load the module athe symlink will be followed and when searching for the module b NodeJS will crawl the folder Projet2 and its parent folders. This behavior can be changed using a flag --preserve-symlinks (flag that we will use infrequently because it creates other problems).

And npm in all this?

Now that we have understood how package resolution works, we will talk about the npm dependency manager and how it installs dependencies. If we install a dependency vite for example we end up with the following structure.

{appDir}
 ├── src
 │   ├── index.js
 └── node_modules
      |── vite
      |── esbuild
      |── postcss
      |── resolve
      |── rollup
      └── fsevents

Note that it places all dependencies at the root of the folder node_modules to leverage the parent folder system in dependency resolution. This hoisting allows several modules that share the same dependency to use the same folder (rather than installing the library several times). But what happens if we install a package a who needs a version of esbuild different for example?

{appDir}
 ├── src
 │   ├── index.js
 └── node_modules
      |── a
      |   |── package.json
      |   |── index.js
      |   └── node_modules
      |       └── esbuild
      |── vite
      |── esbuild
      |── postcss
      |── resolve
      |── rollup
      └── fsevents

In this case npm has no choice but to install esbuild twice. A version will be left in the subfolder node_modules of a so that imports made from this folder resolve this version, while other files outside the module a we will continue to solve the case esbuild at the root.

Be careful, however, if you use npm link to manage cross-dependencies because symbolic link resolution applies. Also, in the case of a monorepo, it will be necessary to be vigilant that the dependencies common to several modules are found in a folder at the root at the risk of duplicating the dependency several times.