---
title: 'Native module'
description: What is native module, how Node.js load and execute it.
---

# Native module

> Some contents are borrowed from https://xcoder.in/2017/07/01/nodejs-addon-history/

## The Nature of Native Modules

Let's start with the most essential C++ module development for Node.js. For example, we have a legitimate native module `pinyin.linux-x64-gnu.node` under Linux, which is actually a binary file that couldn't be seen properly in a text editor, until we came across the binary viewer.

![](/assets/hex.png)

The sharp-eyed reader will see that its Magic Number[^1] is `0x7F454C46` and the ASCII code it presses is ELF, so the answer is obvious: it is a **_DLL_** file for Linux.

In fact, not just on Linux. When a C++ module of Node.js is compiled under OSX, you get a DLL with the suffix `*.node` which is essentially `*.dylib`, and under Windows, you get a DLL with the suffix `*.node` which is essentially `*.dll`.

Such a module, when required in Node.js, is required via `process.dlopen()`. Let's take a look at the DLOpen[^2] function in Node.js [v10.23.0](https://github.com/nodejs/node/blob/v10.23.0/src/node.cc#L1232):

```cpp
// DLOpen is process.dlopen(module, filename, flags).
void DLOpen(const FunctionCallbackInfo<Value>& args) {
  Environment* env = Environment::GetCurrent(args);
  auto context = env->context();

  Local<Object> module;
  Local<Object> exports;
  Local<Value> exports_v;
  // initialize `module`, `module.exports` values
  if (!args[0]->ToObject(context).ToLocal(&module) ||
      // this line is equal to `exports = module.exports`
      !module->Get(context, env->exports_string()).ToLocal(&exports_v) ||
      !exports_v->ToObject(context).ToLocal(&exports)) {
    return;  // Exception pending.
  }

  node::Utf8Value filename(env->isolate(), args[1]);  // Cast
  DLib dlib(*filename, flags);
  bool is_opened = dlib.Open();

  node_module* const mp = static_cast<node_module*>(
      uv_key_get(&thread_local_modpending));
  uv_key_set(&thread_local_modpending, nullptr);

  ...

  // transfer the handle in dynamic lib to the `mp`
  mp->nm_dso_handle = dlib.handle_;
  mp->nm_link = modlist_addon;
  modlist_addon = mp;

  if (mp->nm_context_register_func != nullptr) {
    mp->nm_context_register_func(exports, module, context, mp->nm_priv);
  } else if (mp->nm_register_func != nullptr) {
    mp->nm_register_func(exports, module, mp->nm_priv);
  } else {
    dlib.Close();
    env->ThrowError("Module has no declared entry point.");
    return;
  }

}
```

Logically, the loading process actually looks like this.

- Load the link library via `uv_dlopen`.
- Hook the loaded library into the native module chain table.
- Initialize the module with `mp->nm_register_func()`, and get the module and module.exports that are there.

The flow down is similar to this flowchart:

![nm flow](/assets/nm-flow.png)

## How to build native module

### `node-waf`

Before Node.js 0.8, developers used the `node-waf` to build their library. Of course the `node-waf` is not the node-waf in npm registry, the original `node-waf` has been fallen to disrepair for years.

This thing was configured with a file named `wscript`. From Node.js 0.8, it had `node-gyp` builtin, so people didn't need wscript anymore.

But because this temporary shortage, many libraries using C++ to build Node.js addon contains both `binding.gyp` and `wscript` in that time.

You can see files back to that age in this library [node-mysql-libmysqlclient](https://github.com/Sannis/node-mysql-libmysqlclient/tree/9545ea7485fcc8b07b7c56c5ec3575938bfd4e5f). For node-gyp support it had `binding.gyp` and still preserved the `wscript` file.

### `node-gyp`

This stuff It has been with Node.js since Node.js v0.8, before that its default compilation helper package was `node-waf`(see below), which should be familiar to old Noder.

#### `GYP`

`node-gyp` is based on `GYP`[^3]. It recognizes the `binding.gyp`[^4] file in a package or project, and then generates compilable projects for each system based on that configuration file, such as **Visual Studio project files (\*.sln, etc.)** for Windows and Makefiles for Unix. `node-gyp` can also invoke system compilation tools (such as GCC) to compile the project to a final DLL \*.node file.

> As you can see from the above description, compiling C++ native modules on Windows relies on Visual Studio, which is why you will need to have Visual Studio pre-installed to install some Node.js packages. <br/>
> In fact, for users who don't need Visual Studio, it's not necessary, since node-gyp only relies on its compiler, not the IDE. Those who want to streamline the installation can visit [https://download.microsoft.com/download/5/f/7/5f7acaeb-8363-451f-9425-68a90f98b238/visualcppbuildtools_full.exe](https://download.microsoft.com/download/5/f/7/5f7acaeb-8363-451f-9425-68a90f98b238/visualcppbuildtools_full.exe) directly. cpp-build-tools Download the Visual CPP Build Tools installation, or install it from the `npm install --global --production windows-build-tools` command line to get the compilation tools you deserve.

Now that we have that out of the way, let's take a look at the basic structure of `binding.gyp`:

**binding.gyp**

```text
{
  "targets": [{
    "target_name": "addon1",
    "sources": [ "1/addon.cc", "1/myobject.cc" ]
  }, {
    "target_name": "addon2",
    "sources": [ "2/addon.cc", "2/myobject.cc" ]
  }, {
    "target_name": "addon3",
    "sources": [ "3/addon.cc", "3/myobject.cc" ]
  }, {
    "target_name": "addon4",
    "sources": [ "4/addon.cc", "4/myobject.cc" ]
  }]
}
```

This configuration tells the following story:

- Four C++ native modules are defined.
- The source code for each module is **\*.addon.cc** and **\*.myobject.cc**, respectively.
- The names of the four modules are **addon1** to **addon4**.
- The hidden story: These modules exist in **build/Release/addon\*.node** after compiling.

For more information on the GYP configuration file, you can go to the official documentation, which has a link to GYP in a footnote.

#### Something more in `node-gyp`

In addition to being GYP-based itself, node-gyp does a few extra things. First of all, when we compile a C++ native extension, it goes to the specified directory (usually `~/.node-gyp`) and searches for our current version of Node.js headers and statically linked libraries, and if they don't exist, it feverishly goes to the Node.js website to download them.

This is a directory structure for the specific version of Node.js headers and libraries downloaded from node-gyp on macOS:

```text
/Users/napi-rs/.node-gyp
└── 14.15.1
    └── include
        └── node
            ├── common.gypi
            ├── config.gypi
            ├── cppgc
            ├── js_native_api.h
            ├── js_native_api_types.h
            ├── libplatform
            ├── node.h
            ├── node_api.h
            ├── node_api_types.h
            ├── node_buffer.h
            ├── node_object_wrap.h
            ├── node_version.h
            ├── openssl
            ├── uv
            ├── uv.h
            ├── v8-fast-api-calls.h
            ├── v8-internal.h
            ├── v8-platform.h
            ├── v8-profiler.h
            ├── v8-util.h
            ├── v8-value-serializer-version.h
            ├── v8-version-string.h
            ├── v8-version.h
            ├── v8-wasm-trap-handler-posix.h
            ├── v8-wasm-trap-handler-win.h
            ├── v8.h
            └── v8config.h
```

This header directory will be merged into our `binding.gyp` in the form of an `"include_dirs"` field when `node-gyp` is compiled; in short, all the headers can be `#include` directly.

node-gyp is a command line program that can be run directly from `$ node-gyp` after installation. It has some subcommands for you to use.

- `$ node-gyp configure`: generates project files, such as Makefiles, from binding.gyp in the current directory.
- `$ node-gyp build`: build and compile the current project, which must be preceded by `configure`.
- `$ node-gyp clean`: cleans the resulting build files and output directories, in other words, cleans the directories.
- `$ node-gyp rebuild`: This is equivalent to the execution of `clean`, `configure` and `build` in sequence.
- `$ node-gyp install`: Manually download the header files and library files of the current version of Node.js to the appropriate directory.

## Conclusion

In this chapter, we introduced what the Node.js native addon is and how to compile it. In the next chapter, we will review the history of API changes related to the native addon in Node.js and formally introduce our protagonist: the N-API.

## References

[^1]: https://en.wikipedia.org/wiki/Magic_number_(programming)

[^2]: https://github.com/nodejs/node/blob/v6.9.4/src/node.cc#L2427-L2502

[^3]: GYP means _Generate Your Projects_, a build system developed by Google. For more details: https://gyp.gsrc.io.

[^4]: The config file of GYP usually has an extension of _.gyp, or _.gypi. It is a JSON-like file.