在 Docker 容器中安装 node_module 并将它们与主机同步

我在 Docker 容器中安装 node_modules并与主机同步时遇到了问题。我的多克的版本是 18.03.1-ce, build 9ee9f40和多克作曲的版本是 1.21.2, build a133471

我的 docker-compose.yml看起来像:

# Frontend Container.
frontend:
build: ./app/frontend
volumes:
- ./app/frontend:/usr/src/app
- frontend-node-modules:/usr/src/app/node_modules
ports:
- 3000:3000
environment:
NODE_ENV: ${ENV}
command: npm start


# Define all the external volumes.
volumes:
frontend-node-modules: ~

我的 Dockerfile:

# Set the base image.
FROM node:10


# Create and define the working directory.
RUN mkdir /usr/src/app
WORKDIR /usr/src/app


# Install the application's dependencies.
COPY package.json ./
COPY package-lock.json ./
RUN npm install

很多博客文章和 Stack Overflow 的回答都描述了外部卷的技巧。例如,这个

应用程序工作良好。源代码是同步的。热重载工作也很好。

我唯一的问题是主机上的 node_modules文件夹是空的。是否可以将 Docker 容器中的 node_modules文件夹与主机同步?

我已经读过这些答案了:

  1. Docker-在 node _ module 上组合卷,但是它是空的
  2. 在 Docker 中安装 npm 之后访问 node _ module

不幸的是,他们帮不了我太多。我不喜欢 第一个,因为我不想在我的主机上运行 npm install,因为它可能存在跨平台问题(例如,主机是 Windows 或 Mac,而 Docker 容器是 Debian 8或 Ubuntu 16.04)。第二个对我来说也不好,因为我想在我的 Dockerfile中运行 npm install,而不是在 Docker 容器启动后运行它。

还有,我找到了 这篇博文。作者试图解决我所面临的同样的问题。问题是 node_modules无法同步,因为我们只是将它们从 Docker 容器复制到主机。

我希望 Docker 容器中的 node_modules与主机同步。拜托,考虑到我想要的:

  • 自动安装 node_modules而不是手动安装
  • node_modules安装在 Docker 容器内而不是主机内
  • 使 node_modules与主机同步(如果我在 Docker 容器中安装了一些新的包,它应该自动与主机同步,而不需要任何手动操作)

我需要主机上的 node_modules,因为:

  • 当我需要的时候可以读取源代码
  • IDE 需要在本地安装 node_modules,以便能够访问 devDependencies,如 eslintprettier。我不想在全球范围内安装这些 devDependencies

先谢谢你。

83926 次浏览

There's three things going on here:

  1. When you run docker build or docker-compose build, your Dockerfile builds a new image containing a /usr/src/app/node_modules directory and a Node installation, but nothing else. In particular, your application isn't in the built image.
  2. When you docker-compose up, the volumes: ['./app/frontend:/usr/src/app'] directive hides whatever was in /usr/src/app and mounts host system content on top of it.
  3. Then the volumes: ['frontend-node-modules:/usr/src/app/node_modules'] directive mounts the named volume on top of the node_modules tree, hiding the corresponding host system directory.

If you were to launch another container and attach the named volume to it, I expect you'd see the node_modules tree there. For what you're describing you just don't want the named volume: delete the second line from the volumes: block and the volumes: section at the end of the docker-compose.yml file.

I wouldn't suggest overlapping volumes, although I haven't seen any official docs ban it, I've had some issues with it in the past. How I do it is:

  1. Get rid of the external volume as you are not planning on actually using it how it's meant to be used - respawning the container with its data created specifically in the container after stopping+removing it.

The above might be achieved by shortening your compose file a bit:

frontend:
build: ./app/frontend
volumes:
- ./app/frontend:/usr/src/app
ports:
- 3000:3000
environment:
NODE_ENV: ${ENV}
command: npm start
  1. Avoid overlapping volume data with Dockerfile instructions when not necessary.

That means you might need two Dockerfiles - one for local development and one for deploying a fat image with all the application dist files layered inside.

That said, consider a development Dockerfile:

FROM node:10
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
RUN npm install

The above makes the application create a full node_modules installation and map it to your host location, while the docker-compose specified command would start your application off.

At first, I would like to thank David Maze and trust512 for posting their answers. Unfortunately, they didn't help me to solve my problem.

I would like to post my answer to this question.

My docker-compose.yml:

---
# Define Docker Compose version.
version: "3"


# Define all the containers.
services:
# Frontend Container.
frontend:
build: ./app/frontend
volumes:
- ./app/frontend:/usr/src/app
ports:
- 3000:3000
environment:
NODE_ENV: development
command: /usr/src/app/entrypoint.sh

My Dockerfile:

# Set the base image.
FROM node:10


# Create and define the node_modules's cache directory.
RUN mkdir /usr/src/cache
WORKDIR /usr/src/cache


# Install the application's dependencies into the node_modules's cache directory.
COPY package.json ./
COPY package-lock.json ./
RUN npm install


# Create and define the application's working directory.
RUN mkdir /usr/src/app
WORKDIR /usr/src/app

And last but not least entrypoint.sh:

#!/bin/bash


cp -r /usr/src/cache/node_modules/. /usr/src/app/node_modules/
exec npm start

The trickiest part here is to install the node_modules into the node_module's cache directory (/usr/src/cache) which is defined in our Dockerfile. After that, entrypoint.sh will move the node_modules from the cache directory (/usr/src/cache) to our application directory (/usr/src/app). Thanks to this the entire node_modules directory will appear on our host machine.

Looking at my question above I wanted:

  • to install node_modules automatically instead of manually
  • to install node_modules inside the Docker container instead of the host
  • to have node_modules synchronized with the host (if I install some new package inside the Docker container, it should be synchronized with the host automatically without any manual actions

The first thing is done: node_modules are installed automatically. The second thing is done too: node_modules are installed inside the Docker container (so, there will be no cross-platform issues). And the third thing is done too: node_modules that were installed inside the Docker container will be visible on our host machine and they will be synchronized! If we install some new package inside the Docker container, it will be synchronized with our host machine at once.

The important thing to note: truly speaking, the new package installed inside the Docker container, will appear in /usr/src/app/node_modules. As this directory is synchronized with our host machine, this new package will appear on our host machine's node_modules directory too. But the /usr/src/cache/node_modules will have the old build at this point (without this new package). Anyway, it is not a problem for us. During next docker-compose up --build (--build is required) the Docker will re-install the node_modules (because package.json was changed) and the entrypoint.sh file will move them to our /usr/src/app/node_modules.

You should take into account one more important thing. If you git pull the code from the remote repository or git checkout your-teammate-branch when Docker is running, there may be some new packages added to the package.json file. In this case, you should stop the Docker with CTRL + C and up it again with docker-compose up --build (--build is required). If your containers are running as a daemon, you should just execute docker-compose stop to stop the containers and up it again with docker-compose up --build (--build is required).

If you have any questions, please let me know in the comments.

Hope this helps.

Thanks Vladyslav Turak for answer with entrypoint.sh where we copy node_modules from container to host.

I implemented the similar thing but I run into the issue with husky, @commitlint, tslint npm packages.
I can't push anything into repository.
Reason: I copied node_modules from Linux to Windows. In my case <5% of files are different (.bin and most of package.json) and 95% are the same. example: image with diff

So I returned to solution with npm install of node_modules for Windows first (for IDE and debugging). And Docker image will contain Linux version of node_modules.

I know that this was resolved, but what about:

Dockerfile:

FROM node


# Create app directory
WORKDIR /usr/src/app


# Your other staffs


EXPOSE 3000

docker-composer.yml:

version: '3.2'
services:
api:
build: ./path/to/folder/with/a/dockerfile
volumes:
- "./volumes/app:/usr/src/app"
command: "npm start"

volumes/app/package.json

{
... ,
"scripts": {
"start": "npm install && node server.js"
},
"dependencies": {
....
}
}

After run, node_modules will be present in your volumes, but its contents are generated within the container so no cross platform problems.

No one has mentioned solution with actually using docker's entrypoint feature.

Here is my working solution:

Dockerfile (multistage build, so it is both production and local dev ready):

FROM node:10.15.3 as production
WORKDIR /app


COPY package*.json ./
RUN npm install && npm install --only=dev


COPY . .


RUN npm run build


EXPOSE 3000


CMD ["npm", "start"]




FROM production as dev


COPY docker/dev-entrypoint.sh /usr/local/bin/


ENTRYPOINT ["dev-entrypoint.sh"]
CMD ["npm", "run", "watch"]

docker/dev-entrypoint.sh:

#!/bin/sh
set -e


npm install && npm install --only=dev ## Note this line, rest is copy+paste from original entrypoint


if [ "${1#-}" != "${1}" ] || [ -z "$(command -v "${1}")" ]; then
set -- node "$@"
fi


exec "$@"

docker-compose.yml:

version: "3.7"


services:
web:
build:
target: dev
context: .
volumes:
- .:/app:delegated
ports:
- "3000:3000"
restart: always
environment:
NODE_ENV: dev

With this approach you achieve all 3 points you required and imho it is much cleaner way - not need to move files around.

Binding your host node_modules folder with your container node_modules is not a good practice as you mention. I have seen the solution of creating an internal volume for this folder quite often. Not doing so will cause problems during the building stage.

I ran into this problem when I was trying to build a docker development environment for an angular app, that shows tslib errors when I was editing the files within my host folder cause my host's node_modules folder was empty (as expected).

The cheap solution that helps me, in this case, was to use the Visual Studio Code Extension called "Remote-Containers".

This extension will allow you to attach your Visual Studio Code to your container and edit transparently your files within your container folders. To do so, it will install an internal vscode server within your development container. For more information check this link.

Ensure, however, that your volumes are still created in your docker-compose.yml file.

I hope it helps :D!

Having run into this issue and finding the accepted answer pretty slow to copy all node_modules to the host in every container run, I managed to solve it by installing the dependencies in the container, mirror the host volume, and skip installing again if a node_modules folder is present:

Dockerfile:

FROM node:12-alpine


WORKDIR /usr/src/app


CMD [ -d "node_modules" ] && npm run start || npm ci && npm run start

docker-compose.yml:

version: '3.8'


services:
service-1:
build: ./
volumes:
- ./:/usr/src/app

When you need to reinstall the dependencies just delete node_modules.

I'm not sure to understand why you want your source code to live inside the container and host and bind mount each others during development. Usually, you want your source code to live inside the container for deployments, not development since the code is available on your host and bind mounted.

Your docker-compose.yml

frontend:
volumes:
- ./app/frontend:/usr/src/app

Your Dockerfile

FROM node:10


RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

Of course you must run npm install first time and everytime package.json changes, but you run it inside the container so there is no cross-platform issue: docker-compose exec frontend npm install

Finally start your server docker-compose exec frontend npm start

And then later, usually in a CI pipeline targetting a deployment, you build your final image with the whole source code copied and node_modules reinstalled, but of course at this point you don't need anymore the bind mount and "synchronization", so your setup could look like :

docker-compose.yml

frontend:
build:
context: ./app/frontend
target: dev
volumes:
- ./app/frontend:/usr/src/app

Dockerfile

FROM node:10 as dev


RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app


FROM dev as build


COPY package.json package-lock.json ./
RUN npm install


COPY . ./


CMD ["npm", "start"]

And you target the build stage of your Dockerfile later, either manually or during a pipeline, to build your deployment-ready image.

I know it's not the exact answer to your questions since you have to run npm install and nothing lives inside the container during development, but it solves your node_modules issue, and I feel like your questions are mixing development and deployment considerations, so maybe you thought about this problem in the wrong way.

My workaround is to install dependencies when the container is starting instead of during build-time.

Dockerfile:

# We're using a multi-stage build so that we can install dependencies during build-time only for production.


# dev-stage
FROM node:14-alpine AS dev-stage
WORKDIR /usr/src/app
COPY package.json ./
COPY . .
# `yarn install` will run every time we start the container. We're using yarn because it's much faster than npm when there's nothing new to install
CMD ["sh", "-c", "yarn install && yarn run start"]


# production-stage
FROM node:14-alpine AS production-stage
WORKDIR /usr/src/app
COPY package.json ./
RUN yarn install
COPY . .

.dockerignore

Add node_modules to .dockerignore to prevent it from being copied when the Dockerfile runs COPY . .. We use volumes to bring in node_modules.

**/node_modules

docker-compose.yml

node_app:
container_name: node_app
build:
context: ./node_app
target: dev-stage # `production-stage` for production
volumes:
# For development:
#   If node_modules already exists on the host, they will be copied
#   into the container here. Since `yarn install` runs after the
#   container starts, this volume won't override the node_modules.
- ./node_app:/usr/src/app
# For production:
#
- ./node_app:/usr/src/app
- /usr/src/app/node_modules

A Simple, Complete Solution

You can install node_modules in the container using the external named volume trick and synchronize it with the host by configuring the volume's storage location to point to your host's node_modules directory. This can be done with a named volume using the local driver and a bind mount, as seen in the example below.

The volume's data is stored on your host anyway, in something like /var/lib/docker/volumes/, so we're just storing it inside your project instead.

To do this in Docker Compose, just add your node_modules volume to your front-end service, and then configure the volume in the named volumes section, where "device" is the relative path (from the location of docker-compose.yml) to your local (host) node_modules directory.

docker-compose.yml

version: '3.9'


services:
ui:
# Your service options...
volumes:
- node_modules:/path/to/node_modules


volumes:
node_modules:
driver: local
driver_opts:
type: none
o: bind
device: ./local/path/to/node_modules

The key with this solution is to never make changes directly in your host node_modules, but always install, update, or remove Node packages in the container.

Version Control Tip: When your Node package.json/package-lock.json files change, either when pulling, or switching branches, in addition to rebuilding the Image, you have to remove the Volume, and delete its contents:

docker volume rm example_node_modules
rm -rf local/path/to/node_modules
mkdir local/path/to/node_modules

Documentation:

The best for development

docker-compose.yml

...
frontend:
build: ./app/frontend
ports:
- 3000:3000
volumes:
- ./app/frontend:/usr/src/app
...

./app/frontend/Dockerfile

FROM node:lts


WORKDIR /usr/src/app


RUN npm install -g react-scripts


RUN chown -Rh node:node /usr/src/app


USER node


EXPOSE 3000


CMD [ "sh", "-c", "npm install && npm run start" ]


#FOR PROD
# CMD [ "sh", "-c", "npm install && npm run build" ]

The user node will help you with the rights of host<->guest

The folder node_modules will be accessible from the host and synchronize host<->guest

You could also use dockerized npm install. This is the same as npm install but it runs on a docker container.

The node_modules will be written to the host. It should work out of the box and you can specify which npm version to use. If needed, the container can be extended or customized.

Be aware that some npm packages may require compilation, and the generated binaries may not be compatible with your host machine. If you just need the source code or dist files, this is not an issue.

Disclaimer: I'm the author of Dockerized.