Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarification on using dependencies or devDependencies #520

Closed
dschissler opened this issue Oct 9, 2014 · 26 comments
Closed

Clarification on using dependencies or devDependencies #520

dschissler opened this issue Oct 9, 2014 · 26 comments

Comments

@dschissler
Copy link

I'm looking for clarification on using dependencies or devDependencies. For example, I'm using backbone through npm and I have it under devDependencies. My final build does not require any node environment so I'm wondering what the best convention is for this. It practically doesn't seem to make much of a difference but it would be great to understand the best case convention.

@havenchyk
Copy link

I think, this answer can be useful in your case. This question doesn't relate to webpack, but for any npm package.

@jhnns
Copy link
Member

jhnns commented Oct 10, 2014

Sorry, I don't see how this question relates to webpack.
http://lmgtfy.com/?q=npm+dependencies+devdependencies 😉

@dustingetz
Copy link

I agree with @dschissler that everybody is doing this wrong. A browser app built by webpack has no runtime node dependencies, and thus all frontend dependencies should be listed as devDependencies. The dependencies vs devDependencies naming convention stems historically from node being a server side package manager, which was abused into a frontend package manager, and this is why the fields have names that are counter-intutive when applied to frontend dev, and is why every project ever is getting this wrong. It is as far as I can tell harmless to list frontend dependencies under dependencies, but it is wrong.

The way I ran into this was
NODE_ENV=production npm install
NODE_ENV=production webpack

which of course failed because none of the devDependencies were downloaded by npm install, as described in the npm install docs. I am exposing NODE_ENV=production to webpack, because React inspects that environment variable, perhaps this is an issue with react, I don't fully understand.

@dkreft
Copy link

dkreft commented Aug 10, 2016

@dustingetz I'm a bit confused by your response. You say "all frontend dependencies should be listed as devDependencies" but then you said "...none of the devDependencies were downloaded by npm install" when you set NODE_ENV=production. This would seem to be an argument for tracking dependencies in dependencies instead of devDependencies. Could you please clarify?

@pke
Copy link

pke commented Oct 6, 2016

@dkreft I agree. Everything should just go into dependencies. That way one would not have to worry what NODE_ENV points to when calling npm i.

@dschissler
Copy link
Author

@dkreft @pke You guys are right that @dustingetz doesn't address that big issue but you're solution doesn't address an issue that I will have. So I have a PHP framework and I'm using Webpack to build front end resources as explained above. Then eventually I will use a production build to run some Nodejs websocket tech and I'd rather not download all of the webpack stuff for this purely Nodejs production server. Its possible that it should be a separate package.json for that part of the code but that would complicate other matters.

So I'm considering to export WEBPACK_ENV to webpack to determine whether to run the webpack-dev-server or a production build. Then in will use the constant plugin to send NODE_ENV to the modules to be bundled, since the frameworks have settled on that.

@pbrn46
Copy link

pbrn46 commented Sep 24, 2017

This conversation states why it works, but leaves one unsure of the reasoning behind it. Webpack projects seem to add a layer of confusion, but we are simply not asking ourselves the right questions: "How will this project be used in production?", and "Will we run webpack during production?"

The argument to put most dependencies under dependencies instead of devDependencies can actually go both ways. More commonly I see people start with putting everything in dependencies but should lean towards devDependencies, and figure out how to use it properly. See the two scenarios below.

Scenario 1: If your project needs to run webpack in the production environment before serving:

Packages required to build the webpack project are under dependencies, rather than devDependencies. As @dustingetz says, node is historically used as a "server", but this is actually still be considered true. For webpack (and many react) projects, we are actually using webpack as a build tool, which translates to our project being considered a "source project" waiting to be built. We now also call our project a "build server".

In this scenario, our project is no longer considered the "front end app", but instead "the project that builds the front end app". Assuming statement above to be reasonable, dependencies are the requirements to build this project and devDependencies will have things that are not related to building, such as unit testing.

Scenario 2: If our project does not need to run webpack in the production environment, and has a pre-built copy to serve (or require by other projects):

This scenario is if we do not need to run webpack in production and we only serve the pre-built files in a node server. This means our project will need to ship with our build folder. The webpack and webpack app dependencies can be placed in devDependencies.

This scenario is also used if we are creating library packages, instead servers. Others will install the project as a package, and require the pre-built files in their project. If one wishes to help develop our package, then they can clone and install with devDependencies to enable webpack building.

A caveat to this scenario is that for production, one must have already built using webpack in a development environment.

Summary:

Webpack projects should be considered build-type projects first, and not production-type projects, unless webpack is used in production. Packages needed during production goes in dependencies, and packages used just to develop the project goes in devDependencies. If our production environment needs to build (run webpack), then those dependencies must reside in dependencies.

Of course, one can always put everything in dependencies, but if we are looking to publish our project for others to use or require, then this will consequently bloat their projects' node_modules folder unnecessarily. Scenario 2 should be the way to go: exhaust devDependencies unless it is required otherwise.

That's it! That's all you need to know!

Whichever scenario you find yourself in, I would love to know if there are any other pitfalls and caveats!

Supplementary information:

  1. We are generally in scenario 1 if webpack builds to a folder under .gitignore
  2. React's usage of environment variables, such as the NODE_ENV variable in scenario 1 will generally be correct, as @dkreft points out.
  3. In scenario 2, to rebuild using webpack we must run npm install as NODE_ENV=development (default) rather than NODE_ENV=production. We must define separately the proper NODE_ENV (and other variables) when running webpack depending on whether we are building for production or for development.
  4. Of course, if we just have the project as just a web server that dishes out the already-built webpack project (from elsewhere), then the webpack dependencies don't belong in either dependencies or devDependencies (they belong nowhere since they're already packed in the build! 😛). If we develop our webpack / app within the same project as the server, however, then we are effectively in scenario 2 above and need the appropriate devDependencies.
  5. To keep things simple, most projects, typically small or starter web projects will fall under scenario 1, so they can build and serve at the same time during production. As the project scales, one may find themselves splitting up their server and the webpack app into separate commands or even projects, which will require them to be in scenario 2.
  6. If we use webpack to build an app to deploy somewhere, and no other person or project depends on it, then none of this matters.
  7. When developing on your project, one shouldn't be calling NODE_ENV=production npm install anyway.

@dustingetz

A browser app built by webpack has no runtime node dependencies, and thus all frontend dependencies should be listed as devDependencies.

This quote is true and untrue. A browser app built by webpack has no node dependencies. However, many projects not only serve but also build during production. It is true that this is an unconventional use of the word "build" for node developers (in scenario 1), as building typically refers to doing it before production instead of during. Instead of build and run in separate commands, it's more like "build-to-run" in one.

Sorry for the long text, probably should have been a blog post, but I think this deserves an explanation somewhere on the internet!

@jrmcdona
Copy link

jrmcdona commented Sep 30, 2017

@pbrn46 In what circumstance does one have to run webpack in production before serving?

I have a build server that runs webpack and create the bundles, then I copy these distibution files to the server. So at this point in time I do not need to run webpack yet again in prod.

Who would run webpack in prod at runtime before serving, isn't that really ineffcient?

And in my case should everything go under devDependencies? Of course for development I run locally. Is there any impact on the bundles based on what is in dependencies vs devDependencies?

@pbrn46
Copy link

pbrn46 commented Sep 30, 2017

@jrmcdona For your current situation, and as a general recommendation for everyone, move everything into devDependencies until it is actually needed under dependencies.

Since you are copying your files to a production server, and the project itself doesn't serve in production, then it is recommended to use devDependencies, even though both will work fine for you.

Supplemental information

Webpack in production

Running webpack in production should actually be a special use case. This is where your package is actually meant for the end user to webpack (implicitly) with your package. The best example would be if you are creating something like the create-react-app (see the underlying react-scripts package's package.json), where the package actually compiles the user code with webpack, so webpack would be under dependencies for create-react-app. Another example would be if you are making a package that dynamically builds a .js module for the user to import in their project, but requires extra code from the user, before building the final .js. In any case, it still remains a special use case.

Efficiency

In terms of efficiency for running webpack in production (as a server), it would only be the first build that takes time, as the outputs of webpack are static files after the build. The files are not rebuilt every time a client sends a request (although this can actually be a way webpack is used for the super-advanced-evil-masterminds). For the lazy people who have their server and build in the same project, the usual setup is on every change they would crash the server, run webpack, and restart the server (for example using express with node-mon or supervisor); although this method is still typically used for development before publishing to another server anyway. It is usually a good idea to keep the server in a separate process/project, as they are really two different things.

Redistributing your package for others to program or build

This conversation is actually only applicable if you want to redistribute the project.

Interestingly enough, if you redistribute your project just for people to build, then you need your webpack and build dependencies (like loaders, babel, sass, etc) should be in dependencies. Your testing package to edit your code will be in devDependencies, since that's not required to build.

Now, if you distribute your project with the pre-built files and the only use is for others to import/include your pre-builts, then everything should be in devDependencies.

@dustingetz
Copy link

@pbrn46 can you state that in like two sentences so i can understand what you're talking about, thanks

@pbrn46
Copy link

pbrn46 commented Sep 30, 2017

@dustingetz Haha, sorry for the long text. Prone to being unspecific to use case, the simple answer is I agree webpack should belong in devDependencies, unless the project calls webpack in production. I wanted to have a thorough explanation out there for others who share my interest or to debate why this is.

@havenchyk
Copy link

@pbrn46 maybe I'm late to the party, but IMHO

in dependencies section you can (should?) include only packages that will be used to build and run "production" version of your website (with NODE_ENV=production, minified, gzipped etc), all the other things can (should?) be in devDependencies.

So e.g. yarn you can use with yarn --production flag and it's the only way not to install test libs and e.g. phantomjsif you want just to build your project.

So from that point of view webpack should go to dependencies section

@at-daonguyen
Copy link

at-daonguyen commented Oct 16, 2017

packages that will be used to build and run "production" version of your website

Eh, then babel should be listed in dependencies, right?

@havenchyk
Copy link

@at-daonguyen you got it right, babel in dependencies, phantomjs-prebuilt, eslint etc in devDependencies. But again, it's imho

@nappy
Copy link

nappy commented Nov 6, 2017

I think everything that is needed to create the bundle.js belongs in dependencies unless you commit the compiled files into VCS. Because if in the end they are not present in the tar npm pack provides, in order to use your project, one would need to compile it. And here imho it does not matter if that build happens on a build server or on your web server, in both cases I would consider this "production", because without it you would not have anything to run at all.

@sijakret
Copy link

sijakret commented Feb 1, 2018

I think everything that is needed to create the bundle.js belongs in dependencies unless you commit the compiled files into VCS

At the same time, in my opinion, everything that is bundled into the distribution should not be in dependencies in most scenarios.
Take a look at the vue.js or react.js package.json for example..

https://github.com/vuejs/vue/blob/dev/package.json
https://github.com/facebook/react/blob/master/package.json

..they have 0(!) dependencies, only devDependencies. The dev dependencies are a lot of front-end libraries that people tend to reference as ordinary dependencies. However since the distribution is based around a pre-built bundle vue.js/react.js are doing it exactly the right way in my opinion.

was this thread conclusive?
i totally feel @dschissler's original question and believe that a look at vue/react pretty much gives a clear direction:

If a bundler is (like webpack) is involved and you distribute a compiled bundle (pretty much always the case in front-end development), then you probably mostly need devDependencies and probably little to no dependencies.

@nappy
Copy link

nappy commented Feb 2, 2018

At the same time, in my opinion, everything that is bundled into the distribution should not be in dependencies in most scenarios.

Yes, I think in general the best practice is to add !/build to .npmignore, so that your /build folder is part of the distribution.
That would leave the dependencies empty.
The only slightly confusing thing about this practice is, that for example create-react-app uses npm start for development. If you are strict about this it just should be a no operation because someone depending on your package can not start it with npm install --production && npm start.

@naeramarth7
Copy link

@nappy

create-react-app uses npm start for development
can not start it with npm install --production && npm start

You said it yourself, npm start is for development - and starts the webpack-dev-server that's not intended to be used for production; so why should it be available when installing production dependencies only?

@nappy
Copy link

nappy commented Feb 5, 2018

@naeramarth7 Because npm start defaults to node server.js which is normally what you want to do in production. You can override it by script, but why would you deviate that much from the default and do something only for development there?
If I know nothing about your project, other than its a npm package, I would assume npm install --production && npm start to do something or nothing (other than failing).
I dont say you should put everything into dependencies, only because of this, but imho for start it would have made more sense to put something like serve -s build there, because that is the intended use of your package. (And maybe we would have had less confusion about this topic, if that had been the case)

But yeah, anyway, one could argue that its up to the developer to decide which script should be used in which scenario, because its not clearly defined.

dcsaszar added a commit to Scrivito/scrivito_example_app_js that referenced this issue Apr 24, 2018
* Includes update to npm 6.0.0
* According to webpack/webpack#520 (comment), all deps should reside in `dependencies`
* This also improves compatibility with the default behavior of `npm install x`
dcsaszar added a commit to Scrivito/scrivito_example_app_js that referenced this issue Apr 25, 2018
* According to webpack/webpack#520 (comment), all deps should reside in `dependencies`
* This also improves compatibility with the default behavior of `npm install x`
lucasfevi added a commit to lucasfevi/afropython-site that referenced this issue Oct 31, 2018
@christopher-francisco
Copy link

@pbrn46 You didn't mention the --prod flag, which I think is a very big factor on whether even having dependencies and devDependencies split.

Following your explanation for Case 1, which consists in acknowledging the project as "a project that builds an app" rather than "an app", and furthermore having the unit testing packages under devDependencies while the packages needed for building under dependencies, I can now see myself using the --prod flag in a scenario, giving sense of even splitting the dependencies into these 2.

Said scenario is building my app, specifically into a docker image but I wouldn't limit it to just that, for I don't care about testing dependencies when I'm building static assets for my app and bundling it into a docker image.

Great example btw, @pbrn46

@nappy
Copy link

nappy commented Jan 25, 2019

@chris-fran That approach is perfectly valid. The only drawback of that approach is that you require your dependents to always build at least the optimized --prod build themselves. It would not make sense to include those generated assets in the distribution of that package, because then they would depend on all your build dependencies.
In your case you do not intend to push something to an npm registry at all, because you replace that functionality by distributing through docker.

@jondavidjohn
Copy link

As @dustingetz suggests, it's very common to need to set NODE_ENV to production to get optimized builds, another thing I haven't seen mentioned here is that you can also pass a value to the production flag of your package manager to override this if you're not finding your devDependencies when you expect to.

yarn install --production=false

or

npm install --production=false

https://packtracker.io/blog/devdependencies-are-not-installing-with-npm/

willianantunes added a commit to willianantunes/nextjs-playground that referenced this issue Dec 26, 2019
- Use package-lock.json through command: npm ci
- Fixes regarding Docker Compose setup
- Multistage build to get only what is needed for PRD image

Useful links
- https://git.io/JebrQ
- webpack/webpack#520 (comment)
willianantunes added a commit to willianantunes/nextjs-playground that referenced this issue Dec 30, 2019
- Use package-lock.json through command: npm ci
- Fixes regarding Docker Compose setup
- Multistage build to get only what is needed for PRD image

Useful links
- https://git.io/JebrQ
- webpack/webpack#520 (comment)
@ChineseStuff
Copy link

@pbrn46 hello, thanks for your clear answers, one cuestion. I i'm working with react app with a mock db generated in start script and a jason-server. I'm playing a little with heroku, trying to connect my GH repo with cd/ci heroku workflow, so every time that I push to master heroku re-build project, so should I have webpack, json-server and babel as prod dependencies right?

@gsouf
Copy link

gsouf commented Apr 25, 2020

Haven't read everything but to me the question is more relevant when you dev a library rather than an application.

Library - the library is reused. People don't need your dev deps (ie testing library, etc...)
Application - application is a generally a end product. In most of cases it does not make much differences as stated above because anyway you usually need everything to test and build your app before deploying a compiled version of your app.

@leoskyrocker
Copy link

leoskyrocker commented Feb 10, 2021

I've read through the comments here, but it seems like no one mentioned something that I thought makes it very useful to separate deps into deps and devDeps. Consider the following use case:

Scenario

  • You build static webpages before serving them in production
  • You know there are libraries that you use that are solely for dev purposes, and is never imported in your app
  • You know there are libraries that you'll be importing in your app

Use Case

When building your app pre-production, you use npm install or npm ci with production mode:
NODE_ENV=production npm i or npm ci --production.
Then you run npm run build.
Now, let's assume you or your coworker accidentally imported a dev dependency in your code, but it's really meant for use during development, npm run build is going to blow up because the dependency will not be installed, and we don't really expect it to be needed during build time either.

@govindrai
Copy link

govindrai commented Jun 25, 2021

For us this was a problem because our client/server code shared one package.json. This was making our server host a bunch of client side dependencies that were already taken care of at build time. Basically we only needed 5/25 dependencies on our server. The solution to this was to separate our client and server into separate directories and have their own package.json. Then actual client side dependencies could still be listed as dependencies without bloating our server dependencies.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests