{"action":"created","issue":{"url":"https://api.github.com/repos/npm/npm/issues/4210","repository_url":"https://api.github.com/repos/npm/npm","labels_url":"https://api.github.com/repos/npm/npm/issues/4210/labels{/name}","comments_url":"https://api.github.com/repos/npm/npm/issues/4210/comments","events_url":"https://api.github.com/repos/npm/npm/issues/4210/events","html_url":"https://github.com/npm/npm/issues/4210","id":23407865,"number":4210,"title":"Bundle/Include Dependencies ","user":{"login":"arei","id":1062655,"avatar_url":"https://avatars.githubusercontent.com/u/1062655?v=3","gravatar_id":"","url":"https://api.github.com/users/arei","html_url":"https://github.com/arei","followers_url":"https://api.github.com/users/arei/followers","following_url":"https://api.github.com/users/arei/following{/other_user}","gists_url":"https://api.github.com/users/arei/gists{/gist_id}","starred_url":"https://api.github.com/users/arei/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/arei/subscriptions","organizations_url":"https://api.github.com/users/arei/orgs","repos_url":"https://api.github.com/users/arei/repos","events_url":"https://api.github.com/users/arei/events{/privacy}","received_events_url":"https://api.github.com/users/arei/received_events","type":"User","site_admin":false},"labels":[{"id":327795332,"url":"https://api.github.com/repos/npm/npm/labels/already-looked-at","name":"already-looked-at","color":"FEC9ED","default":false},{"id":104298906,"url":"https://api.github.com/repos/npm/npm/labels/feature-request","name":"feature-request","color":"98DCFF","default":false},{"id":101645795,"url":"https://api.github.com/repos/npm/npm/labels/needs-discussion","name":"needs-discussion","color":"0A98AC","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":31,"created_at":"2013-11-27T18:58:42Z","updated_at":"2016-12-20T00:21:20Z","closed_at":null,"body":"# Feature Description\n\n**Create a means within npm of generating a `.tgz` representation of a npm package AND all of its dependencies such that an `npm install` can be be done using the `.tgz` representation to install the package AND all of its dependencies.**\n\nThis involves two separate and distinct aspects:\n1. Add an ability to npm to create a `.tgz` file that contains a single npm package in its uninstalled form and its entire dependency tree in their uninstalled forms.  See below for additional details.\n2. Change `npm install <tarball file>` to understand that a `<tarball file>` may contains the package and the cache from which all dependencies should be installed. See below for additional details.\n\nThis is a rewrite of the closed ticket #3429.\n# Reason for Feature Request\n\nThe reason for this feature request is to **provide a means for npm to be used to install a given package in an offline environment and without relying on the npm registry being available online.**  By using a bundled npm package tarball that contains the package and all necessary dependencies, npm is not forced to go out to the internet and find the dependencies required.  This functionality is desirable for use in isolated/off-the-grid systems and for usage by package owners whom wish to distribute their package as a complete unit.\n\nConsider this simple problem case: **Install `socket.io` on a system that is 100% offline and not connected to the internet. (The target system has node/npm installed already.)**  \n\n(In the process described below \"LIVE\" indicates a system with an internet connection and \"OFFLINE\" indicates a system without an internet connection.)\n\nYour first attempt might go something like this:\n1. [LIVE] Create a new directory and change to it.\n2. [LIVE] Install socket.io using npm: `npm install socket.io`\n3. [LIVE] tar the `node_modules` directory.\n4. [LIVE] copy the tar to some removable media like a memory stick.\n5. [OFFLINE] move the memory stick to the remote offline system.\n6. [OFFLINE] copy the tar from the removable media.\n7. [OFFLINE] untar the `node_modules`\n\nThe problem with this is that this did not actually install `socket.io` on the remote system, but just duplicated the files of an installed `socket.io`. This ignores any special install process instructions that `socket.io` might want to perform on the system to which it is installed. In some cases it's good enough, but it's not the right answer. \n\nYour next attempt might be to get the `<tarball file>` of `socket.io` and use that.  \n1. [LIVE] Create a new directory and change to it.\n2. [LIVE] Get the 'socket.io' tarball using npm pack: `npm pack socket.io`\n3. [LIVE] copy the tar to some removable media like a memory stick.\n4. [OFFLINE] move the memory stick to the remote offline system.\n5. [OFFLINE] copy the tar from the removable media.\n6. [OFFLINE] Install `socket.io` from the tarball: `npm install socket.io-0.9.16.tgz`\n7. [OFFLINE] Stare confusedly at: \n\n```\nnpm WARN optional dep failed, continuing redis@0.7.3\nnpm ERR! Error: connect ECONNREFUSED\nnpm ERR!     at errnoException (net.js:901:11)\nnpm ERR!     at Object.afterConnect [as oncomplete] (net.js:892:19)\nnpm ERR!  { [Error: connect ECONNREFUSED]\nnpm ERR!   code: 'ECONNREFUSED',\nnpm ERR!   errno: 'ECONNREFUSED',\nnpm ERR!   syscall: 'connect' }\n```\n\nnpm is attempting to go out and get the dependencies from http://registry.npmjs.org which clearly will not work on our remote offline system.\n\nOkay, so you can just get and install all the dependencies. Except there isn't a way to list the dependencies of an npm package from npm. The best way to get the dependency tree in node is to actually install the npm package, so let's do that.  \n1. [LIVE] Create a new directory and change to it.\n2. [LIVE] Install socket.io using npm: `npm install socket.io`\n3. [LIVE] Make a note of everything that was installed: socket.io, socket.io-client/0.9.16, policyfile/0.0.4, base64id/0.1.0, redis/0.7.3, uglify-js/1.2.5, ws, xmlhttprequest/1.4.2, active-x-obfuscator/0.0.1, zeparser/0.0.5, commander, tinycolor, nan, and options\n4. [LIVE] For each item installed, go get the `<tarball file>` using `npm pack`.  Don't forget to change the '/' to an '@' for packages that are given with a specific version.\n5. [LIVE] `npm pack socket.io`\n6. [LIVE] `npm pack socket.io-client@0.9.16`\n7. [LIVE] `npm pack policyfile@0.0.4`\n8. [LIVE] `npm pack base64id@0.1.0`\n9. [LIVE] `npm pack redis@0.7.3`\n10. [LIVE] `npm pack uglify-js@1.2.5`\n11. [LIVE] `npm pack ws`\n12. [LIVE] `npm pack xmlhttprequest@1.4.2`\n13. [LIVE] `npm pack active-x-obfuscator@0.0.1`\n14. [LIVE] `npm pack zeparser@0.0.5`\n15. [LIVE] `npm pack commander`\n16. [LIVE] `npm pack tinycolor`\n17. [LIVE] `npm pack nan`\n18. [LIVE] `npm pack options`\n19. [LIVE] copy the tars to some removable media like a memory stick.\n20. [OFFLINE] move the memory stick to the remote offline system.\n21. [OFFLINE] copy the tars from the removable media.\n22. [OFFLINE] Install `socket.io` from the tarball: `npm install socket.io-0.9.16.tgz`\n23. [OFFLINE] Stare confusedly at:\n\n```\nnpm WARN optional dep failed, continuing redis@0.7.3\nnpm ERR! Error: connect ECONNREFUSED\nnpm ERR!     at errnoException (net.js:901:11)\nnpm ERR!     at Object.afterConnect [as oncomplete] (net.js:892:19)\nnpm ERR!  { [Error: connect ECONNREFUSED]\nnpm ERR!   code: 'ECONNREFUSED',\nnpm ERR!   errno: 'ECONNREFUSED',\nnpm ERR!   syscall: 'connect' }\n```\n\nWell, this is the same problem as before. Basically, there is not an easy way to tell npm to use all these `.tgz` files that we have, or is there?  \n\nnpm does cache it's files and we can set that cache to somewhere, so maybe we can use the cache functionality. \n1. [LIVE] Create a new directory and change to it.\n2. [LIVE] Install socket.io using npm because this is the only way to get our dependency tree. Additionally, let us store our cache some where clean: `npm --cache ./.cache install socket.io`\n3. [LIVE] tar up the .cache folder.\n4. [LIVE] copy the tar to some removable media like a memory stick.\n5. [OFFLINE] move the memory stick to the remote offline system.\n6. [OFFLINE] copy the tar from the removable media.\n7. [OFFLINE] untar the tar, creating .cache in the process.\n8. [OFFLINE] Install `socket.io` from the .cache directory: `npm install --cache ./.cache socket.io`\n9. [OFFLINE] Wait.  (This step will take a little while as npm tries to connect to the registry to see if newer versions are available.  You can speed this up playing around with some of the npm config options regarding fetch.)\n\nSuccess! A little convoluted and a bit of extra work, but a success.  But I think we can do better...\n# Feature Breakdown\n## Goals for the Feature\n- Be able to install a package including all dependencies in an offline environment.\n- Minimize the objects that have to be moved (tar everything up).\n- Minimize the number of steps necessary to create and install.\n- Disable checking the registry for things that are already in the bundle.\n\nSpecifically, there are two aspects to this feature request: Bundling a Package into a `.tgz` that includes its dependencies, and using npm to install from said bundle.\n## Bundle Package\n\nIn this aspect of the feature the objective is to create a `.tgz` that contains the package and all the dependencies necessary to install that package and the other dependencies.\n\nFor a given package...\n- Build the dependency tree for the package including the dependency tree for each dependent package.\n- Download each item in the dependency tree.\n- Bundle all items into a `.tgz`.\n\nSuggestions and additional things to keep in mind...\n- The `.tgz` file created should have the name and version of the initial package.  Additionally the name might include something to indicate that this also bundles the dependencies. `socket.io-0.9.16.withDependencies.tgz` for example.\n- This could be it's own command like `npm box socket.io` or it could be a variant of `npm pack` like `npm pack --includeDependencies socket.io` or `npm pack --all socket.io` or `npm packall socket.io`.\n- One approach might be to simply insert the dependencies into the initial package `.tgz` and then make `npm install <tarball file>` look inside the tarball for the dependencies before going out to the registry.\n- Should include optional modules as if `--optional` was specified.\n## Install from Bundle\n\nIn this aspect of the feature the objective is to take a previously bundled `.tgz` file and install the package and all mecessary dependencies from said file.\n\nFrom a given `.tgz` file...\n- Install the initial package from the `.tgz`.\n- Install all dependent packages from the `.tgz`\n- Fall back to searching the registry for packages when they are missing from a `.tgz` file.\n\nSuggestions and additional things to keep in mind...\n- Allow for all the normal behavior of `npm install` such as global.\n- If a dependency exists in the `.tgz` and is of a valid version, it must be used.\n# Relevant Discussion/Links\n\nThis ticket was initially written up in #3429.  I believe that #3429 was closed due to misunderstanding of the desired feature and/or the problem space.  This rewrite is an attempt to clarify that misunderstanding.\n\n[npmbox](https://github.com/arei/npmbox) is an attempt to provide an example of the desired functionality described in this feature request in a third party tool.  However, npmbox is not a final solution because it depends on npmbox being first installed on the target system and thus we have a chicken and egg problem.\n\n[pac](https://npmjs.org/package/pac) is somewhat related, but also not really the solution desired.  It works by examining your package.json file and then downloading the tarball of any package listed there as a dependency into the .modules folder.  While this might seem like a similar answer, it does not download the entire dependency tree nor does it provide a means to install the package or its dependencies.\n"},"comment":{"url":"https://api.github.com/repos/npm/npm/issues/comments/268117622","html_url":"https://github.com/npm/npm/issues/4210#issuecomment-268117622","issue_url":"https://api.github.com/repos/npm/npm/issues/4210","id":268117622,"user":{"login":"jdmairs","id":5624808,"avatar_url":"https://avatars.githubusercontent.com/u/5624808?v=3","gravatar_id":"","url":"https://api.github.com/users/jdmairs","html_url":"https://github.com/jdmairs","followers_url":"https://api.github.com/users/jdmairs/followers","following_url":"https://api.github.com/users/jdmairs/following{/other_user}","gists_url":"https://api.github.com/users/jdmairs/gists{/gist_id}","starred_url":"https://api.github.com/users/jdmairs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jdmairs/subscriptions","organizations_url":"https://api.github.com/users/jdmairs/orgs","repos_url":"https://api.github.com/users/jdmairs/repos","events_url":"https://api.github.com/users/jdmairs/events{/privacy}","received_events_url":"https://api.github.com/users/jdmairs/received_events","type":"User","site_admin":false},"created_at":"2016-12-20T00:21:20Z","updated_at":"2016-12-20T00:21:20Z","body":"I could not get npmbox to install node-gyp successfully...everything else has worked great.  Comments would be appreciated!"}}
{"actor":{"display_login":"jdmairs"}}