WebPack deployment error: Cannot resolve module 'Long'

[jcalfee]

There is an issue deploying ByteBuffer 3.5.4 under webpack. Here are steps to reproduce:

Setup:

npm install webpack
npm install bytebuffer
echo "module.exports=require('bytebuffer')" > bb.js

Run:

webpack --entry ./bb.js bb_out.js
Hash: 1c6e193b5747ffb98b06
Version: webpack 1.8.11
Time: 271ms
   Asset    Size  Chunks             Chunk Names
bbout.js  192 kB       0  [emitted]  null
   [0] ./bb.js 37 bytes {0} [built]
    + 5 hidden modules

ERROR in ../~/bytebuffer/dist/ByteBufferAB.js
Module not found: Error: Cannot resolve module 'Long' in /home/jcalfee/bitshares/gui/node_modules/bytebuffer/dist
 @ ../~/bytebuffer/dist/ByteBufferAB.js 3271:8-87

[dcodeIO]

Relevant source file: https://github.com/dcodeIO/ByteBuffer.js/blob/master/src/ByteBufferAB.js

Which initializer is used by webpack?

[dcodeIO]

Any findings?

[jcalfee]

Sorry, we have no idea how to answer your question. It would be good to get a proper deployment though, webpack shows a warning because I have built it by hand using browserify. I did see a reference to SlowBuffer, I suspect that may be due to improper deployment.

[dcodeIO]

I have updated Long.js to export the dist file directly. Not sure if this makes a difference, so I tested it with the change. I got are two warnings about name conflicts on "Long" and "long", but it worked:

// testfile.js
var ByteBuffer = require("dist/ByteBufferAB.js");
console.log(ByteBuffer);

webpack testfile.js bundle.js --optimize-dedupe

Returns both ByteBuffer and ByteBuffer.Long properly populated.

[dcodeIO]

I also added a webpack.config.js now which points to the correct entry point. Not sure if this becomes evaluated though.

I think it's important that, for browser usage, dist/ByteBufferAB.js becomes webpacked, not dist/ByteBufferNB.js, which would explain why you are seeing SlowBuffer. Browserify does this right, but I am not sure how to tell webpack to use that file - that's why I created the webpack.config.

[jcalfee]

I cloned master into node_modules calling it bytebuffer .. I ran a require('bytebuffer') and got this result:

Error: Cannot find module 'long'
  at Function.Module._resolveFilename (module.js:338:15)
  at Function.Module._load (module.js:280:25)
  at Module.require (module.js:364:17)
  at require (module.js:380:17)
  at /home/jcalfee/bitshares/graphene-ui/dl/node_modules/bytebuffer/dist/ByteBufferNB.js:28:16
  at Object.<anonymous> (/home/jcalfee/bitshares/graphene-ui/dl/node_modules/bytebuffer/dist/ByteBufferNB.js:3431:3)
  at Module._compile (module.js:456:26)
  at Module._extensions..js (module.js:474:10)
  at Object.Module._extensions..js (/home/jcalfee/bitshares/graphene-ui/dl/node_modules/mocha-traceur/node_modules/traceur-runner/node_modules/traceur/src/node/require.js:70:12)
  at Module.load (/home/jcalfee/bitshares/graphene-ui/dl/node_modules/coffee-script/lib/coffee-script/register.js:45:36)
  at Function.Module._load (module.js:312:12)
  at Module.require (module.js:364:17)
  at require (module.js:380:17)
  at Object.<anonymous> (/home/jcalfee/bitshares/graphene-ui/dl/node_modules/bytebuffer/index.js:16:29)
  at Module._compile (module.js:456:26)
  at Module._extensions..js (module.js:474:10)
  at Object.Module._extensions..js (/home/jcalfee/bitshares/graphene-ui/dl/node_modules/mocha-traceur/node_modules/traceur-runner/node_modules/traceur/src/node/require.js:70:12)
  at Module.load (/home/jcalfee/bitshares/graphene-ui/dl/node_modules/coffee-script/lib/coffee-script/register.js:45:36)
  at Function.Module._load (module.js:312:12)
  at Module.require (module.js:364:17)
  at require (module.js:380:17)
  at Object.<anonymous> (/home/jcalfee/bitshares/graphene-ui/dl/src/common/bytebuffer.js:2:18)
  at Module._compile (module.js:456:26)
  at Object.Module._extensions..js (/home/jcalfee/bitshares/graphene-ui/dl/node_modules/mocha-traceur/node_modules/traceur-runner/node_modules/traceur/src/node/require.js:68:21)
  at Module.load (/home/jcalfee/bitshares/graphene-ui/dl/node_modules/coffee-script/lib/coffee-script/register.js:45:36)
  at Function.Module._load (module.js:312:12)
  at Module.require (module.js:364:17)
  at require (module.js:380:17)
  at Object.<anonymous> (/home/jcalfee/bitshares/graphene-ui/dl/test/crypto.coffee:2:8)
  at Object.<anonymous> (/home/jcalfee/bitshares/graphene-ui/dl/test/crypto.coffee:1:1)
  at Module._compile (module.js:456:26)
  at Object.loadFile (/home/jcalfee/bitshares/graphene-ui/dl/node_modules/coffee-script/lib/coffee-script/register.js:16:19)
  at Module.load (/home/jcalfee/bitshares/graphene-ui/dl/node_modules/coffee-script/lib/coffee-script/register.js:45:36)
  at Function.Module._load (module.js:312:12)
  at Module.require (module.js:364:17)
  at require (module.js:380:17)
  at /home/jcalfee/bitshares/graphene-ui/dl/node_modules/mocha/lib/mocha.js:184:27
  at Array.forEach (native)
  at Mocha.loadFiles (/home/jcalfee/bitshares/graphene-ui/dl/node_modules/mocha/lib/mocha.js:181:14)
  at Mocha.run (/home/jcalfee/bitshares/graphene-ui/dl/node_modules/mocha/lib/mocha.js:393:31)
  at Object.<anonymous> (/home/jcalfee/bitshares/graphene-ui/dl/node_modules/mocha/bin/_mocha:380:16)
  at Module._compile (module.js:456:26)
  at Object.Module._extensions..js (module.js:474:10)
  at Module.load (module.js:356:32)
  at Function.Module._load (module.js:312:12)
  at Function.Module.runMain (module.js:497:10)
  at startup (node.js:119:16)
  at node.js:902:3

[dcodeIO]

When cloning master, make sure to run npm install once to fetch the dependencies.

[lordmortis]

I'm also having this issue. I'm going to dig into it after i fix my deploy and see if i can figure out what's going on. It will only occur on systems with case sensitive filesystems.

[watsoncj]

@jcalfee 's test will pass with warnings on OSX (with a case-insensitive file system)

The test fails when run on a case-sensitive file system.

OSX

webpack --entry ./bb.js bb_out.js
Hash: 5c8a111c08044465faf9
Version: webpack 1.10.5
Time: 241ms
    Asset    Size  Chunks             Chunk Names
bb_out.js  230 kB       0  [emitted]  null
   [0] ./bb.js 37 bytes {0} [built]
    + 5 hidden modules

WARNING in ./~/bytebuffer/~/Long/dist/Long.js
There is another module with an equal name when case is ignored.
This can lead to unexpected behavior when compiling on a filesystem with other case-semantic.
Rename module if multiple modules are expected or use equal casing if one module is expected.

WARNING in ./~/bytebuffer/~/long/dist/Long.js
There is another module with an equal name when case is ignored.
This can lead to unexpected behavior when compiling on a filesystem with other case-semantic.
Rename module if multiple modules are expected or use equal casing if one module is expected.

[dcodeIO]

It's either require("long") with CommonJS (module name) or a plain/AMD "Long.js" dependency (script name).

Does webpack install dependencies automatically or do you have to add them manually? In case of the latter, it's recommended to rename the files to lowercase.

I assume, ideally, that using lowercase everywhere would be better practice, but that requires a bit of work on docs and stuff and I'd have to do this for all of the libraries involved simultaneously.

[watsoncj]

For webpack it's common to install dependencies through npm or possibly bower. Renaming the files might not be feasible in the case where the dependencies are managed.

I think the package.json browser property is designed for this purpose. Both "Long" and "long" can be aliased to the proper file path: https://gist.github.com/defunctzombie/4339901

[dcodeIO]

Both "Long" and "long" can be aliased to the proper file path

Sounds good, but is that file path always known? Like with npm, modules are in node_modules, or how is this handled by webpack?

[watsoncj]

I may be wrong about people using bower with webpack and browserify.

Common case is npm installs bytebuffer and it's node_modules.

[watsoncj]

Proposed fix here: https://github.com/dcodeIO/ByteBuffer.js/pull/59

[watsoncj]

A resolve alias seems to be a good solution if you just need to use ByteBuffer and Long (but not protobufjs).

  // webpack config.js
  resolve: {
    alias: {
      "Long": __dirname + "/node_modules/bytebuffer/node_modules/long/index.js"
    }
  },

Another good option (especially for protobufjs since webpack can't bundle it's usage of require) is the script-loader in combination with externals:

  // webpack.config.js
  entry: [
    // As of protobufjs 4.0.1+ webpack can't handle bundling, so put them on global scope
    'script!./node_modules/protobufjs/node_modules/bytebuffer/node_modules/long/dist/Long.min.js',
    'script!./node_modules/protobufjs/node_modules/bytebuffer/dist/ByteBufferAB.min.js',
    'script!./node_modules/protobufjs/dist/ProtoBuf-light.min.js',
    './app/scripts/app.js'
  ],
  ...
  // make global scope require-able
  externals: {
    dcodeIO: 'dcodeIO',
    protobufjs: 'dcodeIO.ProtoBuf',
    bytebuffer: 'dcodeIO.ByteBuffer'
  },

[Karabur]

Problem here is what webpack handles both AMD-style and CommonJS style definitions In code

    /* AMD */ if (typeof define === 'function' && define["amd"])
        define(["Long"], factory);
    /* CommonJS */ else if (typeof require === 'function' && typeof module === "object" && module && module["exports"])
        module['exports'] = (function() {
            var Long; try { Long = require("long"); } catch (e) {}
            return factory(Long);
        })();

There is two module inclusions, one define(["Long"], factory);, and another require("long") both of them processed by webpack and both threatened as different modules. So actually Long.js library included twice, but only one copy is used. On case sensitive systems, one module will fail with error 'not found'. (I believe amd-style)

In order to fix that they should be both declared with same name, and since CommonJS require is filesystem based, I think they both should be lowercase.

changing AMD code to `define(["long"], factory);' removes an error, but I am not sure if this is correct for AMD loaders.

There are workarounds for that, like specifying CamelCase alias for webpack pointing to node_modules:

    resolve: {
        alias: {
            'Long':path.join(__dirname, 'node_modules/bytebuffer/node_modules/long')
        },
    },

but I believe making library webpack-friendly worth it, taking into account rise in webpack's popularity and what it is not so easy to figure out what happens then you get that error first time (it is rare)

[Karabur]

btw, the same problem in protobuf.js with requiring bytebuffer

[dcodeIO]

That is a long standing issue. Actually, the libraries are correctly named Long, ByteBuffer etc., but npm doesn't allow upper case packages, which results in this situation. Ideally, I should rename everything to lowercase, but I haven't got around to this as it would already break so much stuff.

There is an example config with resolve/alias on the ProtoBuf repository, but sadly every single project has to include this on its own atm.

[watsoncj]

This is much better than having to use the script-loader to include protobufjs or bytebuffer. The script-loader approach can't set globals in IE10.

[jcalfee]

Can you have a pointer to both names Long, long, ByteBuffer, bytebuffer?

[watsoncj]

Yes, as long as you're ok having multiple instances of the ByteBuffer or Long:

  resolve: {
    alias: {
      protobufjs: __dirname + "/node_modules/protobufjs/dist/ProtoBuf-light.js",
      Long: __dirname + "/node_modules/protobufjs/node_modules/bytebuffer/node_modules/long/dist/Long.js",
      long: __dirname + "/node_modules/protobufjs/node_modules/bytebuffer/node_modules/long/dist/Long.js",
      ByteBuffer: __dirname + "/node_modules/protobufjs/node_modules/bytebuffer/dist/ByteBufferAB.js",
      bytebuffer: __dirname + "/node_modules/protobufjs/node_modules/bytebuffer/dist/ByteBufferAB.js",
    }
  },

[dcodeIO]

Started conversion to lower case with Long. Will become 3.X.

[jcalfee]

It is ok if they are just instance pointers .. but I would not want multiple instances...

[dcodeIO]

Here goes ByteBuffer: https://github.com/dcodeIO/ByteBuffer.js/commit/53fcc7c60d9e7140b2852afb25cbbce069a4d88c

[dcodeIO]

And ProtoBuf: https://github.com/dcodeIO/ProtoBuf.js/commit/642560c0bd65c685d1c089cd2f588c6ed35265c7

[Karabur]

Great! Many thanks. Looks like issue can be closed

[Karabur]

Please publish updated libraries to npm. Direct install of protobfjs from github is broken as there is no v5.0.0 bytebuffer package published.

[dcodeIO]

Published bytebuffer, currently checking protobuf.

[marcus-sa]

Well this error keeps occurring to me, and none of the answers above solves it.