How should we manage dependencies?

[wilzbach]

If you haven't done so, please read this excellent article as an intro to the problem of dependency management in web components by @Tjvantoll:

http://tjvantoll.com/2014/08/12/the-problem-with-using-html-imports-for-dependency-management/

@Tjvantoll discusses the following options - for convenience shortly summarized here:

Option 1: Use a CDN

<script src="//cdnjs.cloudflare.com/ajax/libs/moment.js/2.7.0/moment.min.js"></script>

• only works if exactly the same URL (same CDN provider, same protocol, same version) has been choosen
• prevents script concatention
• prevents offline development

Option 2: Enforcing a folder structure on the end user + relative imports

• The Polymer project uses this strategy
• Could be done with either npm or bower
• Requires strong convention (naming, structure)
• The developer would need to upload two files to /dist (1 dev file with relative imports, 1 minimized with all dependencies)
• Can't deal with version conflicts (e.g. two versions of jquery)

The Polymer project is using this approach - here is the distribution guide for 0.5 written by @addyosmani.

Option 3: Feature detection

Summary: using a tiny module loader to detect whether the script has already been loaded

• two modules can still request the same module with a different URL
• has to be done asynchronously in dev environment
• no/harder build (vulcanize) process

Option 4: Not using HTML imports for external dependencies

Summary: no reference in the component's code - the dependency is only referenced in the documentation

---

This also has been discussed at the Polymer project.

Here is my opinion to start the discussion: Option 2 (enforcing a common folder structure) seems to be the lesser evil.

However for dependency conflicts:

• bower's conflict resolution warns you about a conflict and you can choose which version you want to load, but it doesn't help if two components really depend on two different versions (e.g. Angular 1 and Angular 2)
• npm3 (currently in beta) is awesome at detecting whether there will be a semantic conflict (as long as the author uses correct semantic versioning) - however if there is a conflict it will put the conflicting file in the components node_modules folder (depending on what module was installed first). As an ugly workaround an author could just include both paths (node_modules/<lib>/dist/<lib>.html and ../<lib>/dist/<lib>.html)

I am very much looking forward to your thoughts :)

[herkulano]

HTML Imports are also being contested by Mozilla and they will not implement them. They propose the use of ES6 modules instead.

"Mozilla will not ship an implementation of HTML Imports. We expect that once JavaScript modules — a feature derived from JavaScript libraries written by the developer community — is shipped, the way we look at this problem will have changed." https://hacks.mozilla.org/2014/12/mozilla-and-web-components/

ECMAScript 6 provides the following ways of importing:

    // Default exports and named exports
    import theDefault, { named1, named2 } from 'src/mylib';
    import theDefault from 'src/mylib';
    import { named1, named2 } from 'src/mylib';

    // Renaming: import named1 as myNamed1
    import { named1 as myNamed1, named2 } from 'src/mylib';

    // Importing the module as an object
    // (with one property per named export)
    import * as mylib from 'src/mylib';

    // Only load the module, don’t import anything
    import 'src/mylib';

ES6 Modules Final - Article

[Hypercubed]

I'm hoping for an ES6 module solution. Currently SystemJS and Web Components are very tangential.

[Hypercubed]

@greenify, If I understand correctly the four options above are regarding html imports. But html imports is not a necessity of web components. From https://hacks.mozilla.org/2015/06/the-state-of-web-components/:

We’ve been working with Web Components in Firefox OS for over a year and have found using existing module syntax (AMD or Common JS) to resolve a dependency tree, registering elements, loaded using a normal <script> tag seems to be enough to get stuff done.

Using SystemJS might be something to look into.

[herkulano]

SystemJS as a module loader

Added a new branch for this test: https://github.com/herkulano/biojs3-webcomponent-example/tree/dependency-systemjs

Loading module in .js

System.import('../../d3/d3.min.js').then(function() {
  Polymer({
  ...

https://github.com/herkulano/biojs3-webcomponent-example/blob/dependency-systemjs/biojs-component.js#L1

Loading SystemJS in index.html

<script src="../../systemjs/dist/system.js"></script>

https://github.com/herkulano/biojs3-webcomponent-example/blob/dependency-systemjs/demo/index.html#L10

Dependencies listed in bower.json

  "dependencies": {
    "polymer": "Polymer/polymer#^1.0.0",
    "d3": "~3.5.6",
    "systemjs": "https://github.com/systemjs/systemjs.git#0.18.4"
  },

https://github.com/herkulano/biojs3-webcomponent-example/blob/dependency-systemjs/bower.json#L23

It still depends on bower to install all dependencies, i.e., the end-user still has to use bower to install the dependencies.

Any thoughts?

[wilzbach]

I'm hoping for an ES6 module solution. Currently SystemJS and Web Components are very tangential.

I also find the ES6 module specification quite tempting - especially if we have a loader like SystemJS that supports AMD, CommonJS and ES6 modules. However we still would require a common structure to be able to avoid duplications when loading modules?

It still depends on bower to install all dependencies, i.e., the end-user still has to use bower to install the dependencies.

I have created an example that gets all internal dependencies and Polymer from SystemJS.

https://github.com/greenify/polymer-systemjs-loading

Unfortunately it also doesn't solve the problem that we have if someone imports third-party dependencies like System.import('biojs-core'); as SystemJS would need to have a way to figure out where the module is. So maybe the Polymer approach of using a common structure isn't that bad at all?

[Hypercubed]

@greenify we are working on the same thing.

SystemJS knows where the modules are based on the System.config.js. Using jspm this is automatic.

jspm install d3=github:github:mbostock/[email protected]

then in the module you can:

import d3 from 'd3';

[Hypercubed]

It is additionally complicated if you want to use bundling. SystemJS builder won't bundle into html files.

[Hypercubed]

Here is one way to bootstrap a Web Component using SystemJS: http://plnkr.co/edit/w3L3FB

I used commonjs because I couldn't get traceur working in plunker.

[herkulano]

@Hypercubed That's a really cool example! :+1:

It also shows that developers can use whatever flavour they want and still comply with Polymer > Web Components (in the near future).

[Hypercubed]

Thanks. Here is one more demo before I call it a night: http://plnkr.co/edit/JJp6jp

Here I made a dumb systemjs plugin to do the HTML importing. This allows you to use mappings in the system.config.js (and all module loading goodness). I haven't tested it much so I don't know how robust it is.

I can imagine that it could be a lot smarter and process inline css and js for you. I'm surprised it hasn't been done before.

[Hypercubed]

FYI... https://github.com/Hypercubed/systemjs-plugin-html

[herkulano]

@Hypercubed Amazing! :+1:

[wilzbach]

@Hypercubed I like your approach - it is more secure than mine ;-)

Here a few remarks that I have by looking through your code:

Injecting html via systemjs-plugin-html

AFAIK using this approach is problematic when you bundle everything, as address won't be available them - that is the advantage of the systemjs-text plugin.

exports.fetch = function(load) {
  return importHref(load.address);
};

Injecting css via relative links

We probably can't bundle bar-char-component.css easily if we just use a relative link - or can we?

<link rel="import" type="css" href="bar-chart-component.css">

using systemjs plugins

I guess that we need to agree on all the custom plugins of systemjs that we want to use as for a development setup as we probably want to avoid requiring a bundled version in other packages - we would have hard time to filter out duplicates.

In summary

The biggest difference between our two approaches is that you (1) dynamically add <link> tags for the html and css on the document, whereas I (2) use innerHTML. Your approach (1) is probably faster and more secure, whereas (2) requiring ressources probably is better for a bundling process. I will have a look at best practices about the bundling process - see you soon :)

For convenience I have put my current version onto plunkr, too: http://plnkr.co/edit/CbZXA1?p=preview

[Hypercubed]

You are correct. Bundling wont work. The css plugin has it's own mechanism for bundling that I did not implement. The inline css and js are NOT processed at all by systemjs. At this point, as shown in my last plunker, I'm using the plugin as a promise generator. But maybe with some work...

[herkulano]

@greenify Also amazing! :+1:

[Hypercubed]

Found this... https://github.com/nevir/html-exports/tree/master/dist/sysjs-plugin

Haven't had a chance to look in detail.

[herkulano]

@greenify nice

I've forked @greenify's plunk and added some abstraction as a bio.js library to simplify the module loaders for developers: http://plnkr.co/edit/mEKZ2r?p=preview

Some problems of this solution:

• it's starting to get convoluted
• how does the end-user install local dependencies of a component?
• loss of the css scope by Polymer. @Hypercubed's version solves this and the css as a link tag inside the element doesn't seem to be a problem and it would also be easier to package for distribution.
• do we lose the template goodness of Polymer templates [needs testing]
• do we lose any Polymer goodness? [needs testing]

Are we planning to have several versions: ES5, ES6, HTML? Should we have a recommended way and then add flavors for other developers?

UPDATE: I've merged your ideas in this plunker http://plnkr.co/edit/g54ESw?p=preview

[Hypercubed]

For me, as a potential BioJS3 module consumer, I am happy using jspm/SystemJS/ES6. I think, however, npm/Browserify/CommonJS has more traction currently and may be more familiar to BioJS2 users and developers. The Web Components/Polymer way seams to be to forget modules, throw everything into bower and reference via relative path. That approach of course limits what modules you can consume. Would it even allow you to import an existing BioJS2 io module from npm?

That said, I think it is more than possible to create a BioJS3 template that provides all three. Write your code in ES6 and html, generate an ES5/UMD module, generate a HTML Import file that uses relative paths. Provide a package.json file for npm that points to the UMD/CommonJS module, include jspm additions that point to the ES6 module, and a bower.json file that points to the HTML import file.

I think that would be amazing.

[Hypercubed]

@herkulano I don't think we want to be using the System.import directly, this wont work for bundling.

I took your plunker and added and es6 module, a generated (by hand) es5 commonjs file, and a pure web component html import file. Only the ES5 module is being used but should show a way we can provide all three. http://plnkr.co/edit/gwnVq9

[Hypercubed]

Relevant discussion: http://www.codequora.com/questions/25005623/web-components-polymer-and-systemjs

Wondering if we need/want Polymer/HTML imports at all.

[wilzbach]

It is absolutely great to work with you on this :)

For me, as a potential BioJS3 module consumer, I am happy using jspm/SystemJS/ES6. I think, however, npm/Browserify/CommonJS has more traction currently and may be more familiar to BioJS2 users and developers.

Have you seen this great talk by @guybredford (the author of jspm) on how jspm can be used to manage {es6,es5} dependencies from {github,npm} - he also showcases a cool demo with web components!

how does the end-user install local dependencies of a component?

I do imagine two ways:

1. script-tags for the end-user: bundled (optionally minimized) version in /dist for independent inclusion
2. bower/npm components for a developer: unfortunately we can't expect everyone to use es6 and neither to use SystemJS as @Hypercubed pointed out

That said, I think it is more than possible to create a BioJS3 template that provides all three. Write your code in ES6 and html, generate an ES5/UMD module, generate a HTML Import file that uses relative paths. Provide a package.json file for npm that points to the UMD/CommonJS module, include jspm additions that point to the ES6 module, and a bower.json file that points to the HTML import file.

Wow :+1: - I am only a bit afraid that

• this might create redundancies as e.g. a developer would need to (a) update his dependencies for bower and npm and he might forget about one of the config files, (b) it would start to get confusing if the bower and npm packages don't share the same name - AFAIK only jspm provides a convenient way to map package names.
• how a dev will be able to include dependencies if he uses the html import way. One requires a relative path name to an html file and the other just a require with the package name

loss of the css scope by Polymer. @Hypercubed's version solves this and the css as a link tag inside the element doesn't seem to be a problem and it would also be easier to package for distribution. [...] moving the add html outside of the javascript file

(: , but if we decide to go for html imports we would need to find a way to run vulcanize on top of the systemjs bundle. That is why I personally would prefer if the JavaScript version would require the html and css -I need to find a better way than innerHTML tough ;-)

Wondering if we need/want HTML imports at all.

:+1: - wondering about the same!

[Hypercubed]

Yeah, keeping the version numbers in sync between bower.json and package.json is already a hassle. I imagine someone could write a tool that maps jspm package names down to bower relative paths. Maybe some fancy gulp scripts? I think I saw someone made a bower -> jspm upconverter... I'll have to find it again. I would guess jspm -> bower -> html imports would be easier than the reverse. Not sure if it is worth all the tooling. Vulcanizing a SystemJS bundle sounds better. I wonder what @guybedford thinks of all this.

[herkulano]

@Hypercubed looks great!

[herkulano]

Some findings on using JSPM with Polymer

I've added another branch, wanted to try jspm, so couldn't use plunker for this. https://github.com/herkulano/biojs3-webcomponent-example/tree/jspm-test

Polymer Elements
It works well as long as you don't use polymer elements.
When you want to add a polymer element, for instance, a simple paper-button element, because of the structure they have for elements and reliance on bower, you can't use them with jspm anymore.

From paper-button.html

<link rel="import" href="../polymer/polymer.html">
<link rel="import" href="../paper-material/paper-material.html">
<link rel="import" href="../paper-ripple/paper-ripple.html">
<link rel="import" href="../paper-behaviors/paper-button-behavior.html">

Bower
I managed to use bower with jspm with a jspm registry endpoint for bower

Some problems:

• When installing the paper-button the sub-dependencies were not resolved. Didn't spend much time with it though.
• Polymer element's HTML Imports structure rely on bower's flat structure which is very different from jspm's structure, so it breaks polymer elements.

[wilzbach]

Hey,

I used our work @herkulano & @Hypercubed to create a jspm example app that uses the following web components as dependencies:

• bio-element.
• bio-container

In particular bio-container depends on bio-element, other javascript dependencies like Polymer or lodash (as example) are also injected via the SystemJS loader. For simplicity I avoided ES6 code, but feel free to add it. Also the web component polyfill is now conditionally loaded (needs still some work)

Currently

• HTML is loaded via @Hypercubed html plugin
• CSS is loaded via an HTML Import inside the HTML

I guess we only

• have to figure out how we can bundle the app when using the HTML/CSS imports - maybe a combination with vulcanize?
• whether we actually want to (or can) support existing Polymer plugins
• one can depend on BioJS web components without jspm and SystemJS or whether that is important to us (in any case they will always be able to include the bundled version)

Feel free to modify the bio-element, bio-container and the example app.

Happy hacking!

[Hypercubed]

Hi @greenify... Looks amazing. Does this line https://github.com/biojs/bio-container/blob/master/main.html#L9 work? If someone imports main.html via HTML import the script tags are treated by the browser, not by SystemJS, therefore I think require will be undefined when ./bio-container.js is loaded. There is a work around of adding a global copy of System.amdRequire but this is AMD not CJS (see https://github.com/systemjs/systemjs/wiki/Module-Format-Support#requirejs-support).

[wilzbach]

Looks amazing. Does this line https://github.com/biojs/bio-container/blob/master/main.html#L9 work? If someone imports main.html via HTML import the script tags are treated by the browser, not by SystemJS, therefore I think require will be undefined when ./bio-container.js is loaded

Yes you are right (and I knew about this), but I couldn't think of a proper way around. Even if we use AMD - the normal Polymer setup doesn't include an AMD module loader like RequireJS - devs would have to manually configure the path :(

[herkulano]

@greenify awesome! :+1:

[Hypercubed]

~~@herkulano Actually I think using the jspm-bower-endpoint works fine. This sets up a bower_components directory and internal references work, the same as using just bower. The only trouble is you need to ensure that your app is not including Polymer twice. If it does you will get an error. This is working for me:~~

~~https://github.com/Hypercubed/biojs3-webcomponent-example/commit/a6d473bfda18116f8fcecb80531c4b19d6848fc7~~

[Hypercubed]

Sorry, I take that back. I had a bower_components directory leftover from previous tests.