LernaJS

Created: 2017-10-10 10:18:21 -0700 Modified: 2019-08-12 09:17:59 -0700

Basics (reference)

LernaJS is “a tool for managing JavaScript projects with multiple packages.” It should provide a much better workflow than having to produce a new repository and publish modules every time you want to share functionality between related projects.

To target an individual package (e.g. for installation), do something like this:

npx lerna bootstrap —scope @botland/client

This is useful for when you want to reinstall everything as though you were starting from scratch, but only for a certain package.

Getting started (making a Lerna “repo”)

I had tried lerna-wizard, but it isn’t very popular and I ran into an issue quickly.

In testing this, I did these steps:

lerna init —independent
1. Independent Mode (reference) makes it so that you can have individual version numbers for the “sub” projects.
Commit to the repo that gets created
1. I added “lerna-debug.log” to my .gitignore so that errors wouldn’t get checked in.
lerna import <path to external repository>
1. WARNING: DO NOT EVEN RUN THIS COMMAND UNLESS YOU HAVE CHECKED THE STATUS OF THIS ISSUE (see troubleshooting section of this note)
2. Note: the path is on your filesystem still, but it’s imported locally to the brand new repo that you committed to in the last step. This is really helpful since by the end of these steps, you’ll be able to test everything in your Lerna repo without having modified the source repos. That way, if there’s a problem, you can just wipe out the entire Lerna repo if you want.
3. This will ask to merge git history so that your commits remain intact. It doesn’t do an “npm install” or anything though.
4. Note: this will import without interleaving Git history. For example, if you import RepoA and then RepoB, you’ll get all of RepoA’s history first regardless of what dates that may contain, then all of RepoB’s history second regardless of whether it came before RepoA.
lerna bootstrap (consider doing “lerna clean” beforehand to wipe out all node_modules folders)
1. This doesn’t take args. It will “npm install” everything, link everything, run prepublish, and a bunch of other goodness that you need to do probably after every import.

Cloning a Lerna “repo”

This is what the “bootstrap” command is for: it installs all package.json files and links them together.

lerna bootstrap

Adding a brand new module to an existing Lerna repo

Notes:

This assumes you’ve set up the repo using “lerna init”, “import”, “bootstrap”, etc. as shown in the “Getting started” section.
The module that you’re adding needs to have a package.json in it, otherwise it can’t be published. If you don’t want it to be published (i.e. it won’t be usable by any other Lerna packages), then you can put “private”: true in the package.json (the step to have a package.json is “npm init”). You can still technically do “lerna link” to use a private module in your other Lerna modules, but it’s more tedious than just publishing your would-be-private module. I tend to just make even test packages “public” and publish them manually at first via “npm publish —registry <name of my private registry>” (note: you only need —registry in the case that you don’t prefix your modules with “@name_of_scope”.

Make a new folder in “packages”
“npm init” (feel free to put “-y” if you want so that you don’t have to answer questions)
1. Don’t forget to put “@namespace/” before the name if you don’t want it to be publicly published
2. Don’t forget to put “private”: true if you don’t want it to be published. Chances are that the only time you want this to be completely private is if it’s for test code.
Add a “.gitignore” to this folder for node_modules if you haven’t already blocked it recursively.
1. Remember: .gitignore is used for .npmignore if you didn’t define a .npmignore. They’re both structured the same way (e.g. just put “node_modules” in there if you want to ignore everything under node_modules).
Import your module from any other modules that may need it (even if you haven’t finished writing it yet, that way all of these steps can be compact and done before you get to the code).
1. The way I did this so that I didn’t have to publish first was to just manually type into package.json with version 1.0.0 (which is what “npm init” gives you by default).
Run “lerna link” at the end from the root

Adding an existing module to an existing Lerna repo

You can mostly just copy/paste the files from the existing module into the repo, but you need to reinstall all dependencies that should be linked, then run “lerna link” at the end.

Prepublished modules

Every once in a while, you’ll have something like this: a module that you write in ES6 that gets transpiled to ES5 before being published. This is typically done via a “prepublish” step. Lerna does not know to automatically run these, so a workaround could be something like this (see bolded statement):

"scripts": {
    "clean": "rimraf lib && mkdir lib",
    "babel": "node node_modules/babel-cli/bin/babel.js src --out-dir lib",
    "babel:watch": "npm run babel -- --watch",
    "prepublish": "npm run-script clean && npm run-script babel"
  },

Then simply run “npm run babel:watch” from the directory whose module needs to be prepublished.

Notes:

“npm run” is an alias for “npm run-script”
The ”—” to separate arguments to the script is not needed by Yarn

Publishing

First, run “git status” to make sure you have no unmodified files. I had modified “package.json” by adding a dependency, and my first “lerna publish” didn’t actually detect that the file already had changes, so it happily bumped the version number and committed without warning me.

After publishing for the first time, “lerna updated” will be used so that you only publish what changed.

“—exact” is useful so that you’re using exact dependencies instead of ranges when you update the users of those dependencies.

Testing local changes

I needed to use a fork that someone had made in a pull request, so I got the URL of the fork by going to that person’s GitHub and finding their Lerna fork. I cloned the repo, switched to their fork branch (in this case it was “git checkout improve-import-edge-cases”), did “npm install”, “npm build”, “npm link” (so that it was the global “Lerna”), and I was good to go.

Troubleshooting

error code ENEEDAUTH

error need auth auth required for publishing

error need auth You need to authorize this machine using npm adduser

If this happens, it’s probable that you’re not connecting to the registry that you think you are, or that you’re not connecting with the user that you think you should be. This happened to me when I was trying to publish the @botland modules, but “npm config get registry” showed the main NPM registry. I needed to do:

npm adduser —registry=<my url>
lerna publish —registry <my url>

Version numbers aren’t being bumped for private packages

The problem that I was having is that I noticed that in test_shared (a private module), that any other modules that were supposed to be linked were out-of-date. This meant that the test code wasn’t testing up-to-date development code, and “lerna publish” wouldn’t fix this.

I’m still not totally sure what caused this. To fix it though, I had to do this process:

Manually update package.json for test_Shared to point to the up-to-date versions of each Lerna dependency
1. Note: to figure out where to make these changes, you can run this ripgrep command:

rg —maxdepth 2 private

Run “lerna link”

Note: by default, “lerna version” will put carets before your version numbers. This is dictated by “save-prefix”, so you can set it to an empty string if you want.

I got an error that looked like this

lerna ERR! import Failed to apply commit b20444c7.

lerna ERR! import Error: Command failed: git am -3

lerna ERR! import error: Failed to merge in the changes.

lerna ERR! import Applying: * Changed init_database.js so that it can be called from Ansible and correctly report when it fails. To accommodate this, I needed to add an unhandledRejection handler and also change the knex calls such that there wouldn’t be errors if the database or the user already existed.

lerna ERR! import Using index info to reconstruct a base tree…

lerna ERR! import M packages/bot-land/central_server/database/init_database.js

lerna ERR! import Falling back to patching base and 3-way merge…

lerna ERR! import Auto-merging packages/bot-land/central_server/database/init_database.js

lerna ERR! import CONFLICT (content): Merge conflict in packages/bot-land/central_server/database/init_database.js

lerna ERR! import Patch failed at 0001 * Changed init_database.js so that it can be called from Ansible and correctly report when it fails. To accommodate this, I needed to add an unhandledRejection handler and also change the knex calls such that there wouldn’t be errors if the database or the user already existed.

lerna ERR! import The copy of the patch that failed is found in: .git/rebase-apply/patch

lerna ERR! import When you have resolved this problem, run “git am —continue”.

lerna ERR! import If you prefer to skip this patch, run “git am —skip” instead.

lerna ERR! import To restore the original branch and stop patching, run “git am —abort”.

lerna ERR! import

lerna ERR! import Rolling back to previous HEAD (commit 8ab9f82d58b5daefefdd780ece869372e3e78dd3).

lerna ERR! import You may try with —flatten to import flat history.

lerna ERR! execute callback with error

(node:16948) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 3): TypeError: Cannot read property ‘split’ of undefined

I fixed this by specifying the “flatten” argument (reference).

Git is reporting two distinct items despite having the same path

These repro steps caused me to run into this existing issue:

mkdir test_git_paths4
cd test_git_paths4
lerna init
git commit -m “init”
lerna import D:CodeBotLandutil
1. This is just some arbitrary package that happens to have an “index.js” as shown below

Bug: Lerna’s first commit looks like this (notice the backslash):

I worked around this as described in my comment on the issue itself by adding a line of code that replaces the backslashes with a forward slash. This fix gets broken every time Lerna gets updated until they fix the issue properly.

Can’t see history before the Lerna changes in Bot Land

This is really just a generic Git issue where you need to see the history of a file that’s been renamed (or moved):

Ambiguous argument on publish

I tried doing “lerna version” and got the error below:

$ .node_modules\binlerna.cmd version

lerna notice cli v3.1.4

lerna info versioning independent

lerna ERR! Error: Command failed: git rev-list —left-right —count origin/master…master

lerna ERR! fatal: ambiguous argument ‘origin/master…master’: unknown revision or path not in the working tree.

lerna ERR! Use ’—’ to separate paths from revisions, like this:

lerna ERR! ‘git <command> [<revision>…] — [<file>…]’

I solved this by just pushing the repo and rerunning the command based on what I read here.

Can’t install private package

Packages with “private”: true cannot be installed because they’re never published. This is fine as long as “lerna link” was run, as it will symlink the package and not try installing it.

When I hit this with Bot Land after HiDeoo’s first pull request, I did the following:

lerna clean

lerna bootstrap

Maximum call stack size exceeded

If you see something like this:

Unhandled rejection RangeError: Maximum call stack size exceeded

at RegExp.test (<anonymous>)

at C:Usersagd13_000AppDataRoamingnpmnode_modulesnpmnode_modulesaprobaindex.js:38:16

at Array.forEach (<anonymous>)

at module.exports (C:Usersagd13_000AppDataRoamingnpmnode_modulesnpmnode_modulesaprobaindex.js:33:11)

at flatNameFromTree (C:Usersagd13_000AppDataRoamingnpmnode_modulesnpmlibinstallflatten-tree.js:37:3)

at flatNameFromTree (C:Usersagd13_000AppDataRoamingnpmnode_modulesnpmlibinstallflatten-tree.js:39:14)

npm ERR! cb() never called!

Then you probably have to wipe out node_modules entirely (rm -rf ./node_modules) and reinstall everything.

Note that you only may have to do this in the directory where it’s complaining.

eslint cannot be found (or a node module in general)

Suppose you have a script in a package that tries using a node_module installed for Lerna, e.g. eslint. At least on Windows, you can’t just go into the package and do “npm run lint” because it’s not going to be able to find eslint since it isn’t in node_modules.

I don’t have a great solution for this yet, but this command might work:

npx lerna exec —scope @botland/client — npm run lint

Alternatively, “lerna run —parallel lint” from the root will definitely work

Errors when unpublishing or force-publishing (reference)

npm ERR! code E404

npm ERR! 404 no such package available : 5-6822c29089a74391

npm ERR! 404

npm ERR! 404 ‘5-6822c29089a74391’ is not in the npm registry.

npm ERR! 404 You should bug the author to publish it (or use the name yourself!)