Keeping Mozilla and Adobe nanojit versions in sync.
Hi,
This is a long email. Sorry, it needs to be a bit long.
Background.
-----------
Some of you may be aware, if you've been following nanojit development,
that Adobe and Mozilla are diverged. There are two copies of nanojit
embedded in independently-evolving repository histories: the Tamarin
history and the Tracemonkey history.
This is a historical accident, not a matter of policy. We had every
intention to be doing high-frequency / fine-granularity merges, simply a
lack of knowhow about how to do so *easily* and a lack of time to pursue
the correct technique. We've been trading high-priority patches for the
past nine months manually, and gradually diverging on other matters.
We have enough time now to pay attention to the issue again and try to
get it right. Previously -- as long ago as last year -- Edwin and I
tried forming a nanojit-only history with two repositories: one for
Adobe and one for Mozilla. Our hope, at the time, was that these two
repositories would hold tip revisions that were identical to the nanojit
subdirectories embedded in the Tamarin and Tracemonkey repositories,
respectively. We failed in that attempt for two reasons:
1. We did not start from merged repositories, but were playing
catch-up with the ongoing development in both Tamarin and
Tracemonkey repositories.
2. We were manually transporting patches and applying them by hand
or munging them via handmade scripts from one repository to another.
These two factors made it too awkward to regularly merge and we quickly
fell behind. I'm hoping we can find a way around these failure modes
presently, and have spent a little while hunting around for
alternatives. I think I have a way to do it, but it requires a little
discussion and a fair amount of buy-in from the development groups
involved. And thankfully, nobody else.
Proposal.
---------
I'm proposing we pick a flag day both the Tamarin team and the
Tracemonkey team can agree on and simultaneously do the following:
1. Merge. Really really merge. Get everyone on the same IRC channel,
stop feature work, and set about resolving (with bug references,
regression testing and discussion) every remaining difference
between the two repositories, until we're identical. There is, as
far as I know, no fundamental disagreement on technical choices
between the teams, just a bunch of drift.
2. Re-seed the nanojit-only history Edwin and I attempted before,
but starting with this merged state.
3. Using 'hg convert', along with a splicemap and a filemap, splice
the nanojit-only history onto the merged (therefore identical) tips
of both the Tracemonkey and Tamarin repositories.
4. Add incoming hooks to the Tracemonkey and Tamarin repositories
(and local precommit hooks to all the local developer machines on
each team) *prohibiting* local checkins to the nanojit/
subdirectory. All changes to nanojit henceforth get made on the
nanojit-only history and copied to the Tracemonkey and Tamarin
repositories with 'hg convert'.
I've tested this sort of setup, and it can be made to work. It's not
even terribly hard. You wind up with a splicemap file like:
<0th-revision-in-nanojit> <splicepoint-revision-in-Tracemonkey>
and a filemap file like:
rename . js/src/nanojit
and you wind up making day-to-day nanojit changes like so:
~/src/nanojit-only$ hg commit --message="Some nanojit bug."
...
~/src/nanojit-only$ cd ..
~/src$ hg convert \
--source-type=hg --dest-type=hg \
--filemap=filemap --splicemap=splicemap \
--config convert.hg.saverev=True \
nanojit-only tracemonkey
...
~/src$ cd tracemonkey/js/src/obj
~/src/tracemonkey/js/src/obj$ hg up
...
~/src/tracemonkey/js/src/obj$ make check
...
~/src/tracemonkey/js/src/obj$ hg push
(Replace Tracemonkey-specific paths with Tamarin paths in Tamarin's
case)
I'll grant, this is not *totally* simple, but it's simple enough that it
can be internalized and hooked to reject incorrect actions. And
nobody outside our two development groups needs to care that this is
happening; it doesn't require any new machinery.
Alternatives.
-------------
Every alternative I looked at involves adding repository metadata or
non-hg tools, which means convincing *all* hg users in the Adobe and
Mozilla development communities to install new code and learn to use it,
just to carry on doing what they're doing today. This is a steep cost I
wanted to avoid. The proposal above only affects a handful of
people. I'm happy to be pointed to more that this isn't true for. I just
discovered that 'convert' does splicing, file-renaming and hg-to-hg
operation recently. Those are the important bits you need to automate
the patch-flow.
Open questions.
---------------
1. Does the above proposal make sense, seem reasonable, suit
everyone's preferences, etc. etc.? Do you hate it? Can you think of
any general ways to improve it?
2. The 'hg convert' command does keep a mapping file associating
imported and local revision IDs. It may make more sense to check
this file in and commit it after each import, such that we all work
from the same associations; or it may make more sense to keep a
local copy of it elsewhere. I'm not sure. I believe it can be
reconstructed relatively easily in any case (particularly if
original-revision IDs get projected into the synthesized revisions,
as in my example). Any thoughts?
3. The proposal above involves sticking with *two* nanojit-only
repositories. This still permits the two repositories (within the
shared nanojit-only history) to diverge, and requires periodic pulls
from one to the other to absorb one another's work. And it will
necessarily involve a history with two evolving heads, two
"lineages". Would it makes more sense to try having only *one*
nanojit-only repository, that we insist remains functional for
*both* embeddings, both Tracemonkey and Tamarin?
4. If it's agreeable, what's a good flag-day to do it on?
Thanks for the read. Any further thoughts you have on the matter would
be great. Crossposting this to tamarin-devel and
mozilla.dev.tech.js-engine.
-Graydon
RSS Feed