emacs/admin/make-tarball.txt

Instructions to create pretest or release tarballs. -*- coding: utf-8 -*-
-- originally written by Gerd Moellmann, amended by Francesco Potortì
   with the initial help of Eli Zaretskii


Steps to take before starting on the first pretest in any release sequence:

0.  The release branch (e.g. emacs-28) should already have been made
    and you should use it for all that follows.  Diffs from this
    branch should be going to the emacs-diffs mailing list.

1.  Decide on versions of m4 and autoconf, and ensure you will
    have them available for the duration of the release process.

2.  Consider increasing the value of the variable
    'customize-changed-options-previous-release' in cus-edit.el to
    refer to a newer version of Emacs.  (This is now done when cutting
    the release branch, see admin/release-branch.txt.)
    Commit cus-edit.el if changed.

3.  Remove any old pretests from https://alpha.gnu.org/gnu/emacs/pretest.
    You can use 'gnupload --delete' (see below for more gnupload details).
    (We currently don't bother with this.)

General steps (for each step, check for possible errors):

1.   git pull     # fetch from the repository
     git status   # check for locally modified files

    Ensure that you have a clean, unmodified state.
    If you switched in-place from another branch to the release branch,
    there could be inappropriate generated ignored files left over.
    You might want to use "git status --ignored" to check for such files,
    or some form of "git clean -x".  It's probably simpler and safer to
    make a new working directory exclusively for the release branch.

    If the working directory has subdirectories created when making
    previous releases or pretests, remove those subdirectories, as the
    command which updates the ChangeLog file might attempt to recurse
    there and scan any ChangeLog.* files there.

    Make sure the tree is built, or at least configured.  That's
    because some of the commands below run Make, so they need
    Makefiles to be present.

    For Emacs 28, and as long as --with-native-compilation is not the
    default, the tree needs to be configured with native-compilation
    enabled, to ensure all the pertinent *.elc files will end up in
    the tarball.  Otherwise, the *.eln files might not build correctly
    on the user's system.

2.  Regenerate the etc/AUTHORS file:
      M-: (require 'authors) RET
      M-x authors RET

    (This first updates the current versioned ChangeLog.N)

    If this says "Problem updating ChangeLog", find the reason for the
    failure of the command it runs, viz.:

       make -C ROOT change-history-nocommit

    (where ROOT is the top-level directory where you run this).  It
    could be because there are uncommitted changes in ChangeLog.N, for
    example.  One possible way forward is to invoke "C-u M-x authors",
    which will skip updating the versioned ChangeLog.N file.

    After "M-x authors" finishes, if there is an "*Authors Errors*"
    buffer, address the issues.  If there was a ChangeLog typo, fix
    the relevant entry.  If a file was deleted or renamed, consider
    adding an appropriate entry to variables authors-ignored-files,
    authors-valid-file-names, or authors-renamed-files-alist in
    authors.el.

    If necessary, repeat 'C-u M-x authors' after making those changes.
    Save the "*Authors*" buffer as etc/AUTHORS.
    Check the diff looks reasonable.  Maybe add entries to
    authors-ambiguous-files or authors-aliases, and repeat.
    Commit any fixes to authors.el.

3.  Set the version number (M-x load-file RET admin/admin.el RET, then
    M-x set-version RET).  For a pretest, start at version .90.  After
    .99, use .990 (so that it sorts).  Commit the resulting changes
    as one, with nothing else included, and using a log message
    of the format "Bump Emacs version to ...", so that the commit can
    be skipped when merging branches (see admin/gitmerge.el).

    The final pretest should be a release candidate.
    Before a release candidate is made, the tasks listed in
    admin/release-process must be completed.

    Set the version number to that of the actual release (commit in
    one, as described above).  Pick a date about a week from now when
    you intend to make the release.  Use M-x add-release-logs to add
    entries to etc/HISTORY and the ChangeLog file.  It's best not to
    commit these files until the release is actually made.  Merge the
    entries from (unversioned) ChangeLog into the top of the current
    versioned ChangeLog.N and commit that along with etc/HISTORY.
    Then you can tag that commit as the release.

    Alternatively, you can commit and tag with the RC tag right away,
    and delay the final tagging until you actually decide to make a
    release and announce it.  The "git tag" command can tag a specific
    commit if you give it the SHA1 of that commit, even if additional
    commits have been pushed in the meantime.

    Name the tar file as emacs-XX.Y-rc1.tar.  If all goes well in the
    following week, you can simply rename the file and use it for the
    actual release.  If you need another release candidate, remember
    to adjust the ChangeLog and etc/HISTORY entries.

    If you need to change only a file(s) that cannot possibly affect
    the build (README, ChangeLog, NEWS, etc.) then rather than doing
    an entirely new build, it is better to unpack the existing
    tarfile, modify the file(s), and tar it back up again.

    Never replace an existing tarfile!  If you need to fix something,
    always upload it with a different name.

4.    autoreconf -i -I m4 --force
      make bootstrap

      make -C etc/refcards
      make -C etc/refcards clean

    If some of the etc/refcards, especially the non-English ones, fail
    to build, you probably need to install some TeX/LaTeX packages, in
    particular for foreign language support.  For more information,
    search for the string "refcard" in the file admin/release-process.

    (ru-refcard causes numerous "Underfull hbox" and "Overfull hbox"
    messages from TeX, but those seem to be harmless, as the result
    looks just fine.)

5.  Copy lisp/loaddefs.el to lisp/ldefs-boot.el.

    Commit ChangeLog.N, etc/AUTHORS, lisp/ldefs-boot.el, and the
    files changed by M-x set-version.  The easiest way of doing that
    is "C-x v d ROOT-DIR RET", then go to the first modified file,
    press 'M' to mark all modified files, and finally 'v' to commit
    them.  Make sure the commit log message mentions all the changes
    in all modified files, as by default 'v' doesn't necessarily do
    so.

    If someone else made a commit between step 1 and now,
    you need to repeat from step 4 onwards.  (You can commit the files
    from step 2 and 3 earlier to reduce the chance of this.)

6.   ./make-dist --snapshot --no-compress

    Check the contents of the new tar with admin/diff-tar-files
    against the previous release (if this is the first pretest) or the
    previous pretest.  If you did not make the previous pretest
    yourself, find it at <https://alpha.gnu.org/gnu/emacs/pretest>.
    Releases are of course at <https://ftp.gnu.org/pub/gnu/emacs/>.

     ./admin/diff-tar-files emacs-OLD.tar.gz emacs-NEW.tar.gz

    Alternatively:

     tar tJf emacs-OLD.tar.xz | sed -e 's,^[^/]*,,' | sort > old_tmp
     tar tJf emacs-NEW.tar.xz | sed -e 's,^[^/]*,,' | sort > new_tmp
     diff -u old_tmp new_tmp

    If this is the first pretest of a major release, just comparing
    with the previous release may overlook many new files.  You can try
    something like 'find . | sort' in a clean repository, and
    compare the results against the new tar contents.  Another
    alternative is using something like:

     tar cf - emacs-NEW | tar t -C /tmp | grep -Ev "\.(o|d)$" | sort

    Where emacs-NEW is the directory containing your clean repository.
    The output of this command might be easier to compare to the
    tarball than the one you get from find.

7.   tar -xf emacs-NEW.tar; cd emacs-NEW
     ./configure --prefix=/tmp/emacs && make check && make install

    Use 'script' or M-x compile to save the compilation log in
    compile-NEW.log and compare it against an old one.  The easiest way
    to do that is to visit the old log in Emacs, change the version
    number of the old Emacs to __, do the same with the new log and do
    M-x ediff.  Especially check that Info files aren't built, and that
    no autotools (autoconf etc) run.

8.  You can now tag the release/pretest and push it together with the
    last commit:

     cd EMACS_ROOT_DIR && git tag -a TAG -m "Emacs TAG"
     git push
     git push --tags

    Here TAG is emacs-XX.Y.ZZ for a pretest, emacs-XX.Y for a release.
    For a release, if you are producing a release candidate first, use
    emacs-XX.Y-rcN (N = 1, 2, ...) when you tar the RC, and add the
    actual release tag later, when the official release tarball is
    uploaded to ftp.gnu.org.  When adding a tag later, it is safer to
    use the SHA1 of the last commit which went into the release
    tarball, in case there were some intervening commits since then:

     git tag -a TAG -m "Emacs TAG" SHA1
     git push --tags

    In the past, we were not always consistent with the annotation
    (i.e. -m "Emacs TAG").  The preferred format is like this for a
    pretest, release candidate and final release:

     git tag -a emacs-28.0.90  -m "Emacs 28.0.90 pretest"
     git tag -a emacs-28.1-rc1 -m "Emacs 28.1 RC1"
     git tag -a emacs-28.1     -m "Emacs 28.1 release"

9. Decide what compression schemes to offer.
    For a release, at least gz and xz:
      gzip --best --no-name -c emacs-NEW.tar > emacs-NEW.tar.gz
      xz -c emacs-NEW.tar > emacs-NEW.tar.xz
    For pretests, just xz is probably fine (saves bandwidth).

    Now you should upload the files to the GNU ftp server.  In order to
    do that, you must be registered as an Emacs maintainer and have your
    GPG key acknowledged by the ftp people.  For instructions, see
    https://www.gnu.org/prep/maintain/html_node/Automated-Upload-Registration.html
    The simplest method to upload is to use the gnulib
    <https://www.gnu.org/s/gnulib/> script "build-aux/gnupload":

    For a pretest:
     gnupload [--user your@gpg.key.email] --to alpha.gnu.org:emacs/pretest \
       FILE.gz FILE.xz ...

    For a release:
     gnupload [--user your@gpg.key.email] --to ftp.gnu.org:emacs \
       FILE.gz FILE.xz ...

    You only need the --user part if you have multiple GPG keys and do
    not want to use the default.
    Obviously, if you do not have a fast uplink, be prepared for the
    upload to take a while.


    If you prefer to do it yourself rather than use gnupload:

    For each FILE, create a detached GPG binary signature and a
    clearsigned directive file like this:

     gpg -b FILE
     echo directory: emacs/pretest > FILE.directive      (for a pretest)
     echo directory: emacs > FILE.directive              (for a release)
     gpg --clearsign FILE.directive
    Upload by anonymous ftp to ftp://ftp-upload.gnu.org/ the files FILE,
    FILE.sig, FILE.directive.asc.
    For a release, place the files in the /incoming/ftp directory.
    For a pretest, place the files in /incoming/alpha instead, so that
    they appear on https://alpha.gnu.org/.

10. After five minutes, verify that the files are visible at
    https://alpha.gnu.org/gnu/emacs/pretest/ for a pretest, or
    https://ftp.gnu.org/gnu/emacs/ for a release.

    Download them and check the signatures.  Check they build.

11. Send an announcement to: emacs-devel, and bcc: info-gnu-emacs@gnu.org.
    For a pretest, also bcc: platform-testers@gnu.org.
    For a release, also bcc: info-gnu@gnu.org.
    (The reason for using bcc: is to make it less likely that people
    will followup on the wrong list.)
    See the info-gnu-emacs mailing list archives for the form
    of past announcements.  The first pretest announcement, and the
    release announcement, should have more detail.
    Use the emacs-devel topic 'emacs-announce'.  The best way to do
    this is to add a header "Keywords: emacs-announce" to your mail.
    (You can also put it in the Subject, but this is not as good
    because replies that invariably are not announcements also get
    sent out as if they were.)

    To create the included SHA1 and SHA256 checksums, run:

     sha1sum emacs-NEW.tar.xz
     sha256sum emacs-NEW.tar.xz

12. After a release, update the Emacs pages as described below.

13. Bump the Emacs version on the release branch.
    If the released version was XX.Y, use 'set-version' from
    admin/admin.el to bump the version on the release branch to
    XX.Y.50.  Commit the changes.

UPDATING THE EMACS WEB PAGES AFTER A RELEASE

As soon as possible after a release, the Emacs web pages at
https://www.gnu.org/software/emacs/ should be updated.  (See
admin/notes/www for general information.)

The pages to update are:

emacs.html (for a new major release, a more thorough update is needed)
history.html
add the new NEWS file as news/NEWS.xx.y
Copy new etc/MACHINES to MACHINES and CONTRIBUTE to CONTRIBUTE

For every new release, a banner is displayed on top of the emacs.html
page.  Uncomment and the release banner in emacs.html.  Keep it on the
page for about a month, then comment it again.  The new release banner
looks like this:

    <div class="release-banner">
	<div class="container">
	    <h2><em>Emacs 27.1 is out</em>, download it <a href="download.html">here</a>!</h2>
	</div>
    </div>

Regenerate the various manuals in manual/.
The scripts admin/make-manuals and admin/upload-manuals summarize the process.

If you have Texinfo installed locally, make-manuals might fail if it
cannot find epsf.tex.  In that case define in the environment

  TEXINPUTS=:/path/to/texinfo-tree/doc

where /path/to/texinfo-tree is the absolute file name of the top-level
directory where you have the Texinfo source tree.  Then re-run
make-manuals.

Browsing <https://web.cvs.savannah.gnu.org/viewvc/?root=emacs> is one
way to check for any files that still need updating.