* lisp/url/url-util.el (url-domain): Don't talk DNS to determine
the domain, because this is slow.
* test/lisp/url/url-util-tests.el (url-domain-tests): Add tests
for `url-domain'.
* lisp/net/gnutls.el (open-gnutls-stream): IDNA-encode hostnames
before passing them on to gnutls for verification.
* lisp/net/network-stream.el (network-stream-open-starttls): Ditto.
* lisp/url/url-http.el (url-http--get-referer): Be IDNA-aware.
(url-http-create-request): Don't de-Unicodify host names, because
they may be IDNA names (that are later encoded).
* lisp/url/url-util.el (url-domain): Be IDNA-aware when doing
domain name computations.
* doc/misc/url.texi (Customization): Describe the new user
option url-lastloc-privacy-level.
* lisp/net/eww.el (eww-render): Set url-current-lastloc to the
url we are rendering, to get the referer header right on
subsequent requests.
* lisp/url/url-http.el (url-http--get-referer): New function
to determine which referer to send, if any, considering the
users privacy settings and the target url we are visiting.
(url-http-referer): New variable keeping track of the referer
computed by url-http--get-referer
(url-http-create-request): Use url-http-referer instead of the
optional argument to set up the referer header. Leave
checking of privacy settings to url-http--get-referer.
(url-http): Set up url-http-referer by using
url-http--get-referer.
* lisp/url/url-queue.el (url-queue): New struct member
context-buffer for keeping track of the context a queued job
started from.
(url-queue-retrieve): Store the current buffer in the queue
object.
(url-queue-start-retrieve): Make sure url-retrieve is called
in the context of the original buffer, if available.
* lisp/url/url-util.el (url-domain): New function to determine
the domain of a given URL.
* lisp/url/url-vars.el (url-current-lastloc): New variable to
keep track of the desired "last location" (referer header).
(url-lastloc-privacy-level): New custom setting for more
fine-grained control over how lastloc (referer) is sent to
servers (Bug#27012).
Most of this change is to boilerplate commentary such as license URLs.
This change was prompted by ftp://ftp.gnu.org's going-away party,
planned for November. Change these FTP URLs to https://ftp.gnu.org
instead. Make similar changes for URLs to other organizations moving
away from FTP. Also, change HTTP to HTTPS for URLs to gnu.org and
fsf.org when this works, as this will further help defend against
man-in-the-middle attacks (for this part I omitted the MS-DOS and
MS-Windows sources and the test tarballs to keep the workload down).
HTTPS is not fully working to lists.gnu.org so I left those URLs alone
for now.
* lisp/url/url-util.el (url-encode-url): Simplify.
url-generic-parse-url copes with multibyte strings just fine
(https://debbugs.gnu.org/cgi/bugreport.cgi?bug=24117#185).
* test/lisp/url/url-parse-tests.el
(url-generic-parse-url/multibyte-host-and-path): New test.
* lisp/url/url-http.el (url-http-create-request): IDNA-encode
the Host: header.
* lisp/url/url-util.el (url-encode-url): Don't hex-encode
domain names, but leave them as UTF-8, so that they can be
IDNA-encoded later when contacting the host.
Problem reported by Artur Malabarba in:
http://lists.gnu.org/archive/html/emacs-devel/2015-11/msg01513.html
Most of these fixes are to documentation; many involve fixing
longstanding quoting glitches that are independent of the
recent substitute-command-keys changes. The changes to code are:
* lisp/cedet/mode-local.el (mode-local-augment-function-help)
(describe-mode-local-overload):
Substitute docstrings before displaying them.
* lisp/emacs-lisp/cl-macs.el (cl--transform-lambda):
Quote the generated docstring for later substitution.
Mostly these fixes prevent the transliteration of apostrophes
that should stay apostrophes. Also, prefer curved quotes in
Bahá’í proper names, as that’s the preferred Bahá’í style and
these names are chock-full of non-ASCII characters anyway.
* lisp/emacs-lisp/eieio-core.el (eieio-defclass-autoload)
(eieio-defclass-internal):
* lisp/emacs-lisp/eieio.el (defclass):
* lisp/hi-lock.el (hi-lock-mode):
Don’t transliterate Lisp apostrophes when generating a
doc string or diagnostic.
* lisp/international/mule-diag.el (list-coding-systems-1):
* lisp/international/ogonek.el (ogonek-jak, ogonek-how):
* lisp/mail/sendmail.el (sendmail-query-user-about-smtp):
* lisp/vc/ediff-mult.el (ediff-redraw-registry-buffer):
* lisp/vc/ediff-ptch.el (ediff-fixup-patch-map):
Substitute quotes before putting them in the help buffer.
* doc/misc/org.texi (Dynamic blocks):
* lisp/allout-widgets.el (allout-widgets-hook-error-handler):
* lisp/calendar/appt.el (appt-display-message):
* lisp/calendar/icalendar.el (icalendar--convert-float-to-ical):
* lisp/calendar/timeclock.el (timeclock-in, timeclock-when-to-leave)
(timeclock-last-period, timeclock-day-base):
* lisp/eshell/em-ls.el (eshell-ls-file):
* lisp/eshell/esh-util.el (eshell-parse-ange-ls):
* lisp/generic-x.el (named-database-print-serial):
* lisp/net/newst-backend.el (newsticker--get-news-by-url-callback)
(newsticker-get-news, newsticker--sentinel-work)
(newsticker--image-get, newsticker--image-sentinel):
* lisp/net/tramp-sh.el (tramp-get-remote-touch):
* lisp/progmodes/opascal.el (opascal-debug-log):
* lisp/textmodes/remember.el (remember-mail-date)
(remember-store-in-files):
* lisp/vc/vc-annotate.el (vc-annotate-display-autoscale)
(vc-default-annotate-current-time):
* lisp/vc/vc-bzr.el (vc-bzr-shelve-snapshot):
* lisp/vc/vc-cvs.el (vc-cvs-annotate-current-time):
* lisp/vc/vc-rcs.el (vc-rcs-annotate-current-time):
* lisp/url/url-util.el (url-get-normalized-date):
* lisp/erc/erc-backend.el (TOPIC):
* lisp/gnus/gnus-delay.el (gnus-delay-article):
* lisp/gnus/gnus-sum.el (gnus-summary-read-document):
* lisp/gnus/gnus-util.el (gnus-seconds-today, gnus-seconds-month):
* lisp/gnus/message.el (message-make-expires-date):
* lisp/org/org-archive.el (org-archive-subtree)
(org-archive-to-archive-sibling):
* lisp/org/org-clock.el (org-resolve-clocks, org-clock-get-sum-start)
(org-clock-special-range):
* lisp/org/org-timer.el (org-timer-seconds):
* lisp/org/org.el (org-read-date-analyze, org-get-cursor-date):
* lisp/org/ox-html.el (org-html-format-spec):
* lisp/org/ox-icalendar.el (org-icalendar--vtodo):
Omit unnecessary call to current-time.
* lisp/calendar/time-date.el (time-to-seconds) [!float-time]:
* lisp/calendar/timeclock.el (timeclock-time-to-date):
* lisp/vc/vc-annotate.el (vc-annotate-convert-time):
Use current time if arg is nil, to be compatible with float-time.
(time-date--day-in-year): New function, with most of the guts of
the old time-to-day-in-year.
(time-to-day-in-year): Use it.
(time-to-days): Use it, to avoid decoding the same time stamp twice.
* lisp/calendar/timeclock.el (timeclock-update-mode-line):
* lisp/cedet/srecode/args.el (srecode-semantic-handle-:time):
* lisp/gnus/gnus-util.el (gnus-seconds-year):
* lisp/org/org.el (org-get-cursor-date):
Don't call current-time twice to get the current time stamp,
as this can lead to inconsistent results.
* lisp/completion.el (cmpl-hours-since-origin):
* lisp/erc/erc.el (erc-emacs-time-to-erc-time):
* lisp/ido.el (ido-time-stamp):
* lisp/vc/vc-annotate.el (vc-annotate-convert-time):
Simplify by using float-time.
* lisp/completion.el (save-completions-to-file):
* lisp/url/url-cache.el (url-cache-prune-cache):
Rename local var to avoid confusion.
* lisp/gnus/gnus-util.el (gnus-float-time):
* lisp/net/rcirc.el (rcirc-float-time):
* lisp/org/org-compat.el (org-float-time):
Simplify to an alias because time-to-seconds now behaves like float-time
with respect to nil arg.
* lisp/subr.el (progress-reporter-do-update):
Don't call float-time unless needed.
* lisp/erc/erc.el (erc-current-time): Simplify by using erc-emacs-time-to-erc-time.
* lisp/org/org-clock.el (org-clock-get-table-data): Omit unnecessary, lossy
conversion from floating point to Emacs time and back.
(org-resolve-clocks): Prefer two-argument floor.
* url/url-util.el (url-build-query-string): New function.
(url-parse-query-string): Allow that '=' is not required and split
URL parameters on ';', not just '&'.
* ffap.el (ffap-url-unwrap-local): Make it work right.
Use url-generic-parse-url, and handle host names and Windows
filenames properly.
(ffap-url-unwrap-remote): Use url-generic-parse-url.
(ffap-url-unwrap-remote): Accept list values, specifying a list of
URL schemes to work on.
(ffap--toggle-read-only): New function.
(ffap-read-only, ffap-read-only-other-window)
(ffap-read-only-other-frame): Use it.
(ffap-fixup-url): Don't check ffap-ftp-regexp, since it is not
necessary for ffap-url-unwrap-remote.
* url-parse.el (url-path-and-query, url-port-if-non-default): New
functions.
(url-generic-parse-url): Don't set the portspec slot if it is not
specified; that is what `url-port' is for.
(url-port): Only require the scheme to be specified to call
url-scheme-get-property.
* url-util.el (url-encode-url): Use url-path-and-query.
* url-vars.el (url-mime-charset-string): Load mm-util lazily.
Fixes: debbugs:9131
Fix 2012-04-10 change to url.el.
* url-http.el (url-http-create-request): Ignore obsolete
attributes slot of url-object.
* url-parse.el: Improve RFC 3986 conformance.
(url-generic-parse-url): Do not populate the ATTRIBUTES slot,
since this is not reliable for general RFC 3986 URIs. Keep the
whole path and query inside the FILENAME slot. Improve docstring.
(url-recreate-url-attributes): Mark as obsolete.
(url-recreate-url): Handle missing scheme and userinfo.
* url-util.el (url-encode-url): New function for URL quoting.
(url-encoding-table, url-host-allowed-chars)
(url-path-allowed-chars): New constants.
(url--allowed-chars): New helper function.
(url-hexify-string): Use them.
* url-vars.el (url-nonrelative-link): Make the regexp stricter.
* url.el (url-retrieve-internal): Use url-encode-url.
Fixes: debbugs:7017
