The current version of HTTP/1.1 (RFC 7231) no longer requires
confirmation on 301, 302 or 307 status codes, therefore we do not have
to ignore redirects for other requests than GET and HEAD.
* lisp/url/url-http.el (url-http-parse-headers): Do not ignore 301, 302
and 307 redirects for other requests than GET and HEAD.
* lisp/net/gnutls.el (open-gnutls-stream): Compute the
gnutls-boot parameters and pass them to the process object.
(gnutls-negotiate): New parameter :return-keywords that won't
connect to anything, just compute the keywords.
* lisp/url/url-http.el (url-http): Revert async TLS sentinel
hack, which is no longer necessary.
* src/gnutls.c (Fgnutls_asynchronous_parameters): Rename from
gnutls-mark-process.
* src/process.c (connect_network_socket): If we're connecting to
an asynchronous TLS socket, complete the GnuTLS boot sequence here.
* src/process.h: New parameter gnutls_async_parameters.
* doc/misc/emacs-gnutls.texi (Help For Developers): Mention
the nowait parameter.
* lisp/net/gnutls.el (open-gnutls-stream): Allow asynchronous
connections with the new nowait parameter.
* lisp/net/network-stream.el (network-stream-open-tls): Pass
on :nowait to open-gnutls-stream.
* lisp/url/url-http.el (url-http): Don't overwrite the
sentinel created by open-gnutls-stream.
* src/gnutls.c (Fgnutls_mark_process): New function.
* src/process.c (send_process): Don't write to GnuTLS sockets that
haven't been initialised yed.
* src/process.h: New slot gnutls_wait_p.
* lisp/url/url-http.el (url-http-create-request): IDNA-encode
the Host: header.
* lisp/url/url-util.el (url-encode-url): Don't hex-encode
domain names, but leave them as UTF-8, so that they can be
IDNA-encoded later when contacting the host.
* test/lisp/url/url-parse-tests.el: Add tests covering url-generic-parse-url.
* test/lisp/url/url-expand-tests.el: Add tests covering url-expand-file-name.
* lisp/url/url-parse.el (url-generic-parse-url): Keep empty fragment
information in URL-struct.
* lisp/url/url-parse.el (url-path-and-query): Do not artificially turn empty
path and query into nil path and query, respectively.
* lisp/url/url-expand.el (url-expander-remove-relative-links): Do not turn
empty path into an absolute ("/") path.
* lisp/url/url-expand.el (url-expand-file-name): Properly resolve
fragment-only URIs. Do not just return them unchanged.
* lisp/url/url-expand.el (url-default-expander): An empty path in the relative
reference URI should not drop the last segment.
Backport:
(cherry picked from commit b792ecea17)
* lisp/url/url-http.el (url-user-agent): Move from here...
* lisp/url/url-vars.el (url-user-agent): ...to here. This is to
keep all the URL defcustoms in one place, and also have it defined
whenever the URL library is loaded.
* doc/misc/url.texi (Customization): Document 'url-user-agent'.
* test/lisp/url/url-parse-tests.el: Add tests covering url-generic-parse-url.
* test/lisp/url/url-expand-tests.el: Add tests covering url-expand-file-name.
* lisp/url/url-parse.el (url-generic-parse-url): Keep empty fragment
information in URL-struct.
* lisp/url/url-parse.el (url-path-and-query): Do not artificially turn empty
path and query into nil path and query, respectively.
* lisp/url/url-expand.el (url-expander-remove-relative-links): Do not turn
empty path into an absolute ("/") path.
* lisp/url/url-expand.el (url-expand-file-name): Properly resolve
fragment-only URIs. Do not just return them unchanged.
* lisp/url/url-expand.el (url-default-expander): An empty path in the relative
reference URI should not drop the last segment.
* lisp/url/url-handlers.el: Add autoload cookie so that
`package-list-packages' doesn't bug out (bug#21927) (tiny change)
Backport:
(cherry picked from commit 7a7b5b492f)
* lisp/url/url-handlers.el (url-insert-file-contents): Don't
signal an error if VISIT is non-nil, to more faithfully emulate
the behavior of 'insert-file-contents'. (Bug#22160)
Problem reported by Artur Malabarba in:
http://lists.gnu.org/archive/html/emacs-devel/2015-11/msg01513.html
Most of these fixes are to documentation; many involve fixing
longstanding quoting glitches that are independent of the
recent substitute-command-keys changes. The changes to code are:
* lisp/cedet/mode-local.el (mode-local-augment-function-help)
(describe-mode-local-overload):
Substitute docstrings before displaying them.
* lisp/emacs-lisp/cl-macs.el (cl--transform-lambda):
Quote the generated docstring for later substitution.
(package--with-response-buffer): Use `url-insert-buffer-contents'.
The previous code had some issues with decoding. Refactoring that
function allows us to use the decoding from url-handlers while still
treating both sync and async requests the same.
* lisp/url/url-handlers.el (url-insert-file-contents): Move some code to
`url-insert-buffer-contents'.
(url-insert-buffer-contents): New function
* lisp/url/url-handlers.el (url-insert-file-contents): Move some code to
`url-insert-buffer-contents'.
(url-insert-buffer-contents): New function
(package--with-response-buffer): Use `url-insert-buffer-contents'.
The previous code had some issues with decoding. Refactoring that
function allows us to use the decoding from url-handlers while still
treating both sync and async requests the same.
This patch should not change behavior. It typically omits backslashes
where they are redundant (e.g., in the string literal "^\$").
In a few places, insert backslashes where they make regular
expressions clearer: e.g., replace "^\*" (equivalent to "^*") with
"^\\*", which has the same effect as a regular expression.
Also, use ‘\ %’ instead of ‘\%’ when avoiding confusion with SCCS IDs,
and similarly use ‘\ $’ instead of ‘\$’ when avoiding confusion with
RCS IDs, as that makes it clearer that the backslash is intended.
Mostly these fixes prevent the transliteration of apostrophes
that should stay apostrophes. Also, prefer curved quotes in
Bahá’í proper names, as that’s the preferred Bahá’í style and
these names are chock-full of non-ASCII characters anyway.
* lisp/emacs-lisp/eieio-core.el (eieio-defclass-autoload)
(eieio-defclass-internal):
* lisp/emacs-lisp/eieio.el (defclass):
* lisp/hi-lock.el (hi-lock-mode):
Don’t transliterate Lisp apostrophes when generating a
doc string or diagnostic.
* lisp/international/mule-diag.el (list-coding-systems-1):
* lisp/international/ogonek.el (ogonek-jak, ogonek-how):
* lisp/mail/sendmail.el (sendmail-query-user-about-smtp):
* lisp/vc/ediff-mult.el (ediff-redraw-registry-buffer):
* lisp/vc/ediff-ptch.el (ediff-fixup-patch-map):
Substitute quotes before putting them in the help buffer.
Fixes: debbugs:20010
* lisp/url/url-handlers.el (url-insert-file-contents): Call
after-insert-file-set-coding like insert-file-contents, to set
buffer-file-coding-system.
* lisp/fringe.el (fringe-bitmap-p): Make it a plain function.
* lisp/tooltip.el (tooltip-region-active-p): Remove.
* lisp/net/shr.el (shr-char-breakable-p, shr-char-kinsoku-bol-p)
(shr-char-kinsoku-eol-p, shr-char-nospace-p): Use define-inline.
* lisp/url/url-future.el (url-future-done-p, url-future-completed-p)
(url-future-errored-p, url-future-cancelled-p):
* lisp/url/url-dav.el (url-dav-http-success-p): Use define-inline.
* lisp/vc/ediff-init.el (ediff-odd-p): Remove.
(ediff-background-face): Use cl-oddp instead.
(ediff-buffer-live-p): Make it a defsubst.
* url-http.el (url-http): Respect `url-request-noninteractive'.
* url-queue.el (url-queue-start-retrieve): Fetching through
url-queue should always be noninteractive.
* url-vars.el (url-request-noninteractive): New variable.
* url-http.el (url-user-agent): New variable, can be function or
string. Suggested by Vibhav Pant <vibhavp@ubuntu.com>.
(url-http-user-agent-string): Use it.
* url-http.el (url-user-agent): New variable, can be function or
string. Suggested by Vibhav Pant <vibhavp@ubuntu.com>.
* doc/misc/org.texi (Dynamic blocks):
* lisp/allout-widgets.el (allout-widgets-hook-error-handler):
* lisp/calendar/appt.el (appt-display-message):
* lisp/calendar/icalendar.el (icalendar--convert-float-to-ical):
* lisp/calendar/timeclock.el (timeclock-in, timeclock-when-to-leave)
(timeclock-last-period, timeclock-day-base):
* lisp/eshell/em-ls.el (eshell-ls-file):
* lisp/eshell/esh-util.el (eshell-parse-ange-ls):
* lisp/generic-x.el (named-database-print-serial):
* lisp/net/newst-backend.el (newsticker--get-news-by-url-callback)
(newsticker-get-news, newsticker--sentinel-work)
(newsticker--image-get, newsticker--image-sentinel):
* lisp/net/tramp-sh.el (tramp-get-remote-touch):
* lisp/progmodes/opascal.el (opascal-debug-log):
* lisp/textmodes/remember.el (remember-mail-date)
(remember-store-in-files):
* lisp/vc/vc-annotate.el (vc-annotate-display-autoscale)
(vc-default-annotate-current-time):
* lisp/vc/vc-bzr.el (vc-bzr-shelve-snapshot):
* lisp/vc/vc-cvs.el (vc-cvs-annotate-current-time):
* lisp/vc/vc-rcs.el (vc-rcs-annotate-current-time):
* lisp/url/url-util.el (url-get-normalized-date):
* lisp/erc/erc-backend.el (TOPIC):
* lisp/gnus/gnus-delay.el (gnus-delay-article):
* lisp/gnus/gnus-sum.el (gnus-summary-read-document):
* lisp/gnus/gnus-util.el (gnus-seconds-today, gnus-seconds-month):
* lisp/gnus/message.el (message-make-expires-date):
* lisp/org/org-archive.el (org-archive-subtree)
(org-archive-to-archive-sibling):
* lisp/org/org-clock.el (org-resolve-clocks, org-clock-get-sum-start)
(org-clock-special-range):
* lisp/org/org-timer.el (org-timer-seconds):
* lisp/org/org.el (org-read-date-analyze, org-get-cursor-date):
* lisp/org/ox-html.el (org-html-format-spec):
* lisp/org/ox-icalendar.el (org-icalendar--vtodo):
Omit unnecessary call to current-time.
* lisp/calendar/time-date.el (time-to-seconds) [!float-time]:
* lisp/calendar/timeclock.el (timeclock-time-to-date):
* lisp/vc/vc-annotate.el (vc-annotate-convert-time):
Use current time if arg is nil, to be compatible with float-time.
(time-date--day-in-year): New function, with most of the guts of
the old time-to-day-in-year.
(time-to-day-in-year): Use it.
(time-to-days): Use it, to avoid decoding the same time stamp twice.
* lisp/calendar/timeclock.el (timeclock-update-mode-line):
* lisp/cedet/srecode/args.el (srecode-semantic-handle-:time):
* lisp/gnus/gnus-util.el (gnus-seconds-year):
* lisp/org/org.el (org-get-cursor-date):
Don't call current-time twice to get the current time stamp,
as this can lead to inconsistent results.
* lisp/completion.el (cmpl-hours-since-origin):
* lisp/erc/erc.el (erc-emacs-time-to-erc-time):
* lisp/ido.el (ido-time-stamp):
* lisp/vc/vc-annotate.el (vc-annotate-convert-time):
Simplify by using float-time.
* lisp/completion.el (save-completions-to-file):
* lisp/url/url-cache.el (url-cache-prune-cache):
Rename local var to avoid confusion.
* lisp/gnus/gnus-util.el (gnus-float-time):
* lisp/net/rcirc.el (rcirc-float-time):
* lisp/org/org-compat.el (org-float-time):
Simplify to an alias because time-to-seconds now behaves like float-time
with respect to nil arg.
* lisp/subr.el (progress-reporter-do-update):
Don't call float-time unless needed.
* lisp/erc/erc.el (erc-current-time): Simplify by using erc-emacs-time-to-erc-time.
* lisp/org/org-clock.el (org-clock-get-table-data): Omit unnecessary, lossy
conversion from floating point to Emacs time and back.
(org-resolve-clocks): Prefer two-argument floor.
* lisp/url/url-http.el (url-http-handle-authentication):
* lisp/url/url-news.el (url-news-fetch-message-id):
Use M-x report-emacs-bug in help messages. [Backport]
src/decompress.c (init_zlib_functions): Move the message about zlib
being unavailable from here...
(Fzlib_decompress_region): ...to here.
lisp/url/url-http.el (url-http-create-request): Recheck zlib availability
on windows-nt each time it might be required.
Currently, when `url-retrieve' is called for an https url it modifies
the variable `url-gateway-method'. This has been changed to
explicitly pass the requested gateway method to other functions.
When `url-retrieve' is being processed then (via
`accept-process-output') another `url-retrieve' call from a pending
timer can be started. The second call would always see the modified
`url-gateway-method' of the first one, which in general does not match
the url.
2014-09-28 Ulf Jasper <ulf.jasper@web.de>
* url-gw.el (url-open-stream): New optional parameter
`gateway-method'. If non-nil use it instead of global variable
`url-gateway-method'.
* url/url-http.el (url-http): New optional parameter
`gateway-method', pass it to `url-http-find-free-connection'.
(url-http-find-free-connection): New optional parameter
gateway-method, pass it to `url-open-stream'.
(url-https-create-secure-wrapper): Do not modify
`url-gateway-method' but explicitly provide 'tls as gateway-method
parameter to `url-https'.
* notes/exit-value: Remove specific discussion of VMS.
* doc/emacs/programs.texi (Program Modes): Don't advertise VMS DCL support
any more.
* doc/misc/ediff.texi (Merging and diff3): Don't mention lack of support
for VMS diff, we no longer support VMS.
* lisp/progmodes/ada-mode.el:
* lisp/net/tramp.el (tramp-handle-file-symlink-p):
* lisp/net/tramp-ftp.el (tramp-ftp-file-name-handler): Remove a comment
about VMS, which we no longer support.
* lisp/progmodes/ada-xref.el (ada-xref-current): Remove mention of VMS,
and fix a FIXME, using convert-standard-filename in place of
removed ada-convert-file-name.
* lisp/url/url-handlers.el: Remove a comment about VMS, which we no longer
support.
* lisp/calendar/parse-time.el (parse-time-rules):
Allow years up to most-positive-fixnum.
* lisp/calendar/time-date.el (date-to-time):
Pass "Specified time is not representable" errors through.
* lisp/url/url-cookie.el (url-cookie-expired-p): Treat out-of-range
expiration dates as if they were far in the future.
* src/editfns.c (decode_time_components): Store an invalid timespec
on overflow, instead of returning false, so that the caller can
distinguish overflow from other errors.
(lisp_time_argument, lisp_seconds_argument): If the time is out
of range, signal a time overflow instead of an invalid time spec.
* src/keyboard.c (decode_timer): Treat time overflow like other
timespec errors.
Fixes: debbugs:18176
(url-insert-file-contents): Condition on url-http-response-status
for the HTTP/S specific part.
* lisp/url/url-http.el (url-http-end-of-headers): Remove duplicate defvar.
Fixes: debbugs:17549
(url-http-parse-response, url-http-end-of-headers, url-recreate-url)
(url-http-target-url): Remove unused declarations.
(package-handle-response): Remove.
(package--with-work-buffer): Use url-insert-file-contents and simplify.
(package--download-one-archive): Use current-buffer instead of
dynamic binding of `buffer'.
(describe-package-1): Do not decode readme-string.
* lisp/url/url-handlers.el (url-http-parse-response): Add autoload.
(url-insert-file-contents): Signal file-error in case of HTTP error.
* lib-src/Makefile.in (regex.o): Remove reference to no-longer-used macros
CONFIG_BROKETS and INHIBIT_STRING_HEADER. "BROKETS" was a
misspelling anyway....
* src/nsterm.h (updateCollectionBehavior): Rename from
updateCollectionBehaviour. All uses changed.
* url-http.el (url-handle-content-transfer-encoding): Support
decompressing gzipped content.
* url-vars.el (url-mime-encoding-string): If we have built-in gzip
support, say that we accept gzipped content.
(success): Remove var.
(url-http-handle-authentication): Return the value that `success'
should take instead of setting `success' directly. Don't set `status'
since it's not used.
(url-http-parse-headers): Avoid unneeded setq.
Move the `setq success'.
(url-http): Use pcase.
(url-http-file-exists-p): Simplify.
* net/eww.el (eww-mode): Undo isn't necessary in eww buffers,
probably.
(eww-mode-map): Add a menu bar.
(eww-add-bookmark): New command.
(eww-bookmark-mode): New mode and commands.
* calculator.el (electric, ehelp): No need to load when compiling.
(Electric-command-loop, electric-describe-mode): Declare.
* doc-view.el (doc-view-current-converter-processes): Move before use.
* files.el (dired-unmark):
* progmodes/gud.el (gdb-input): Update declarations.
* emacs-lisp/easy-mmode.el (define-globalized-minor-mode):
Move MODE-set-explicitly definition before use.
* erc/erc-log.el (erc-network-name): Declare.
* erc/erc-notify.el (pcomplete--here): Declare.
(pcomplete/erc-mode/NOTIFY): Require pcomplete.
* international/mule-diag.el (mule-diag):
Don't use obsolete window-system-version.
* mail/feedmail.el (smtpmail): No need to load when compiling.
(smtpmail-via-smtp, smtpmail-smtp-server): Declare.
* mail/mail-utils.el (rfc822): No need to load when compiling.
(rfc822-addresses): Autoload it.
(mail-strip-quoted-names): Trivial simplification.
* mail/rmail.el (rmail-mime-message-p, rmail-mime-toggle-raw): Declare.
(rmail-retry-failure): Don't assume that rmail-mime-feature == rmailmm.
* net/snmp-mode.el (tempo): Don't duplicate requires.
* progmodes/prolog.el (info): No need to load when compiling.
(comint): Require before shell requires it.
(Info-goto-node): Autoload it.
(Info-follow-nearest-node): Declare.
(prolog-help-info, prolog-goto-predicate-info): No need to require info.
* textmodes/artist.el (picture-mode-exit): Declare.
* textmodes/reftex-parse.el (reftex-parse-from-file):
Trivial rewrite so the compiler can parse it better.
* url/url-dav.el (url-http): Require it.
(url-http-head-file-attributes): Don't autoload it.
* url/url-proxy.el (url-http): Autoload it.
* url-expand.el (url-default-expander): Don't calculate a default
url port before checking url-type.
* url-http.el (url-http-end-of-document-sentinel): Bind relevant
url-request-* variables around the call to url-http.
Fixes: debbugs:11469 debbugs:12374
function is a good handler for FOO.
(url-copy-file, url-file-local-copy, url-insert-file-contents)
(url-file-name-completion, url-file-name-all-completions)
(url-handlers-create-wrapper): Explicitly register as handler.
* url-dav.el (url-dav-supported-p): Added doc-string and remove
check for feature `xml' and function `xml-expand-namespace' which
never existed in Emacs proper.
(url-dav-process-response): Remove all indentation and newlines
from XML before parsing. Change call to `xml-parse-region' to do
namespace expansion with simple qualified names (Bug#11916).
(url-dav-request): Add autoload.
(url-dav-directory-files): Properly deal with empty directories.
Unhex URL before generating relative URLs.
(url-dav-file-directory-p): Fix check for 'DAV:collection.
* url/url-util.el (url-build-query-string): New function.
(url-parse-query-string): Allow that '=' is not required and split
URL parameters on ';', not just '&'.
The previous changes would make redirection fail, since the expansion
would end up with URLs that looked like "http:www.bing.com/hello".
* url-expand.el (url-default-expander): Copy over the fullness of
the new URL object based on the definition URL object.
* ffap.el (ffap-url-unwrap-local): Make it work right.
Use url-generic-parse-url, and handle host names and Windows
filenames properly.
(ffap-url-unwrap-remote): Use url-generic-parse-url.
(ffap-url-unwrap-remote): Accept list values, specifying a list of
URL schemes to work on.
(ffap--toggle-read-only): New function.
(ffap-read-only, ffap-read-only-other-window)
(ffap-read-only-other-frame): Use it.
(ffap-fixup-url): Don't check ffap-ftp-regexp, since it is not
necessary for ffap-url-unwrap-remote.
* url-parse.el (url-path-and-query, url-port-if-non-default): New
functions.
(url-generic-parse-url): Don't set the portspec slot if it is not
specified; that is what `url-port' is for.
(url-port): Only require the scheme to be specified to call
url-scheme-get-property.
* url-util.el (url-encode-url): Use url-path-and-query.
* url-vars.el (url-mime-charset-string): Load mm-util lazily.
Fixes: debbugs:9131
Fix 2012-04-10 change to url.el.
* url-http.el (url-http-create-request): Ignore obsolete
attributes slot of url-object.
* url-parse.el: Improve RFC 3986 conformance.
(url-generic-parse-url): Do not populate the ATTRIBUTES slot,
since this is not reliable for general RFC 3986 URIs. Keep the
whole path and query inside the FILENAME slot. Improve docstring.
(url-recreate-url-attributes): Mark as obsolete.
(url-recreate-url): Handle missing scheme and userinfo.
* url-util.el (url-encode-url): New function for URL quoting.
(url-encoding-table, url-host-allowed-chars)
(url-path-allowed-chars): New constants.
(url--allowed-chars): New helper function.
(url-hexify-string): Use them.
* url-vars.el (url-nonrelative-link): Make the regexp stricter.
* url.el (url-retrieve-internal): Use url-encode-url.
Fixes: debbugs:7017
* etc/publicsuffix.txt: New file.
* lisp/url/url-cookie.el (url-cookie-two-dot-domains): Remove.
(url-cookie-host-can-set-p): Use `url-domsuf-cookie-allowed-p'
instead of the variable above.
Fixes: debbugs:1401
* lisp/url/url-http.el (url-http-end-of-document-sentinel): Handle
keepalive expiry by calling url-http again.
(url-http): New arg, for the above.
Fixes: debbugs:10223
* url-queue.el (url-queue-kill-job): Delete the process sentinel
before killing the process to avoid a race condition between the
two processes killing off the process buffer.
* url-queue.el (url-queue-callback-function): Remove the job from
the queue so that we don't kill the current buffer, which will
then make the callback function kill a random buffer.
* lisp/url/url-future.el (url-future-test): Move to test/automated.
* test/automated/url-future-tests.el (url-future-tests): Move from
lisp/url/url-future.el and rename.
* url-http.el (url-http-create-request): Don't send cookies unless
requested.
(url-http-parse-headers): Don't store cookies unless requested.
* url.el (url-retrieve): Ditto
* url-queue.el (url-queue-retrieve): Take an optional
`inhibit-cookies' parameter.
* url-parse.el (url): Add the `use-cookies' slot to the URL struct
to be able to keep track of whether to do cookies or not on a
per-URL basis.
* url.el (url-retrieve-number-of-calls): New variable.
(url-retrieve-internal): Use it to expire the cache once in a
while.
* url-cache.el (url-cache-prune-cache): New function.
* url-queue.el (url-queue-setup-runners): New function that uses
`run-with-idle-timer' for extra asynchronicity.
(url-queue-remove-jobs-from-host): New function.
(url-queue-callback-function): Remove jobs from the same host if
connection failed.
* lisp/usr/url-auth.el (url-basic-auth): Allow reading usernames etc
when the minibuffer is already in use, eg in url-handler-mode.
Fixes: debbugs:10298
