471 Commits

Author SHA1 Message Date
Darshit Shah
76ab37bff9 contrib/make-release: Update regexes to match the NEWS file 2023-05-11 02:22:45 +02:00
Darshit Shah
774705838f cfg.mk: Replace uses of filesystem with file system 2023-05-11 01:03:23 +02:00
Darshit Shah
e234354080 Update NEWS 2023-05-11 00:48:44 +02:00
Darshit Shah
3a5ca80a1a bootstrap.sh: Update script from gnulib 2023-05-10 23:59:55 +02:00
Darshit Shah
c3fadea0a8 gnulib: Pull forward 2023-05-10 23:57:52 +02:00
jinfuchiang
2b723806a5 Update deprecated option '--html-extension' to '--adjust-extension'
* doc/wget.texi: Update option '--html-extension' to '--adjust-extension'.
  Renamed in Wget 1.12 to better reflect its behavior.

Copyright-paperwork-exempt: Yes
2023-04-09 20:29:07 +02:00
Tim Rühsen
9a35fe609c Don't write core dump if --secure-option value isn't suppported. 2023-03-19 17:39:45 +00:00
Tim Rühsen
d96d20630b * src/retr.c (fd_read_body): Increase bufsize from 8k to 64k 2023-02-18 17:26:23 +01:00
Tim Rühsen
c77c95033a * src/retr.c (fd_read_body): Simplify gzip initialization 2023-02-18 17:26:23 +01:00
Tim Rühsen
af1100f299 * src/retr.c (fd_read_body): Use MAX instead of max 2023-02-18 17:26:23 +01:00
Tim Rühsen
77929eda1b Simplify url_error function.
* src/url.c (url_error): simplify, remove url arg, return const char *.
* src/url.h (url_error): remove url arg, return const char *.
* src/html-url.c (get_urls_file): Simplify call to url_error(),
  remove call to free().
* src/http.c (metalink_from_http): Likewise.
* src/main.c (main): Likewise.
* src/metalink.c (retrieve_from_metalink, fetch_metalink_file): Likewise.
* src/recur.c (retrieve_tree): Likewise.
* src/res.c (res_retrieve_file): Likewise.
* src/retr.c (retrieve_url, retrieve_from_file): Likewise.
2023-02-18 17:25:56 +01:00
Tim Rühsen
218f6fee30 * src/url.c: Refactored url_error() 2023-02-18 17:25:56 +01:00
Tim Rühsen
2339d79b05 * tests/valgrind-suppressions: Add rule for 'strcmp-sse2.S in libdl.so' 2023-02-18 17:25:56 +01:00
Tim Rühsen
dcd2ed739e * configure.ac: Remove CC flags -Wc90-c99-compat and -Wlong-long 2023-02-17 14:00:02 +01:00
Tim Rühsen
24e876357a * fuzz/*: Use grep >/dev/null instead of grep -q 2023-02-17 13:46:58 +01:00
Tim Rühsen
8247a34521 * src/retr.c: Don't include hash.h 2023-02-17 13:46:58 +01:00
Tim Rühsen
4223996930 Update copyright years 2023-02-17 13:46:58 +01:00
Tim Rühsen
27d3fcba33 * gnulib: Update 2023-02-17 13:46:58 +01:00
Tim Rühsen
c69030a904 * configure.ac: Disable nettle if NTLM is explicitly disabled 2022-12-11 13:31:38 +01:00
Tim Rühsen
485217d0ff * configure.ac: Allow disabling NTLM if nettle present (Savannah #63431) 2022-12-10 16:43:38 +01:00
Tim Rühsen
9835085544 * fuzz/Makefile.am: Add -lz to fuzzer libs 2022-09-24 18:41:31 +02:00
Tim Rühsen
aab539bb44 * src/main.c (print_help): Add --retry-on-host-error to help text 2022-03-20 21:07:54 +01:00
Tim Rühsen
cb114fbbf7 Fix HSTS portability by using int64_t instead of time_t.
* src/hsts.c: Use int64_t instead of time_t.
* src/http.c: Use int64_t for parsing Strict-Transport-Security.
2022-03-20 20:57:10 +01:00
Tim Rühsen
1cda2bb5d5 src/http.c (time_to_rfc1123): Fix -Wformat-nonliteral 2022-03-20 20:53:10 +01:00
Tim Rühsen
565f566fab * src/warc.c (warc_process_cdx_line): Fix variable type to idx_t 2022-03-20 20:49:07 +01:00
Tim Rühsen
c7e6e378e5 * src/main.c (secs_to_human_time): Use snprintf instead of sprintf 2022-03-20 20:45:59 +01:00
Tim Rühsen
59d08d3290 * src/main.c (main): Remove unused variable 2022-03-20 20:44:32 +01:00
Tim Rühsen
8d5cdef9a9 * src/netrc.c (test_parse_netrc): Check if HAVE_FMEMOPEN is defined 2022-03-20 15:57:21 +01:00
Darshit Shah
9f93ffb44b maint: post-release administrivia
* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.
2022-02-26 17:27:55 +01:00
Darshit Shah
40747a11e4 * NEWS: Update NEWS items for release 2022-02-26 17:20:26 +01:00
Darshit Shah
b6f3c6153e * .gitignore: Remove empty line at EOF 2022-02-26 17:02:26 +01:00
Darshit Shah
e1fa51206c * .gitignore: Update file 2022-02-26 16:54:51 +01:00
Darshit Shah
6d4a4e56c8 * src.hsts.c(hsts_read_database): Use SCNd64 for portable format flags 2022-02-26 16:51:21 +01:00
Darshit Shah
14a7f68f46 * configure.ac: Add some warning flags to ignore 2022-02-26 16:50:48 +01:00
Darshit Shah
ccc7866fea * cfg.mk: Remove passing syntax-checks from skip list 2022-02-26 16:24:57 +01:00
Darshit Shah
2730a00c0d Fix issues from syntax-check
* doc/wget.texi: s/time stamp/timestamp/
* src/ftp-ls.c(clean_line): Same
  (ftp_parse_vms_ls): Don't cast return value of x*alloc
* src/ftp.c: Don't cast return value of alloca
* src/css-url.c: wget.h should be the first include
* testenv/Makefile.am: Use $(PATH_SEPARATOR) instead of : when declaring
  a path
* testenv/Test-no_proxy-env.py: s/can not/cannot/
2022-02-26 16:20:30 +01:00
Darshit Shah
be936bda56 * Update Copyright years 2022-02-26 15:49:52 +01:00
Darshit Shah
7ba0a44939 * bootstrap: Update script 2022-02-26 15:45:03 +01:00
Darshit Shah
98c23153a2 * gnulib: Pull forward 2022-02-26 15:45:02 +01:00
Darshit Shah
3a470a90f2 Cleanup some incorrect uses of AM Conditionals
* configure.ac: Replace IRI_IS_ENABLED with WITH_IRI and
      METALINK_IS_ENABLED with WITH_METALINK
    * src/Makefile.am: Add the .c and .h files for IRI, Metalink and
      Xattr support to wget_SOURCES directly when needed instead of
      adding them as object files.
    * testenv/Makefile.am: Rename METALINK_IS_ENABLED to WITH_METALINK
2022-02-26 15:45:02 +01:00
Darshit Shah
f5263969fa Replace incorrect usage of AC_LIBOBJ in configure.ac
AC_LIBOBJ is to be used for providing replacement functions for
compatibility reasons. Not for conditional compilation of entire source
files. Autotools provides other methods for achieving that. However, our
build system has (ab-)used the AC_LIBOBJ macros for doing this for over
15 years. That stops today.

    * configure.ac: Replace uses of AC_LIBOBJ with automake conditionals
      that can be used in Makefiles
    * src/Makefile.am: Use the defined conditionals to select which
      files get built.
2022-02-26 15:20:29 +01:00
Darshit Shah
cc5ec2a158 Fix case where installed gettext is newer than minimum version
* configure.ac: Use AM_GNU_GETTEXT_REQUIRE_VERSION to specify a minimum
  version of gettext needed. This should allow newer versions of gettext
  to be copied for autipoint to use when available.

This commit hopefully fixes the annoying issue when building from
source on a system that has a newer version of gettext.
2022-02-25 17:25:30 +01:00
Tim Rühsen
f7ce79fd85 * src/netrc.c (test_parse_netrc): Free netrc structure 2022-02-13 18:00:07 +01:00
Tim Rühsen
d139fecbe8 * .gitlab-ci.yml: Fix path to llvm-symbolizer 2022-02-13 17:57:39 +01:00
Tim Rühsen
74a9d9e7c4 * src/netrc.c (test_parse_netrc): New unit test function 2022-02-13 17:28:02 +01:00
Aarni Koskela
446afdca21 * src/http.c (parse_strict_transport_security): Fix typo in string
Copyright-paperwork-exempt: Yes
2022-01-31 18:01:51 +01:00
Tim Rühsen
e6fa409a4d * .gitlab-ci.yml (Scan-Build): Allow failure due to two false positives 2022-01-23 14:10:49 +01:00
Tim Rühsen
a24e67e239 * configure.ac: Use pkg-config for gpgme, libidn2 and nettle 2022-01-23 14:10:49 +01:00
Tim Rühsen
c984cb316a * src/ftp.c: Small cleanups 2022-01-22 19:53:26 +01:00
Nik Soggia
35a6317b99 Print newline after dot progress bar in non-verbose mode
* src/progress.c (dot_finish): Print new in all progress bar contexts
  instead of just verbose

Copyright-paperwork-exempt: Yes
2022-01-16 00:01:39 +01:00
Tim Rühsen
9474a2c6f4 * .gitlab-ci.yml: Fix artifact path for the Scan-Build runner 2022-01-12 22:10:01 +01:00
Tim Rühsen
e7a4d818fa * src/main.c (main): Unlink output document when --unlink is given 2022-01-09 18:58:41 +01:00
Tim Rühsen
f354529708 fuzz/*.in: Update fuzzer corpora 2021-12-22 19:49:24 +01:00
Tim Rühsen
67d4cb3ab6 * .gitlab-ci.yml (CoverageReports): Fix artifacts paths 2021-12-22 17:00:47 +01:00
Tim Rühsen
d2af84fbb3 * tests/valgrind-suppressions: Fix libidn rule 2021-12-22 14:32:50 +01:00
Tim Rühsen
bfb5bedf7d * .gitlab-ci.yml: Fix artifacts paths 2021-12-22 14:29:00 +01:00
Tim Rühsen
8c5a620f0f * tests/valgrind-suppressions: Extend libidn rule 2021-12-22 14:01:47 +01:00
Tim Rühsen
c34c2529dc * src/log.c (logprintf): Check earlier for verbosity 2021-12-22 13:07:23 +01:00
Tim Rühsen
c7a37d82ee * src/http.c (http_loop): Fix memleak 2021-12-22 13:06:34 +01:00
Tim Rühsen
c81042295e Switch fuzzing build from C++ to C
* Makefile.am (oss-fuzz): Build with $CC instead of $CXX.
* README.md: Remove CXX and CXXFLAGS exports.
2021-12-21 19:20:52 +01:00
Darshit Shah
f75fcf2985 * src/http.c (http_loop): Hide password when printing status with -nv
Reported-By: Per Lundberg <perlun@gmail.com>
Closes: #61492
2021-12-01 23:38:52 +01:00
Darshit Shah
22611a77ba * gnulib: Pull forward 2021-12-01 23:03:56 +01:00
Darshit Shah
e1bacd2fa5 * src/hsts.c (hsts_read_database): Read time_t values as long long 2021-12-01 22:42:42 +01:00
Thomas Niederberger
faeb4d90c2 * src/main.c (print_help): Add command line option for TLS 1.3 2021-12-01 22:17:11 +01:00
Darshit Shah
65e6d5b3b8 * retr.c (rotate_backups): Non existent files are not errors in this function 2021-10-11 23:06:38 +02:00
Darshit Shah
aecf5fbf1b * ftp.c (ftp_loop_internal): Fix computation of total_downloaded_bytes
When continuing a FTP download, or not starting one because the file is
already fully retrieved, don't include the size of the file in the
total_downloaded_bytes. Only the actual amount of data retrieved over
the network should be considered there.

Fixes: #61277
Reported-By: Michal Ruprich <formaiko>
2021-10-08 20:37:51 +02:00
Darshit Shah
3ea9658c07 Remove suprious print statements
* src/gnutls.c: Remove fprintf statements. We should never print to
  console directly. Always honor the log levels.
  Fixes: #61125
2021-09-08 17:52:32 +02:00
Darshit Shah
31b82825a2 maint: post-release administrivia
* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.
2021-09-07 21:04:48 +02:00
Darshit Shah
d5588ac221 * NEWS: Update NEWS items for release 2021-09-07 20:57:24 +02:00
Darshit Shah
204a4bccf4 * cfg.mk: Disable the indent check 2021-09-07 20:57:24 +02:00
Darshit Shah
5a79362a56 * gnulib: Pull forward 2021-09-07 20:12:55 +02:00
WB
ebb96761f5 Fix #60956 (improve --page-requisites)
* src/html_url.c (tag_handle_link): Check for "alternate stylesheet",
  "icon" and "manifest".
2021-08-21 19:51:12 +02:00
Tim Rühsen
7899e1d17b * src/html-url.c (tag_handle_meta): Fix integer overflow 2021-08-07 14:29:02 +02:00
Tim Rühsen
254b2d3c7c * src/recur.c (download_child): Remove temporary robots.txt.tmp 2021-07-05 15:43:13 +02:00
Josef Moellers
718ab3f79b Long pathnames patch 2021-06-14 08:33:57 +02:00
Tim Rühsen
d73e8b42f7 * .gitlab-ci.yml: Remove scan-build deps from pages stage 2021-06-07 20:32:41 +02:00
Tim Rühsen
4a5a010eb3 * .gitlab-ci.yml: Move scan-build/ to public/ only if it exists 2021-06-07 00:11:11 +02:00
Tim Rühsen
89fc8d84b7 * .gitlab-ci.yml: Move scan-build results to parent directory 2021-06-06 23:48:42 +02:00
Tim Rühsen
a0e0965d1e * .gitlab-ci.yml: Move coverage results to parent directory 2021-06-06 23:34:46 +02:00
Tim Rühsen
96971867ea * .gitlab-ci.yml: Don't allow failure of scan-build 2021-06-06 23:26:33 +02:00
Tim Rühsen
5192dddbdd * .gitlab-ci.yml: Pages needs Build-Tarball 2021-06-06 23:18:24 +02:00
Tim Rühsen
91c42c799a * src/url.c (append_uri_pathel): Add cheap extra check to help static analyzers 2021-06-06 15:34:12 +02:00
Tim Rühsen
c778ac20b4 * src/http.c (gethttp): Add cheap extra check to help static analyzers 2021-06-06 15:34:06 +02:00
Tim Rühsen
4d3ee1604a * .gitlab-ci.yml: Exclude src/css_.o from scan-build 2021-06-06 15:34:03 +02:00
Tim Rühsen
a209bb1fac * src/main.c (main): Removed unused variable 2021-06-06 15:33:59 +02:00
Tim Rühsen
36e250e09a Revert "Long pathnames patch"
This reverts commit affad27664.

Manual tests with very long path names did not work with this patch.
We have to wait for a patch including automated tests.
2021-06-06 14:10:22 +02:00
Tim Rühsen
f3299cc606 * .lgtm.yml: Add autoconf-archive 2021-06-06 02:39:28 +02:00
Tim Rühsen
de4b94a5ae * .gitlab-ci.yml: Re-arrange stages 2021-06-06 02:39:28 +02:00
Tim Rühsen
c1599deb1f Remove .git/, .gitmodules and gnulib/ from bootstrap artefacts 2021-06-06 02:39:28 +02:00
Darshit Shah
61df89eb91 Updated Gitlab CI Pipeline
* .gitlab-ci.yml: COmpletely overhauled CI pipeline with dependencies
  for a faster and more robust CI.
2021-06-06 02:39:17 +02:00
jmoellers
affad27664 Long pathnames patch 2021-05-29 17:17:27 +00:00
Tim Rühsen
170896a76a * fuzz/wget_read_hunk_fuzzer.c: Add explicit cast for C++ compilation 2021-05-22 16:48:19 +02:00
Tim Rühsen
d4431a0b97 * fuzz/wget_ntlm_fuzzer.c: Add explicit cast for C++ compilation 2021-05-22 16:48:14 +02:00
Tim Rühsen
027d294114 * src/http.c (initialize_request): Send Host HTTP header first
This solves an issue where the server expects the Host: header
as first one. This seems plausible (ahem) as the Host: header is the
only one that is required.
2021-05-03 17:49:58 +02:00
Tim Rühsen
5fe8d26904 Improve wget_options_fuzzer
* fuzz/fuzzer.h: Ignore -Wunused-parameter.
* fuzz/wget_options_fuzzer.c: Let getaddrinfo() fail in while fuzzing.
* fuzz/wget_options_fuzzer.in/*: Update corpora from OSS-Fuzz.
2021-05-02 19:43:06 +02:00
Nekun
aabdf6eb66 Fix typo in VMS support code
* src/utils.c: Remove unpaired brace

Copyright-paperwork-exempt: Yes
2021-05-02 14:19:01 +02:00
Nils
1aada296dd Use "nofollow" instead of "no-follow" in messages
* src/html-url.c (get_urls_html_fm): Remove misleading debug message.
* src/recur.c (retrieve_tree): Fix no-follow -> follow in DEBUGP.

The attribute in html is "nofollow" so it is more consistent to call it
so than to hyphenate it.

Copyright-paperwork-exempt: Yes
2021-04-15 21:03:56 +02:00
Nils
f1cccd2c45 Print message for no-follow attribute only if norobots respected
* src/html-url.c (get_urls_html_fm): Remove misleading log message.
* src/recur.c (retrieve_tree): Add log message into correct if block.

Commit e39be32838 added a message that
said links will not be followed whenever the nofollow attribute is found
in a page. It didn't take into account that with -e robots=off (and
equivalents) links will still be followed.

This bug has been noticed multiple times:
* https://www.reddit.com/r/DataHoarder/comments/mprq89/wget_respects_nofollow_attribute_despite_e/
* https://gist.github.com/simonw/27e810771137408fd7834ad153750c41#gistcomment-3648191
* https://superuser.com/questions/1494761/wget-wont-ignore-no-follow-attributes

This commits makes it so that this message is only printed when a
nofollow link is found and the norobots convention is respected.

Copyright-paperwork-exempt: Yes
2021-04-15 21:02:28 +02:00
Tim Rühsen
11c626efc4 * configure.ac: Add AM_GNU_GETTEXT_VERSION 2021-04-15 20:56:27 +02:00
Tim Rühsen
db6166d011 * testenv/test/base_test.py: Rename valgrind-suppression-ssl -> valgrind-suppressions-ssl 2021-04-11 19:41:03 +02:00
Tim Rühsen
a65bc7c0e8 * testenv/test/base_test.py: Add --show-leak-kinds=all valgrind argument 2021-04-11 19:41:00 +02:00
Tim Rühsen
3a1ac38f00 * tests/WgetTests.pm: Add --show-leak-kinds=all valgrind argument 2021-04-11 19:40:56 +02:00
Tim Rühsen
d745ea0eb6 Fix testenv valgrind suppression
* testenv/Makefile.am: Remove valgrind-suppressions from EXTRA_DIST.
* testenv/valgrind-suppression-ssl: Rename to testenv/valgrind-suppressions-ssl.
2021-04-11 19:09:45 +02:00
Tim Rühsen
82afc6e6f0 * bootstrap.conf: Add 'rename' to gnulib modules
The missing module could be a reason for
https://savannah.gnu.org/bugs/?60346.

Reported-by: jrharris42@gmail.com
2021-04-11 13:16:21 +02:00
Tim Rühsen
90631a6fe5 * src/wget.h: Use strtoll() for str_to_wgint
This fixes a regression reported at https://savannah.gnu.org/bugs/?60353.

Reported-by: Michal Ruprich
2021-04-11 12:53:20 +02:00
Shamil Gumirov
fd2a061f6a Minor output fix to use quote_n() instead of quote()
* src/ftp.c (ftp_retrieve_list): change quote to quote_n
* src/iri.c (do_conversion): change quote to quote_n
* src/url.c (convert_fname): change quote to quote_n

The implementation quote() reuses the buffer it returns which
leads to printing the same string for each quote() call in one
output line. Instead, quote_n() should be used as highlighted in
the doc:
https://www.gnu.org/software/gnulib/manual/html_node/Quoting.html

Copyright-paperwork-exempt: Yes
2021-04-11 12:42:07 +02:00
Tim Rühsen
27b12dad12 * src/Makefile.am: Add metalink.c and xattr.c to EXTRA_wget_SOURCES 2021-04-05 12:37:28 +02:00
Tim Rühsen
c8d7a66217 * testenv/Makefile.am: Allow tarball build without python3 2021-04-05 12:37:15 +02:00
Tim Rühsen
6d58a04682 Fix spelling errors
* NEWS: trough -> through.
* README.checkout: Likewise.
* contrib/spell-checker: Add exclusion 'parm'
2021-04-04 17:14:57 +02:00
Tim Rühsen
7840db6c0b Remove gettext version requirements
* bootstrap.conf: Remove gettext version requirements
* configure.ac: Likewise.
2021-04-04 17:14:54 +02:00
Darshit Shah
cf788d60e4 Use a LOG_COMPILER for running fuzz tests
This prevents needing a special case in the fuzz tests to detect
valgrind usage and a new exec. Instead, we simply detect the environment
in a shell script and start the test with valgrind in the first place.

* fuzz/test-runner.sh: New script for running the fuzz tests
* fuzz/main.c (main): Remove code for re-running under valgrind
* fuzz/Makefile.am: Set the LOG_COMPILER to test-runner.sh and remember
  to add it to the list of extra files for packaging
2021-03-16 21:48:13 +01:00
Tim Rühsen
db4d5b6eb3 * configure.ac: Add AC_PROG_CC_C99 2021-03-07 19:56:36 +01:00
Darshit Shah
e9cf8da1a6 Bump gettext version to 0.20
* bootstrap.conf: Increase minimum version of gettext needed to 0.20
* configure.ac: Same
2021-03-07 14:50:11 +01:00
Darshit Shah
b743178b9d * bootstrap: Update script 2021-03-07 14:40:41 +01:00
Darshit Shah
65cc47b504 * gnulib: Pull forward 2021-03-07 14:39:53 +01:00
Darshit Shah
f7835691b4 Fix double free in FTP Code
* src/ftp.c(getftp): Don't free `target`. If it is not pointing to
  targetbuf, then it still pointing to its original location of u->dir.
  This location will be free'd later. Doing so now causes a double free
  and hence crashes Wget
* tests/Test-ftp-dir.px: New test to show double free error
* tests/Makefile.am: Add new test
2021-03-02 12:03:14 +01:00
Tim Rühsen
51ee45f017 * doc/wget.texi: Replace '' with in example
Needed to hide from texi2pod.pl replacement of '' -> ".

Reported-by: Vincent Lefevre
2021-01-31 18:23:36 +01:00
Tim Rühsen
7d9ed223fc Use gnulib's utime.h
* bootstrap.conf: Remove utime-h (included by utime).
* configure.ac: Remove header checks for utime.h and sys/utime.h.
* src/utils.c: Simply #include <utime.h>.
2021-01-23 19:28:58 +01:00
Tim Rühsen
f81191951a * configure.ac: Detect unresolved AX_ macros 2021-01-23 19:09:36 +01:00
Tim Rühsen
ea6143d56c * configure.ac: Use noyywrap for AC_PROG_LEX 2021-01-16 20:09:07 +01:00
Tim Rühsen
ad36a467ac Fix --quota on systems with 32bit long type
* src/init.c (cmd_bytes_sum): Use WGINT_MIN and WGINT_MAX in check.
* src/options.h (struct options): Make 'quota' of type wgint.
* src/retr.c: Make 'total_downloaded_bytes' of type wgint.
* src/utils.h: Fix comment.
* src/wget.h: Add WGINT_MIN, remove SUM_SIZE_INT.
2021-01-16 20:00:39 +01:00
Tim Rühsen
adc5aa0196 * configure.ac: Remove obsolete AC_HEADER_STDC 2021-01-16 19:34:53 +01:00
Tim Rühsen
5c8569ba5d * configure.ac: Replace ` with ' 2021-01-16 19:31:04 +01:00
Tim Rühsen
9f1c8dac77 * configure.ac: Use yywrap param for AC_PROG_LEX 2021-01-16 19:19:01 +01:00
Darshit Shah
6c84dc38bc maint: post-release administrivia
* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.
2021-01-09 11:01:24 +01:00
Darshit Shah
f720bdda9f * NEWS: Update NEWS items for release 2021-01-09 10:55:08 +01:00
Darshit Shah
90d43efbea * gnulib: Pull forward 2021-01-09 10:55:02 +01:00
Darshit Shah
0031bdedfc * cfg.mk: Add manual_title for web docs 2021-01-09 10:54:27 +01:00
Darshit Shah
72ede7a028 * contrib/make-release: Update to use maintainer-makefile 2021-01-09 10:54:10 +01:00
Darshit Shah
473eef671b * bootstrap.conf: Add gendocs and gnu-web-doc-update modules 2021-01-09 01:49:07 +01:00
Darshit Shah
8c35102b37 * configure.ac(AM_INIT_AUTOMAKE): Create lzip archives when making a distribution 2021-01-09 01:03:43 +01:00
Darshit Shah
ae203984e8 Prepare for making releases
The maintainer makefile provides targets for making a release. They need
these things:
* cfg.mk: Set the hash of the NEWS file as it was at the beginning of
  this release
* .prev-version: Create the .prev-version file
2021-01-09 00:46:24 +01:00
Darshit Shah
5f6f86aa26 Prepare NEWS file for new release
* NEWS: Fix the copyright notice
* NEWS: Add header for new release
2021-01-09 00:46:06 +01:00
Darshit Shah
e9641d989b Use PRId64 to correctly identify the format specifier
* src/utils.c: Use PRId64 to correctly identify the format specifier for
wgint values. This fixes a warning on 32-bit systems where wgint is a
long long int instead of the long int that the format specifier
indicated.

Reported-by: Jeffrey Walton
2021-01-07 21:35:20 +01:00
Darshit Shah
9f3df123bb * src/retr.c(rotate_backups): Simplify logic for handling filename rotation 2021-01-03 15:59:49 +01:00
Matt Whitlock
1e89e5f66c * configure.ac: Don't use Bashisms 2021-01-03 15:22:22 +01:00
Darshit Shah
5a7f2f7e87 Run make update-copyright 2021-01-01 12:31:01 +01:00
Darshit Shah
2e5d281468 * gnulib: Pull forward 2021-01-01 12:15:48 +01:00
Darshit Shah
8923dcc788 * NEWS: Finalize release 2020-12-31 16:39:39 +01:00
Darshit Shah
c2517256e4 * gnulib: Update for release 2020-12-31 16:27:48 +01:00
Darshit Shah
37f0dca4e2 * src/main.c: Disable use-askpass on VMS 2020-12-30 23:04:13 +01:00
Steven M. Schweda
8af2171a34 Fixes for running on VMS
time_t on VMS is typically unsigned.  (Lazy man's solution to 2038?)
I added "(time_t)" type casts to negative values ("-1"), and changed
tests to avoid complaints.

* src/hsts.c (hsts_add_entry): Explicitly cast potentially negative time
  values to time_t to handle VMS quirks.
  (hsts_store_entry): Same
  (get_hsts_store_filename): Use new ajoin_dir_file function to join
  filenames
  (test_hsts_read_database): Same
* src/init.c (struct options): use-askpass is not implemented on VMS
  (ajoin_dir_file): New Function to join filenames in a platform
  agnostic manner
  (wgetrc_user_file_name): Use ajoin_dir_file to join paths. Doing this
  correctly, eliminates the need for a special case on VMS
* src/init.h: Add prototype for ajoin_dir_file
* src/log.c (check_redirect_output): Ignore on VMS
* src/main.c(option_data): Disable use-askpass on VMS
  (print_help): Same
  (get_hsts_database): Use ajoin_dir_file to join paths
  (print_version): Add VMS specific information to Version output
* src/utils.c (fork_to_background): Fix signature on VMS

Co-authored-by: Darshit Shah <darnir@gnu.org>
2020-12-30 22:50:32 +01:00
Tim Rühsen
7ec15b9c92 Remove SIZEOF_WGINT as wgint is always int64_t
* src/http.c (test_parse_range_header): Remove use of SIZEOF_WGINT.
* src/utils.c (human_readable): Remove superfluous HR_NUMTYPE,
*   (number_to_string): Remove use of SIZEOF_WGINT.
* src/utils.h: Remove use of SIZEOF_WGINT and HR_NUMTYPE.
* src/wget.h: Remove #define SIZEOF_WGINT.
2020-12-29 12:44:20 +00:00
Tim Rühsen
a16149e5bb src/wget.h: Cleanup code around wgint 2020-12-29 12:44:20 +00:00
Darshit Shah
db88ad441e Remove portability handling for str[n]casecmp
* src/mswindows.c: Gnulib ensures we always have str{n}casecmp
* configure.ac: Don't need to define HAVE_STR[N]CASECMP anymore
2020-12-29 12:44:20 +00:00
Darshit Shah
8b1aeab783 Remove portability handling code for wgint
Gnulib's stdint.h module promises a C99 compliant stdint.h file on all
platforms. Thus allowing us to directly use the fixed wodth integer
type, int64_t wihout needing to resort to all the checks being
performed.

* src/wget.h: Assume that int64_t is always available and use it
* src/mswindows.h: Remove portability code since gnulib handles it
* configure.ac: Remove sizeof checks for integer types that are no
  longer used
2020-12-29 12:44:20 +00:00
Tim Rühsen
dd2d960ba1 * .lgtm.yml: Call autoreconf to support Debian oldstable 2020-12-29 11:24:15 +01:00
Tim Rühsen
5b7d068a4b Fix --accept-regex/i--reject-regex for FTP
* src/ftp.c (ftp_retrieve_glob): Call accept_url() with the full URL

Reported-by: Frans de Boer <frans@fransdb.nl>
2020-12-28 23:33:48 +01:00
Darshit Shah
2e4504c553 * tests/valgrind-suppressions: Make suppression for libidn false positive more generic 2020-12-28 22:44:33 +01:00
Darshit Shah
a2f23b30c2 * Makefile.am: Clean up gnulib_po/stamp-po 2020-12-28 02:52:43 +01:00
Darshit Shah
73c474edf6 tests/valgrind-suppressions: Add suppressions for false positives 2020-12-28 02:34:02 +01:00
Darshit Shah
4fad8a7e2d testenv/Test--https.py: Fix missing import 2020-12-28 02:10:10 +01:00
Darshit Shah
9718667301 .gitlab-ci.yml: Everyone needs an autoreconf now 2020-12-28 02:08:14 +01:00
Darshit Shah
733ba71b13 .gitlab-ci.yml: Even Debian Stable requires running autoreconf 2020-12-28 01:57:13 +01:00
Darshit Shah
010cb79b43 Fix usage of Magic number of tests
testenv/test/base_test.py: Add new variable SKIP_TEST
testenv/Test--https-crl.py: Use SKIP_TEST instead of magic number
testenv/Test--https.py: Same
testenv/Test-hsts.py: Same
testenv/Test-no_proxy-env.py: Same
testenv/Test-pinnedpubkey-der-https.py: Same
testenv/Test-pinnedpubkey-der-no-check-https.py: Same
testenv/Test-pinnedpubkey-hash-https.py: Same
testenv/Test-pinnedpubkey-hash-no-check-fail-https.py: Same
testenv/Test-pinnedpubkey-pem-fail-https.py: Same
testenv/Test-pinnedpubkey-pem-https.py: Same
2020-12-28 01:54:16 +01:00
Darshit Shah
7c4d053902 .gitlab-ci.yml: All modern builds need autoreconf 2020-12-28 01:36:22 +01:00
Darshit Shah
13b3bc7f8f .po/POTFILES.in: Remove files containing no translateable strings 2020-12-28 01:31:31 +01:00
Darshit Shah
c80b1271d7 Fix issues reported by codespell 2020-12-28 01:23:34 +01:00
Darshit Shah
6898efd978 .gitlab-ci.yml: Need --force option to autopoint 2020-12-28 01:20:51 +01:00
Darshit Shah
d82f158e3d * .gitlab-ci.yml: Don't need autoreconf everywhere. Only on Arch
This (partially) reverts commit 0309d13794.
2020-12-28 01:10:32 +01:00
Darshit Shah
29917a3052 configure.ac: Revert to requiring gettext 0.19 2020-12-28 01:09:03 +01:00
Darshit Shah
0309d13794 .gitlab-ci.yml: Call autoreconf to refresh gettext files 2020-12-28 01:02:38 +01:00
Darshit Shah
5ef6756f0a * NEWS: Update NEWS items 2020-12-28 00:50:05 +01:00
Darshit Shah
749cfc9717 configure.ac: Raise minimum gettext version to 0.20 2020-12-28 00:48:01 +01:00
Darshit Shah
1942eaf409 Run autoupdate on configure.ac
* configure.ac, m4/{wget, wget_manywarnings}.m4: Run autoupdate
2020-12-28 00:39:30 +01:00
Darshit Shah
99b5977169 .gitignore: Ignore gnulib_po directory 2020-12-28 00:35:04 +01:00
Darshit Shah
0c2a33b5dc configure.ac: Replace obsolete macro AC_AIX with AC_USE_SYSTEM_EXTENSIONS 2020-12-28 00:12:39 +01:00
Darshit Shah
f4a3c1a35c bootstrap.conf: Remove unneeded text block 2020-12-27 23:47:49 +01:00
Darshit Shah
b99d519a57 m4/getpagesize.m4: Don't track autogenerated file 2020-12-27 22:29:42 +01:00
Darshit Shah
70bacc432d Remove unneeded files from gnulib
m4/exitfail.m4: Remove unneeded file
m4/wchar.m4: Same
m4/wctype.m4: Same

The latest versions of these files will be copied by gnulib-tool upon
its invokation. No need to track them around here
2020-12-27 22:06:29 +01:00
Darshit Shah
a11bfc2d4e Use a separate domain for translating gnulib
Use the --po-domain option to gnulib-tool to create a new textdomain
that can be used by gnulib files for translations. This way, we don't
have to maintain the list of all files that require translations in
gnulib.

* bootstrap.conf: Use --po-domain and --po-base options to create a
  separate base for gnulib translations
* src/main.c(i18n_initialize): Call bindtextdomain on wget-gnulib to
  include those translations as well
* Makefile.am: Add new directory gnulib_po to SUBDIRS
* configure.ac: Generate gnulib_po/Makefile.in
* lib/Makefile.am: Set AM_CPPFLAGS to empty since gnulib.mk expects it
  to be set
2020-12-27 21:15:45 +01:00
Darshit Shah
8d68a6307b po/POTFILES.in: Update the list of files that need translation 2020-12-27 20:53:18 +01:00
Darshit Shah
fd0f21b04c m4/getpagesize.m4: Update from gnulib 2020-12-27 20:47:12 +01:00
Darshit Shah
7e1bc0de8a configure.ac: Increase minimum required version to 2.64 2020-12-27 20:21:51 +01:00
Darshit Shah
3396b943e0 gnulib: Pull forward 2020-12-27 19:29:32 +01:00
Darshit Shah
3636b2a5af main.c (main): Warn when trying to use password without username 2020-12-22 22:25:26 +01:00
Tim Rühsen
015afd7cc7 * src/http.c (http_cleanup): Reset wget_cookie_jar after freeing
This silences the wget_options_fuzzer which triggered #28610 on
OSS-Fuzz. This issue can not happen with the Wget utility.
The fuzzer runs main(),...,cleanup() in a loop which the Wget utility
never does.
2020-12-13 18:23:39 +01:00
Tim Rühsen
794b7b1dbe * src/main.c: Add description to --help output of wait options 2020-11-08 18:46:11 +01:00
Tomas Hozza
314a4f42be testenv: Add test for handling of no_proxy environment variable
* testenv/Test-no_proxy-env.py: Added new test for no_proxy env
* testenv/Makefile.am: Added the new test to Makefile

Added new test with 5 cases, which are testing various combinations
of no_proxy environment variable definition and requested URLs.
The test is skipped if the system does not support resolution of
localhost subdomains to lcalhost address.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2020-07-11 22:54:26 +02:00
Tomas Hozza
8ffebe2160 testenv: Allow definition of environment variables for wget execuion
* testenv/README: Added description for new EnvironmentVariable hook
* testenv/conf/environment_variable.py: Added implementation of
EnvironmentVariable hook
* testenv/test/base_test.py: Modified exec_wget() to enable use of
EnvironmentVariable hook

Added new test hook called EnvironmentVariables, for defining environment
variables when wget is executed in tests. This is handy for testing
environment variables, which are accepted by wget.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2020-07-11 22:54:26 +02:00
Tomas Hozza
28196b6807 testenv: HTTPTest.begin() should return exit value
* testenv/test/http_test.py: Ensure that HTTPTest.begin() always returns
a value

Previously the HTTPTest.begin() method always returned None. However this
is not consistent with the begin() implementation of the parent class
(BaseTest). This change ensures that HTTPTest.begin() returns a value.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2020-07-11 22:54:26 +02:00
Darshit Shah
1fe9d4cf09 Minor documentation clarification
* src/wget.texi: Clarify behaviour of --quota
  Thanks to TheWild on IRC for pointing it out
2020-07-07 21:47:06 +02:00
Tim Rühsen
1656a1628c * src/ftp.c (ftp_loop_internal): Check for VERIFCERTERR to avoid SIGABRT
There is a bug that causes wget to exit with SIGABRT when trying to
receive files through FTP from a server with a certificate that failed
the verification.

The bug is filed in RedHat Bugzilla for Fedora:
https://bugzilla.redhat.com/show_bug.cgi?id=1475861

Reported-by: Artem Egorenkov <aegorenk@redhat.com>
2020-06-29 18:04:52 +02:00
Tim Rühsen
e830f5f42b * src/host.c (lookup_host): Fix uninitialized pointer access in c-ares code
Reported-by: Swapnil More
2020-06-21 11:37:28 +02:00
Tim Rühsen
470a7dfc84 * src/gnutls.c (ssl_init): Small cleanup fixing output of ncerts 2020-05-22 15:49:12 +02:00
Tim Rühsen
c23eaff56f * src/convert.c (downloaded_files_free): Only compile if DEBUG_MALLOC or TESTING is defined 2020-05-01 17:54:58 +02:00
Tim Rühsen
5a141065c4 * src/netrc.c (free_netrc): Only compile if DEBUG_MALLOC or TESTING is defined 2020-05-01 17:54:58 +02:00
Вячеслав Петрищев
7a3a82faf8 Fix SSL/TLS timeout issues.
* connect.c (fd_read, fd_peek): Let implementation take care about timeout.
* gnutls.c (_do_handshake, _do_reauth, wgnutls_read_timeout): Fix support for interactive timeout.
* gnutls.c (wgnutls_peek): Let wgnutls_read_timeout() take care about timeout.
* openssl.c (openssl_read_peek): Fix 0 (-1) timeout.
* retr.c (fd_read_body): Avoid wrong 'interactive timeout'.
2020-05-01 17:53:47 +02:00
Вячеслав Петрищев
c12a295496 Set interactive to true for bar progress.
* src/progress.c (bar_set_params): Set interactive to true.
* src/retr.c (fd_read_body): Avoid call fd_read with 0 timeout.
2020-05-01 17:53:21 +02:00
Вячеслав Петрищев
fb5e2e628d New implementation for openssl_read, openssl_peek ssl_connect_with_timeout (openssl)
Add new implementation for openssl_read, openssl_peek
ssl_connect_with_timeout (openssl). Thats allow continue
read after ETIMEDOUT (if timeout < opt.read_timeout)
without 'Retrying', and do not create thread (under MSWin)
for every read. Old implementation, (with fix for 'timeout')
avaible for build with: -DOPENSSL_RUN_WITHTIMEOUT

Add timeout for transport_implementation {reader,peeker};

* src/init.c (cmd_time): Add check for negative value.
* src/connect.h (transport_implementation {reader,peeker}): Change function prototype.
* src/connect.c (select_fd_nb): New function, aviod conversion to blocked under MSWin.
* src/gnutls.c
   (gnutls_read): Fix: using timeout, seting ETIMEDOUT, conversion to blocked.
   (gnutls_peek): Likewise.
   (wgnutls_errstr): Add errmsg for ETIMEDOUT.
   (_do_handshake) (_do_reauth): Fix conversion to blocked.
* src/openssl.c:
   Add new implementation for openssl_read, openssl_peek, ssl_connect_with_timeout.
   (init_prng): when option --random-file given warn user if RAND_load_file() fail.
2020-05-01 17:51:48 +02:00
Tim Rühsen
9dacc2836a Add missing linefeed in output strings
* src/ftp.c (getftp): Add linefeed to error message.
* src/res.c (res_parse_from_file): Likewise.
* src/utils.c (fopen_stat): Likewise.
  (open_stat): Likewise.
2020-04-18 18:39:50 +02:00
Tim Rühsen
6e5cf727c5 * src/metalink.c: Include filename.h instead of dosname.h 2020-03-28 19:26:12 +01:00
Tim Rühsen
78ef6fab13 * gnulib: Update 2020-03-28 19:23:35 +01:00
Tomas Hozza
706e71564c Don't print message about loading crl or ca-cert files with --no-verbose
* src/gnutls.c (ssl_init): Use LOG_VERBOSE verbosity for informative
  message related to loading CRL or CA certificate file.

Before change [1], wget didn't produce any output related to loading CA
certificates when --no-verbose option has been used. When --no-verbose
option is used, only error messages and basic information should get
printed. Information about loading CRL or CA certificate is probably not
a basic information. Any error when loading the CRL or CA certificate
will be still printed with --no-verbose.

Some users rely on wget not printing such information and they consider
it a regression.

Reported as https://bugzilla.redhat.com/show_bug.cgi?id=1807267

[1] http://git.savannah.gnu.org/cgit/wget.git/commit/?id=e4a8fe84e2b813b65d91aec29298eecabe4850a5

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2020-03-24 14:58:01 +01:00
Tim Rühsen
103aaf7740 Add NTLM fuzzer
* configure.ac: Create WITH_NTLM conditional.
* fuzz/Makefile.am: Add wget_ntlm_fuzzer.
* fuzz/wget_ntlm_fuzzer.c: New file.
* fuzz/wget_ntlm_fuzzer.in/*: Initial fuzz corpora.
2020-03-06 18:52:37 +01:00
Tim Rühsen
4221b5656a * src/http.c (check_auth): Fix memory leak 2020-03-06 12:09:02 +01:00
Tim Rühsen
fe5fbdd947 * src/http-ntlm.c (mkhash): Fix read buffer overflow 2020-03-06 11:37:57 +01:00
Tim Rühsen
7c017e604c * README.checkout: Fix gperf typo and tab indentation 2020-03-05 21:08:07 +01:00
Tim Rühsen
0a0177c694 * .gitlab-ci.yml: Generate online docs and coverage reports for Gitlab 2020-03-05 21:08:07 +01:00
Tim Rühsen
238074e2b8 Add code coverage reporting
* configure.ac: Add AX_CODE_COVERAGE.
* Makefile.am: Add rules 'check-coverage' and 'fuzz-coverage'.
* fuzz/Makefile.am: Amend LDADD, AM_CFLAGS and AM_CPPFLAGS.
* src/Makefile.am: Amend LDADD, AM_CFLAGS and AM_CPPFLAGS.

This add a new configure option, --enable-code-coverage.

With this option enabled, generate test code coverage with
  make code-coverage
and generate coverage for the fuzz test corpora with
  make fuzz-coverage
2020-03-05 20:24:07 +01:00
Tim Rühsen
6377ae1647 * src/http.c: Don't cast 3rd param to request_set_header. 2020-03-05 19:19:42 +01:00
Tim Rühsen
4e5a75566a * README: Fix typo 'can retrieves' -> 'can retrieve' 2020-03-05 19:10:55 +01:00
Darshit Shah
2db7fe688d * doc/wget.texi: Add maintainers and mention wget2 2020-03-03 20:50:05 +01:00
Darshit Shah
87815e223d doc/wget.texi: Clarify usage of --level 2020-03-03 20:10:51 +01:00
Tim Rühsen
eb02512a40 * src/http.c (gethttp): Remove unneeded free of hstat.(message|error) 2020-03-03 19:33:53 +01:00
Tim Rühsen
1ac3812e8a * src/http.c (gethttp): Fix memleaks
Reported-by: Вячеслав Петрищев
2020-03-03 18:38:41 +01:00
Tim Rühsen
910c0abe7a * src/http.c (read_response_body): Fix memleak
Reported-by: Вячеслав Петрищев
2020-03-03 10:26:09 +01:00
Tim Rühsen
be51eeceab * contrib/spell-checker: Add lightening to exceptions 2020-02-29 18:23:45 +01:00
Tim Rühsen
f4a57afd42 Always build ssl_cleanup
* src/gnutls.c (ssl_cleanup): Don't build conditionally.
* src/openssl.c (ssl_cleanup): Likewise
2020-02-29 17:25:41 +01:00
Tim Rühsen
3e2f450c6a * doc/wget.texi: Fix spelling error
Reported-by: Jim Cathey
2020-02-28 22:03:35 +01:00
Tim Rühsen
fded294d7a * src/convert.c (convert_cleanup): Free downloaded_css_set 2020-02-27 20:04:24 +01:00
Tim Rühsen
c0d76ec935 * src/openssl.c (ssl_cleanup): Only compile for DEBUG_MALLOC or TESTING 2020-02-27 19:59:15 +01:00
Tim Rühsen
472823283c * src/gnutls.c (ssl_cleanup): Only compile for DEBUG_MALLOC or TESTING 2020-02-27 19:59:05 +01:00
Tim Rühsen
b0e94e582b * src/netrc.c (netrc_cleanup): Only compile for DEBUG_MALLOC or TESTING 2020-02-27 19:56:40 +01:00
Tim Rühsen
8215479ffb * src/log.c (log_cleanup): Only compile for DEBUG_MALLOC or TESTING 2020-02-27 19:56:05 +01:00
Tim Rühsen
30139c6745 * src/host.c (host_cleanup): Only compile for DEBUG_MALLOC or TESTING 2020-02-27 19:55:31 +01:00
Tim Rühsen
4588045b28 * src/spider.c (spider_cleanup): Only compile for DEBUG_MALLOC or TESTING 2020-02-27 19:54:48 +01:00
Tim Rühsen
765271fdf5 * src/res.c (res_cleanup): Only compile for DEBUG_MALLOC or TESTING 2020-02-27 19:53:57 +01:00
Tim Rühsen
e52e418bdd * src/convert.c (convert_cleanup): Only compile for DEBUG_MALLOC or TESTING 2020-02-27 19:53:15 +01:00
Tim Rühsen
a3d3322a48 * src/html-url.c (cleanup_html_url): Only compile for DEBUG_MALLOC or TESTING 2020-02-27 19:51:55 +01:00
Tim Rühsen
ee187476de * src/http.c (http_cleanup): Free hashtable basic_authed_hosts 2020-02-27 19:33:47 +01:00
Tim Rühsen
8cd4dcc278 * src/cookies.c (cookie_jar_delete): Set psl to NULL after calling psl_free 2020-02-27 14:38:01 +01:00
Tim Rühsen
4d8bf37870 * src/init.c (cleanup): Call ssl_cleanup and connect_cleanup 2020-02-27 14:30:53 +01:00
Tim Rühsen
b209ca91fb Add ssl_cleanup to free memory
* src/ssl.h: Add declaration for ssl_cleanup.
* src/openssl.c: Add ssl_cleanup stub.
* src/gnutls.c: Implement ssl_cleanup.
2020-02-27 14:29:25 +01:00
Tim Rühsen
7146b9dc77 * src/http.c (http_cleanup): Call invalidate_persistent to free resources 2020-02-27 14:28:10 +01:00
Tim Rühsen
b3dc802efb * src/cookies.c: Free PSL in cookie_jar_delete 2020-02-27 14:27:05 +01:00
Tim Rühsen
6938c84aa4 * src/connect.c (connect_cleanup): Only needed with DEBUG_MALLOC or TESTING 2020-02-27 14:26:31 +01:00
Tim Rühsen
8010507d73 * src/http.c (http_loop): Avoid unneeded strdup 2020-02-27 11:35:50 +01:00
Вячеслав Петрищев
f747eea059 * src/http.c (gethttp): Fix memleak 2020-02-27 11:12:27 +01:00
Tim Rühsen
46a6e2957e * src/http.c: Do not reveal OS type to server via User-Agent: header
Following the "privacy by design" principle, which is now European law by
virtue of the GDPR since 2018, the client should not transmit the operating
system.

Fixes #57884
Reported-by: Bruno Haible
2020-02-23 18:51:10 +01:00
Tim Rühsen
04b7369490 * tests/unit-tests.c: Fix 'multiple definition of...' with gcc 10 2020-02-22 13:41:04 +01:00
Tim Rühsen
8e130c32ba Split unique_name into two functions with just one arg
* src/http.c (check_file_output): Call unique_name_passthrough.
* src/metalink.c (badhash_suffix): Call unique_name.
* src/url.c (url_file_name): Call unique_name_passthrough.
* src/utils.c: Split unique_name, amend description.
* src/utils.h: Amend unique_name, add unique_name_passthrough.
2020-02-22 12:59:18 +01:00
Tim Rühsen
88d841c651 * .gitlab-ci.yml: Set tag 'linux' explicitly for each runner 2020-02-22 08:53:56 +01:00
Tim Rühsen
d08bdbc878 * src/http.c (print_response_line): Fixed comparison
Introduced in 68868bbb37 (13.2.2020).

Reported-by: Gisle Vanem
2020-02-22 08:47:20 +01:00
Tim Rühsen
aba6e67eb7 Add valgrind suppressions
* testenv/valgrind-suppression-ssl: Add gnutls suppressions
* tests/valgrind-suppressions: Likewise
* tests/valgrind-suppressions-ssl: Likewise
2020-02-21 22:43:01 +01:00
Tim Rühsen
e1c4a2705f Add --gen-suppressions=all to valgrind options
* testenv/test/base_test.py: Add --gen-suppressions=all to valgrind options
* tests/Test-proxied-https-auth.px: Likewise
* tests/WgetTests.pm: Likewise
2020-02-21 22:42:22 +01:00
Tim Rühsen
f4dfddfaf3 * cfg.mk: Exclude sc_prohibit_gnu_make_extensions 2020-02-21 21:56:55 +01:00
Tim Rühsen
9934e228ac * src/retr.c (fd_read_body): Check timer before accessing (Coverity #600494) 2020-02-21 17:46:41 +01:00
Tim Rühsen
c65bddc086 * src/warc.c (warc_write_start_record): Check return values of fseek, fflush (Coverity #1419650) 2020-02-21 17:38:41 +01:00
Tim Rühsen
c5d0e58d0d * src/ftp-ls.c (eat_carets): Fix caret decoding issue (Coverity #1419653) 2020-02-21 14:55:03 +01:00
Tim Rühsen
365c00a4a2 * src/warc.c (warc_write_block_from_file): Check for write error (Coverity #1419654) 2020-02-21 14:41:40 +01:00
Tim Rühsen
4e519f5a80 * src/warc.c (warc_write_end_record): Check return value of fseeko (Coverity #1419657) 2020-02-21 14:37:40 +01:00
Tim Rühsen
08aa947c9d Add valgrind suppressions for HTTPS tests (GnuTLS false positive)
* testenv/valgrind-suppression-ssl: Add new rule
* tests/valgrind-suppressions: Likewise
* tests/valgrind-suppressions-ssl: Likewise
2020-02-21 13:20:45 +01:00
Tim Rühsen
43820b67bf * src/init.c (setoptval): Add missing comma (fix compilation) 2020-02-20 17:04:02 +01:00
Tim Rühsen
b8413cdd3d * .gitlab-ci.yml: Add spell-checking to VPATH/Debian 2020-02-20 16:24:43 +01:00
Tim Rühsen
b53c52c4c1 Fix typos found by codespell
*/*: Fix typos

Thanks to https://fossies.org/features.html#codespell

Reported-by: Jens Schleusener
2020-02-20 16:21:33 +01:00
Tim Rühsen
ce745c6a08 * contrib/spell-checker: Add new file 2020-02-20 16:20:52 +01:00
Tim Rühsen
9df2e0d746 * src/init.c (setoptval): Silence Coverity (Coverity #1419648) 2020-02-20 13:46:36 +01:00
Tim Rühsen
ada3396c72 * src/main.c (init_switches): Remove unneeded code (Coverity #1419658) 2020-02-20 13:20:21 +01:00
Tim Rühsen
9b1b9961ef * src/ftp-ls.c (ftp_parse_vms_ls): Fix use of uninitialized stack mem (Coverity #609373) 2020-02-20 13:17:31 +01:00
Tim Rühsen
79438d4d87 * src/retr.c (retrieve_url): Fix memory leak (Coverity #1316408) 2020-02-19 20:36:36 +01:00
Tim Rühsen
1c16819558 * src/http.c (set_file_timestamp): Fix stack memory leak (Coverity #1419655) 2020-02-19 19:13:48 +01:00
Tim Rühsen
d3f5311087 * src/main.c (main): Initialize variable (Coverity #1419660) 2020-02-19 18:54:06 +01:00
Tim Rühsen
4d1f6f5c09 * .gitlab-ci.yml: Fix linebreak 2020-02-19 18:44:14 +01:00
Tim Rühsen
1dfd1aa1cf * src/http.c (metalink_from_http): Fix memleak (Coverity #1419661, #1419656) 2020-02-19 17:07:52 +01:00
Tim Rühsen
2b48b50aa5 * .gitlab-ci.yml: Add 'coverity' runner 2020-02-19 17:06:06 +01:00
Tim Rühsen
e6c76e84aa * contrib/mk_authors: Create list of authors and translators since last version 2020-02-19 15:36:24 +01:00
Tim Rühsen
21bb9d54cb * src/progress.c (eta_to_human_short): Silence -Wformat-overflow 2020-02-19 14:44:21 +01:00
Tim Rühsen
8eda2da138 * .gitlab-ci.yml: Don't copy gnulib manually 2020-02-14 23:33:45 +01:00
Tim Rühsen
fa92fc336b * NEWS: Update 2020-02-14 16:31:59 +01:00
Tim Rühsen
e6b76459db Remove alloca includes and macros
* bootstrap.conf: Remove gnulib module 'alloca'.
* src/sysdep.h: Remove include of alloca.h.
* src/wget.h: Likewise,
  remove macros BOUNDED_TO_ALLOCA and STRDUP_ALLOCA.
2020-02-14 16:13:50 +01:00
Tim Rühsen
7aac99d6e5 * src/utils.h: Remove alloca_array 2020-02-14 16:09:36 +01:00
Tim Rühsen
28beee5e75 * src/convert.c (convert_links_in_hashtable): Remove use of alloca 2020-02-14 16:08:09 +01:00
Tim Rühsen
ca56ca94ee * src/convert.c (local_quote_string): Remove use of alloca 2020-02-14 16:00:05 +01:00
Tim Rühsen
951bcac6df * src/convert.c (replace_attr_refresh_hack): Remove use of alloca 2020-02-14 15:38:53 +01:00
Tim Rühsen
43ad3af58d * gnulib: Update 2020-02-14 15:31:10 +01:00
Tim Rühsen
a534d93a5f * bootstrap.conf: Use fnmatch-gnu instead of fnmatch 2020-02-14 15:30:14 +01:00
Tim Rühsen
6d1fcd1c48 * src/convert.c (write_backup_file): Remove use of alloca 2020-02-14 13:15:21 +01:00
Tim Rühsen
b62956872c * src/utils.c (fnmatch_nocase): Remove FNM_CASEFOLD emulation, gnulib takes care of it 2020-02-14 12:59:25 +01:00
Tim Rühsen
589b3743fa * src/cookies.c (cookie_header): Remove use of alloca 2020-02-14 12:50:32 +01:00
Tim Rühsen
d61d82c43c * src/cookies.c (cookie_header): Remove use of alloca 2020-02-14 12:43:12 +01:00
Tim Rühsen
f071030659 * src/cookies.c (cookie_handle_set_cookie): Remove use of alloca 2020-02-14 12:32:22 +01:00
Tim Rühsen
36ccb62e92 * src/cookies.c (parse_set_cookie): Explicitly convert from double to time_t 2020-02-14 12:04:41 +01:00
Tim Rühsen
e5cf56d7fa * src/cookies.c (parse_set_cookie): Remove use of alloca 2020-02-14 12:03:50 +01:00
Tim Rühsen
92c0d39894 * src/warc.c (warc_uuid_str): Add second param to windows_uuid_str 2020-02-14 11:53:10 +01:00
Tim Rühsen
78a848c69a * src/cookies.c (parse_set_cookie): Remove use of alloca 2020-02-14 11:48:19 +01:00
Tim Rühsen
8610b0b355 * src/main.c (main): Code clean, reduce allocations 2020-02-14 11:14:02 +01:00
Tim Rühsen
6bd21adabc * src/main.c: Modify macro IF_SSL to work with latest struct changes 2020-02-14 10:31:12 +01:00
Tim Rühsen
b92b628fd5 * configure.ac: List 'no' for --with-ssl in help output 2020-02-14 10:29:52 +01:00
Tim Rühsen
a274740598 * src/ftp.c (ftp_retrieve_dirs): Remove use of alloca 2020-02-13 20:19:26 +01:00
Tim Rühsen
94bae4dd18 * src/ftp.c (ftp_retrieve_list): Remove use of alloca 2020-02-13 20:11:45 +01:00
Tim Rühsen
d0695e269f * src/ftp.c (ftp_loop_internal): Remove use of alloca 2020-02-13 20:02:25 +01:00
Tim Rühsen
6a3e96275b * src/ftp.c (getftp): Remove use of alloca 2020-02-13 20:00:09 +01:00
Tim Rühsen
6d181cb8ef * src/http.c: Include xstrndup.h unconditionally 2020-02-13 19:34:36 +01:00
Tim Rühsen
89e09e59f2 * src/ftp-basic.c (ftp_request): Remove use of alloca 2020-02-13 17:10:01 +01:00
Tim Rühsen
5505aced03 * src/html-parse.c (name_allowed): Remove use of alloca 2020-02-13 17:01:29 +01:00
Tim Rühsen
6cc9acfd61 * src/netrc.c (search_netrc): Remove use of alloca 2020-02-13 16:55:41 +01:00
Tim Rühsen
e91e96ccb5 * src/init.c (parse_line): Remove use of alloca 2020-02-13 16:50:21 +01:00
Tim Rühsen
c3c04ffeba Remove use of alloca in setoptval
* src/init.c (setoptval): Remove use of alloca
* src/init.h: Add define MAX_LONGOPTION
* src/main.c (struct cmdline_option): Make 'long_name' a char array
2020-02-13 16:39:36 +01:00
Tim Rühsen
cbea3d41dd * src/http.c (check_auth): Remove use of alloca 2020-02-13 16:27:38 +01:00
Tim Rühsen
68868bbb37 * src/http.c (print_response_line): Remove use of alloca 2020-02-13 16:11:51 +01:00
Tim Rühsen
a3f2ba708d * src/http.c (request_set_user_header): Remove use of alloca 2020-02-13 16:11:23 +01:00
Tim Rühsen
f1ce7f5bd6 * src/http.c (metalink_from_http): Remove use of alloca 2020-02-13 16:00:19 +01:00
Tim Rühsen
704208b766 * src/http.c (basic_authentication_encode): Remove use of alloca 2020-02-13 15:54:16 +01:00
Tim Rühsen
0bc0729bcc * src/http.c (gethttp): Remove use of alloca 2020-02-13 15:40:48 +01:00
Tim Rühsen
572f6f9b39 * src/http.c (gethttp): Remove use of alloca 2020-02-13 15:34:13 +01:00
Tim Rühsen
e22dbe5af0 * src/http.c (set_file_timestamp): Remove use of alloca 2020-02-13 15:30:50 +01:00
Tim Rühsen
f460e1d049 * src/main.c (main): Remove use of alloca 2020-02-13 15:17:27 +01:00
Tim Rühsen
c0d8cceb93 * src/url.c (append_uri_pathel): Replace alloca by fixed array / sprintf 2020-02-13 15:04:17 +01:00
Tim Rühsen
c2d03b6293 * src/url.c (append_uri_pathel): Replace alloca by fixed array / xmalloc 2020-02-13 13:32:16 +01:00
Tim Rühsen
9adde1e441 * src/utils.c (make_directory): Replace alloca by fixed array / xmalloc 2020-02-13 12:09:34 +01:00
Tim Rühsen
4dd9dee2e7 * src/utils.c (unique_name_1): Replace alloca by xmalloc 2020-02-13 11:48:06 +01:00
Tim Rühsen
233f982f9d Add buffer size param to warc_uuid_str
* src/http.c (gethttp): Add buffer size param to warc_uuid_str.
* src/warc.c: Likewise
* src/warc.h: Likewise
2020-02-13 11:28:18 +01:00
Tim Rühsen
e5d0dda561 * src/http-ntlm.c (ntlm_output): Remove alloca 2020-02-12 17:05:01 +01:00
Tim Rühsen
17fad78290 * src/http-ntlm.c (mkhash): Replace alloca by fixed length array 2020-02-12 16:56:57 +01:00
Tim Rühsen
1ebd88063d * src/http-ntlm.c (ntlm_input): Replace alloca by fixed length array 2020-02-12 16:44:59 +01:00
Tim Rühsen
c65c23cfc7 * src/gnutls.c (ssl_init): Replace alloca by fixed length arrays 2020-02-12 16:24:45 +01:00
Tim Rühsen
94b9332f50 * src/retr.c (fd_read_body): Fix signedness warnings 2020-02-12 16:14:40 +01:00
Tim Rühsen
fd75f5d986 * src/retr.c (rotate_backups): Replace alloca by fixed length arrays 2020-02-12 16:13:40 +01:00
Tim Rühsen
473db08580 * src/wget.h: Add inline to _unhex 2020-02-12 16:12:09 +01:00
Tim Rühsen
0bb97fecb7 * src/metalink.h: Include dirname.h, remove declaration of last_component 2020-02-12 16:10:58 +01:00
Tim Rühsen
5debe0a96f * configure.ac: Add -Wno-undef -Wno-float-equal to gcc's WARN_CFLAGS 2020-02-12 16:09:57 +01:00
Tim Rühsen
8bc1d1447b * src/convert.c (convert_links): Fix previous commit 2020-02-12 10:52:06 +01:00
Tim Rühsen
fe4ade75cf * src/convert.c (convert_links): Add \n (Fixes #57795) 2020-02-12 09:57:27 +01:00
Ander Juaristi
928c54d27e Do not overwrite restval if len is smaller
* src/http.c (http_loop): overwrite 'restval' only if 'len' is
   greater than it. Else substract 'len' to 'restval'.

When retrying a request, only send a 'Range' header if the previous
request returned data.
2020-01-25 18:35:05 +01:00
Вячеслав Петрищев
33bc3aae51 Fix and cleanup progress bar code
*src/progress.c
 (struct dot_progress) accumulated, rows: Type changed to wgint
 (print_row_stats): Fix missing unit name 'T'
 (dot_update): Add ability to reduce dot_draw runtime
 (bar_update): Avoid integer overflow
2020-01-24 19:41:52 +01:00
Вячеслав Петрищев
d5dc68b1a9 * src/progress.c: Fix MINIMUM_SCREEN_WIDTH 2020-01-24 19:41:45 +01:00
Вячеслав Петрищев
68548acf0f * src/progress.c (prepare_filename): Replace zero-width mbc (\xe2\x80\x8b, ...) 2020-01-24 19:41:36 +01:00
Вячеслав Петрищев
ec68da677f Reenabling filename scrolling code in progress bar
* src/progress.c: Reenabling filename scrolling code
  (bar_create):  Reenabling file name scrolling.
  (bar_create):  Memory allocation, filename preparation.
  (bar_finish):  Filename  memory deallocation.
  (prepare_filename):  New function, replaces unprintable chars.
2020-01-24 19:39:51 +01:00
Вячеслав Петрищев
f7a1e6cd7b Fix multibyte handling in progress bar
* src/progress.c
  (count_cols): Fix return value if invalid multibyte sequence was encountered
  (bar_draw): Fix xrealoc size
2020-01-24 19:38:52 +01:00
Tim Rühsen
e2c0c2fbe5 * src/progress.c (print_row_stats): Fix two integer overflows 2020-01-10 13:56:39 +01:00
Tim Rühsen
bc7e82d3e9 * .gitlab-ci.yml: Fix path for llvm-symbolizer 2020-01-10 13:56:39 +01:00
Tim Rühsen
c1a46e9a3b * fuzz/wget_progress_fuzzer.in/*: Add fuzzer corpora 2020-01-10 13:56:39 +01:00
Tim Rühsen
f27daa1c17 * src/retr.c (calc_rate): Remove unwanted assertions 2020-01-10 13:56:39 +01:00
Tim Rühsen
07eebd2a20 Fix buffer overflows in progress 'bar' code
* src/progress.c (progress_interactive_p): Sanitize input.
  (progress_update): Likewise.
  (bar_create): Use larger BUF_LEN.
  (bar_create): Remove superfluous memset.
  (bar_create): Fix filename layout.
  (bar_create): Remove filename scrolling code, it caused many buffer
  overflows later in bar_create.
  (bar_create): Support TB/s download speed.
2020-01-10 13:56:39 +01:00
Tim Rühsen
34573bdf46 Add new fuzzer for the progress bar code
* Makefile.am: Add wget_progress_fuzzer.
* wget_progress_fuzzer.c: New file.
2020-01-10 13:56:39 +01:00
Darshit Shah
221f1ba3ef Update copyright year to 2020 2020-01-10 12:56:08 +01:00
Darshit Shah
c7fdfe3d60 * cfg.mk: Set update-copyright to use intervals 2020-01-10 12:52:36 +01:00
Tim Rühsen
6bd74e33d6 Fix segfault in progress bar in certain locales
* src/progress.c (create_image): Protect memset from negative count

Reported-by: JunDong Xie
2019-12-27 13:43:22 +01:00
Tim Rühsen
042f2727bf * src/progress.c (count_cols): Fix return value if USE_NLS_PROGRESS_BAR is undefined
Taking patch from #54126
Reported-by: Vyacheslav
Copyright-paperwork-exempt: Yes
2019-12-27 13:42:58 +01:00
Tim Rühsen
ce8ce5bfc0 * src/progress.c: Allow const names for set_progress_implementation. 2019-12-27 13:42:30 +01:00
Tim Rühsen
61b8078672 * src/progress.c (dot_draw): Avoid integer overflow 2019-12-27 13:42:01 +01:00
Tim Rühsen
03a8e59d85 * fuzz/README.md: Add --enable-assert to fuzzing build 2019-12-27 13:41:20 +01:00
Tim Rühsen
abe1ab1916 * src/progress.c (print_row_stats): Fix UB if eta < 0 2019-12-27 13:41:01 +01:00
Tim Rühsen
542524855a * src/progress.c (dot_update, dot_finish): Sanitize input 2019-12-27 13:40:52 +01:00
Tim Rühsen
f5d1dcf718 * src/retr.c (calc_rate): Fix division by 0 2019-12-27 13:40:33 +01:00
Tim Rühsen
1c1cba3a43 * src/retr.c (calc_rate): Add TB/s 2019-12-27 13:39:55 +01:00
Tim Rühsen
0179138fe5 * src/progress.c (create_image): Sanitize input param 'dl_total_time' 2019-12-27 13:39:50 +01:00
Tim Rühsen
ed135257b2 * fuzz/README.md: Update clang instructions 2019-12-27 13:39:11 +01:00
Lauri Nurmi
dd710bde96 Mark note to translators properly and elaborate its contents.
* src/progress.c (create_image): Likewise

The note was not even picked by xgettext to be included in the .pot,
so barely any translators even saw the note.
Also, this is one of the most cryptic messages to translate in wget,
unless the translator happens to guess where the string is used,
or looks at the source.

Copyright-paperwork-exempt: Yes
2019-12-26 19:02:36 +01:00
Lauri Nurmi
dc8bbdbb05 * src/main.c (print_help): Don't cut translatable string mid-sentence.
Complicates translating.

Copyright-paperwork-exempt: Yes
2019-12-26 19:00:11 +01:00
Tim Rühsen
777049389f * fuzz/wget_read_hunk_fuzzer.c: Use cast to fix C++ error 2019-11-24 16:33:34 +01:00
Tim Rühsen
96d74948a3 * src/Makefile.am: Add build_info.c to EXTRA_DIST 2019-11-21 12:16:28 +01:00
Tim Rühsen
b1f566719c * .gitlab-ci.yml: Skip tests on tarball build due to missing perl 2019-11-21 12:16:04 +01:00
Tim Rühsen
0751d053f5 * tests/WgetTests.pm: Support to set the wget executable for testing 2019-11-21 11:24:48 +01:00
Tim Rühsen
4cdda68752 * .gitlab-ci.yml: Add --with-ssl=no 2019-11-21 11:01:27 +01:00
Tim Rühsen
48f762a648 * .gitlab-ci.yml: Allow failure for Valgrind and Scan-Build CI runner 2019-11-21 10:56:10 +01:00
Tim Rühsen
feb52bfab7 * src/ftp.c (delelement): Make sure free'd pointer is set to NULL 2019-11-21 10:56:10 +01:00
Tim Rühsen
1656255767 * configure.ac: Remove -fno-sanitize-recover=integer
With this option on, several overflows from gnulib code (all
harmless and on purpose) would break our tests.
2019-11-21 10:56:10 +01:00
Tim Rühsen
2ba8da8f22 Replace XDIGIT_TO_NUM by _unhex() with proper unsigned handling
* src/wget.h: Replace XDIGIT_TO_NUM by _unhex()
* src/html-parse.c (decode_entity): Use _unhex()
* src/host.c (is_valid_ipv6_address): Use _unhex()
* src/url.c (url_unescape_1): Use unsigned char to avoid UB
* src/res.c (free_specs): Likewise

Code taken from Wget2. XDIGIT_TO_NUM had a signed issue, detected by
scan-build static anylyzer.
2019-11-21 10:56:10 +01:00
Tim Rühsen
a255d9f0c8 * src/connect.c (select_fd): Check for negative fd 2019-11-21 10:56:10 +01:00
Tim Rühsen
823a9c8e0d * .gitlab-ci.yml: Enable Windows executables for MinGW runner 2019-11-21 10:56:10 +01:00
Tim Rühsen
57ac97e6d8 * .gitlab-ci.yml: Add CI runners from Wget2 2019-11-21 10:56:10 +01:00
Tomas Hozza
bac6fbab6c testenv: enable running tests on different wget binary
* testenv/README: Document $WGET_PATH
* testenv/test/base_test.py: Use $WGET_PATH instead ../src/wget if set

Previously tests in testenv/ directory were run only on wget binary
which was built from sources in src/ directory. However as a
wget maintainer in a Linux distribution, I would like to be able to run
upstream tests on the wget binary distributed with the distribution.

This change enables one to define WGET_PATH environment variable
to a path to wget binary which should be used by tests.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2019-11-21 10:50:34 +01:00
Tim Rühsen
1e4262e98d * AUTHORS: Add Darshit Shah and Tim Rühsen as maintainers 2019-11-21 10:49:56 +01:00
Tim Rühsen
6f3baab02c * src/cookie.c: Fix cookie file header to be recognized by 'file' 2019-11-19 12:37:46 +01:00
Tim Rühsen
b4dba956ce * .mailmap: Allow mapping contributors for statistics 2019-11-19 12:37:46 +01:00
Darshit Shah
f3b25875ab * configure.ac: Ignore -Wchkp which is deprecated 2019-10-03 13:49:32 +02:00
Darshit Shah
cd21cb3812 Update gnulib
* gnulib: Pull submodule
* bootstrap: Update file from gnulib
2019-10-03 13:45:02 +02:00
Tim Rühsen
b3f86f90cc Revert "Add . to perl path for all perl tests"
This reverts commit 79be99aff4.

/usr/bin/env 8.28 and below doesn't know -S.
2019-05-30 12:06:01 +02:00
Darshit Shah
60dc09b507 Revert " Hi, Thank you again Darshit for your response. The RejectHeaderField rule rejects ANY header"
I accidentally commited this patch.

This reverts commit 4ce2f93600.
2019-05-30 11:19:46 +02:00
Darshit Shah
02c059a857 * .gitignore: Update gitignore file 2019-05-30 11:19:15 +02:00
sulfastor
4ce2f93600 Hi, Thank you again Darshit for your response. The RejectHeaderField rule rejects ANY header
of the header field while RejectHeader rejects ONLY the specified full header.
	Since we wanted to be sure a header field is not sent to the server we wrote this rule.

	* doc/wget.texi: Added --disable-header documentation.
        * fuzz/wget_options_fuzzer.dict: Update with --disable-header inputs.
        * src/http.c (disabled_header): Checks for disabled headers
	(request_set_header): Doesn't let header to be set if disabled
	(gethttp): frees disabled header to let overriding
        * src/init.c (cmd_dis_header), (check_user_disabled_header) added new option disabled_headers.
        * src/main.c: added new option --disable-header, added help description
        * src/options.h: added new option --disable-header
	* src/utils.h (vec_remove_header)
	* src/utils.c (vec_remove_header) removes all header instances from vector
        * testenv/Makefile.am: Added new test files
        * testenv/server/http/http_server.py: Added new rule RejectHeaderField
        * testenv/conf/reject_header_field.py: Added new rule RejectHeaderField
        * testenv/README: Added help description for new rule
        * testenv/Test-disable-default-headers.py: Test without using --header
        * testenv/Test-disable-headers-after.py: Test using --header before --disable-header
        * testenv/Test-disable-headers-before.py: Test using --header after --disable-header

Signed-off-by: sulfastor <torresmoisesa@gmail.com>, adham elkarn <adhamelkarn@hotmail.com>
2019-05-30 10:01:24 +02:00
Tim Rühsen
0cdbc715a9 * src/log.c (log_vprintf_internal): Don't log twice 2019-05-28 16:26:03 +02:00
AviSoomirtee
95a677c7f5 * src/openssl.c (ssl_init): post handshake auth for OpenSSL
Copyright-paperwork-exempt: Yes
2019-05-25 18:10:37 +02:00
Tim Rühsen
96f48bf412 * bootstrap.conf: Explicitly set source_base and test_base 2019-05-24 11:14:19 +02:00
Tim Rühsen
95de9cd97f * src/convert.c (convert_basename): Do not pass NULL to strrchr() 2019-05-07 17:06:04 +02:00
Tim Rühsen
fac8f8099d * src/url.c (append_uri_pathel): Check for possible NULL dereference 2019-05-07 17:05:55 +02:00
Tim Rühsen
7da620c7ce * fuzz/main.c: Fix paths for WIN32 2019-05-07 17:05:32 +02:00
Tim Rühsen
e3790ad2a3 * bootstrap.conf: Work around VPATH issue 2019-05-07 17:05:26 +02:00
Tim Rühsen
021458ddf2 * configure.ac: Update gettext version to 0.19.3 2019-05-07 17:05:10 +02:00
Tim Rühsen
64618a9852 * bootstrap.conf: Remove rsync as prerequisite
'./bootstrap --skip-po' won't need wget nor rsync
2019-05-07 17:05:05 +02:00
Tim Rühsen
c9fa9d599e * bootstrap.conf: Replace prerequisite xz by gzip 2019-05-07 17:04:56 +02:00
Tim Rühsen
64f6f6847e Add sanitizer flags for ./configure
* bootstrap.conf: Add 'warning' gnulib module
* configure.ac: Add --enable-fsanitize-* flags
* tests/Makefile.am: Add compiler flags for unit testing
2019-05-03 11:27:30 +02:00
Tim Rühsen
94e2dd5ecd Add --enable-manywarnings from wget2
* .gitignore: Exclude /lib/Makefile.am
* bootstrap.conf: Use --makefile-name=gnulib.mk with gnulib-tool
* configure.ac: Use wget_MANYWARNINGS()
* lib/Makefile.am: Define empty noinst_LIBRARIES and MAINTAINERCLEANFILES
* m4/wget_manywarnings.m4: New file from GNU Wget2
* src/Makefile.am: Use WARN_CFLAGS for AM_CFLAGS
2019-05-02 12:53:19 +02:00
Tim Rühsen
14dc0c6af0 Add CONTRIBUTING.md
* CONTRIBUTING.md: New file for contributors
* README: Mention CONTRIBUTING.md
2019-04-28 20:36:14 +02:00
Tim Rühsen
6edf7a30b6 * fuzz/*.in/*: Update corpora from OSS-fuzz 2019-04-26 14:47:17 +02:00
Tim Rühsen
c34e9a8c47 * fuzz/get_ossfuzz_corpora: Remove -f from unzip, mkdir *.in/ 2019-04-26 14:14:12 +02:00
Tim Rühsen
370b778db2 * .gitlab-ci.yml: Update Gitlab CI WORKDIR to /usr/local 2019-04-26 11:04:03 +02:00
Eneas U de Queiroz
14e3712b8c * src/openssl.c: fix ssl_init for openssl 1.1.1
ssl_init fails with openssl 1.1.1 when openssl.cnf is not found.
Redundant calls to intialization functions were removed as
OPENSSL_config takes care of them for openssl versions < 1.1.0.
For versions > 1.1.0, OPENSSL_init_ssl is preferred.

Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Copyright-paperwork-exempt: Yes
2019-04-26 10:33:49 +02:00
Leif Ryge
263546473f * doc/wget.texi: Fix typo in man page
Copyright-paperwork-exempt: Yes
2019-04-11 14:47:30 +02:00
raminfp
4574771a6a [log] miss check If logging is inhibited 2019-04-11 08:50:42 +00:00
Tim Rühsen
562eacb76a * src/iri.c (do_conversion): Remove unneeded debug lines 2019-04-05 13:01:57 +02:00
Darshit Shah
a220ead435 * NEWS: Update NEWS for new release 2019-04-05 11:59:19 +02:00
Tim Ruehsen
692d5c5215 Fix a buffer overflow vulnerability
* src/iri.c(do_conversion): Reallocate the output buffer to a larger
  size if it is already full
2019-04-05 11:54:30 +02:00
Darshit Shah
2ae601a719 * NEWS: Update NEWS for new release 2019-04-05 11:54:30 +02:00
Tim Rühsen
0eaa5f1771 * src/openssl.c (ssl_init): Check for X509_V_FLAG_PARTIAL_CHAIN 2019-04-03 14:54:45 +02:00
Darshit Shah
39f17b3555 * gnulib: Pull forward 2019-04-01 20:15:05 +02:00
Tim Rühsen
4c89c85a28 * fuzz/main.c (test_all_from): Fix indentation 2019-03-26 09:42:50 +01:00
Tim Rühsen
5d87635c66 Fix corner case in processing server response
* src/http.c (response_head_terminator): Don't access uninitialized data
* fuzz/wget_read_hunk_fuzzer.c: Sync response_head_terminator()
2019-03-25 16:05:56 +01:00
Tim Rühsen
4046cd2a71 Add new fuzzer wget_read_hunk_fuzzer.c
* fuzz/Makefile.am: Add wget_read_hunk_fuzzer
* fuzz/wget_read_hunk_fuzzer.c: New file
* fuzz/wget_read_hunk_fuzzer.in/*: Fuzz corpora
* src/connect.c: Add connect_cleanup()
* src/connect.h: Add prototype for connect_cleanup()
2019-03-25 14:57:07 +01:00
Tim Rühsen
ae6636a28f * fuzz/wget_netrc_fuzzer.c: Fix fuzzer 2019-03-03 17:05:58 +01:00
Tim Rühsen
acef0fb4c6 * fuzz/wget_ftpls_fuzzer.c: Fix fuzzer 2019-03-03 17:05:44 +01:00
Jeffrey Walton
7c1c8eb3b1 * src/openssl.c (ssl_init): Trust partial cert chain 2019-02-22 12:46:53 +01:00
Tim Rühsen
19661f1d9a * src/ftp-ls.c (ftp_parse_vms_ls): Use snprintf instead of strcpy/strcat 2019-02-20 10:32:47 +01:00
Darshit Shah
e39be32838 * src/html-url.c(get_urls_html_fm): Add message in verbose mode with no-follow attribute 2019-02-19 23:25:05 +01:00
Tim Rühsen
8d8abcffdf Fix fuzz/ tests for OpenBSD
* fuzz/wget_*_fuzzer.c: Take care when calling exit()
2019-02-19 17:18:49 +01:00
Tim Rühsen
93d21e4217 * fuzz/Makefile.am: Remove hard-coded gcc flags 2019-02-19 15:52:58 +01:00
Tim Rühsen
07f9fbd0b3 Fix STDERR closing/restoring in fuzzers
* fuzz/fuzzer.h: Add CLOSE_STDERR and RESTORE_STDERR
* fuzz/wget_*_fuzzer.c: Use CLOSE_STDERR and RESTORE_STDERR
2019-02-19 15:50:34 +01:00
Tim Rühsen
74866d9ea0 * configure.ac: Rearrange AM_ICONV before gl_INIT 2019-02-19 15:49:41 +01:00
Tim Rühsen
1098de2b49 Do not hard-code -ldl in fuzz/Makefile.am
* configure.ac: Search for dlopen and add library to $FUZZ_LIBS
* fuzz/Makefile.am: Link with $FUZZ_LIBS instead of -ldl

This fixes linking on BSD systems.

Reported-by: Nam Nguyen
2019-02-19 15:11:04 +01:00
Tim Rühsen
c7f31d62b2 * doc/wget.texi: Correct --logfile -> --output-file 2019-02-14 10:39:54 +01:00
Darshit Shah
c53552e8cd Update gnulib 2019-02-10 11:51:03 +01:00
Darshit Shah
8b975e83d3 Update copyright statements 2019-02-10 11:50:17 +01:00
Leon Klingele
9e7c1554bd docs: --no-cache also sets the 'Cache-Control: no-cache' header
* doc/wget.texi: Add Cache-Control to docs
* src/wget.h: Add Cache-Control to comment of SEND_NOCACHE

Copyright-paperwork-exempt: Yes
2019-01-23 22:04:01 +01:00
Tim Rühsen
caf30fbe28 * .gitlab-ci.yml: Add minimal build 2019-01-20 19:58:10 +01:00
Tim Rühsen
f1eda916a6 * src/init.c (cleanup): Check HAVE_HSTS
Reported-by: Simon Dales
2019-01-20 19:42:53 +01:00
André Wolski
5dceb6626b NTLM restart authentication (trivial change)
* src/http-ntlm.c (ntlm_input): Continue on NTLMSTATE_LAST,
  error on NTLMSTATE_TYPE3

The code comes from the cURL project. Thanks to Daniel Stenberg
for donating the code.

https://lists.gnu.org/archive/html/bug-wget/2018-12/msg00030.html
2018-12-31 10:24:27 +01:00
Tim Rühsen
fc65b4521a * contrib/spell-checker: Remove trailing whitespace 2018-12-28 19:05:42 +01:00
Tim Rühsen
fdb7fe613c Fix typos detected by codespell (via contrib/spell-checker) 2018-12-28 18:58:15 +01:00
Tim Rühsen
c0427ebf3c * contrib/spell-checker: Add script for spell checking 2018-12-28 18:55:20 +01:00
Tim Rühsen
af16b0f38f * fuzz/Makefile.am: Fix order of libraries for linking 2018-12-27 20:48:36 +01:00
Tim Rühsen
6ba1cb587c * src/wget.h: #undef _Noreturn when building with C++ 2018-12-27 20:47:58 +01:00
Tim Rühsen
5811c2222b * src/gnutls.c (ssl_connect_wget): Fix call to gnutls_set_default_priority() 2018-12-27 20:46:55 +01:00
Tim Rühsen
70cddf3e29 * NEWS: Add release changes for 1.20.1 2018-12-26 20:35:41 +01:00
Tim Rühsen
3cdfb594cf Don't save user/pw with --xattr
Also the Referer info is reduced to scheme+host+port.

* src/ftp.c (getftp): Change params of set_file_metadata()
* src/http.c (gethttp): Change params of set_file_metadata()
* src/xattr.c (set_file_metadata): Remove user/password from origin URL,
  reduce Referer value to scheme/host/port.
* src/xattr.h: Change prototype of set_file_metadata()
2018-12-26 14:38:24 +01:00
Tim Rühsen
c125d24762 Don't use extended attributes (--xattr) by default
* src/init.c (defaults): Set enable_xattr to false by default
* src/main.c (print_help): Reverse option logic of --xattr
* doc/wget.texi: Add description for --xattr

Users may not be aware that the origin URL and Referer are saved
including credentials, and possibly access tokens within
the urls.
2018-12-26 14:06:38 +01:00
Tim Rühsen
7c0a30d724 * .travis.yml: Email to wget-dev instead bug-wget mailing list 2018-12-13 15:22:17 +01:00
Darshit Shah
3becef17cb * NEWS: Prepare for new version 2018-11-30 01:11:19 +01:00
Darshit Shah
8f780fa274 * contrib/make-release: Add a small checklist for pending tasks 2018-11-13 17:44:51 +01:00
Darshit Shah
e55916e5de Prepare NEWS for new release 2018-11-13 16:24:17 +01:00
Darshit Shah
c4a6b89b2e * configure.ac: gnulib now expects autoconf >=2.63 2018-11-13 16:20:19 +01:00
Darshit Shah
3d4e1f47a1 * gnulib: Update library 2018-11-13 16:20:19 +01:00
Jay Satiro
61271d87f6 * src/init.c: Stop freeing the pointer returned by ws_mypath()
.. since ws_mypath() saves the address it returns in a static pointer
for reuse, to also be returned in later calls.
2018-11-13 15:51:51 +01:00
Darshit Shah
2bc2d2f803 * src/ftp.c(ftp_retrieve_glob): Honor {accept,reject}-regex switches as well 2018-11-13 15:51:51 +01:00
Darshit Shah
8c741da256 * src/ftp.c (ftp_retrieve_glob): Refactor to prevent looping over listing multiple times 2018-11-13 15:51:51 +01:00
Tim Rühsen
d3383d619b * .gitlab-ci.yml: Split into GnuTLS and OpenSSL build 2018-11-11 18:43:46 +01:00
Tim Rühsen
62dced2f43 * Makefile.am: dist clean po/stamp-po 2018-11-11 17:51:37 +01:00
Tim Rühsen
04116409a8 Remove auto-generated files from po/ 2018-11-11 16:58:09 +01:00
Tim Rühsen
7b8325b416 Add VPATH build 2018-11-11 16:57:41 +01:00
Tim Rühsen
11fad3fa72 Revert "Bail out on unexpected 416 server errors"
This reverts commit 6f3b995993.

The code is obviously wrong, see https://savannah.gnu.org/bugs/?54963
Also, the example from the original post doesn't work any more.
With other words, the broken server behavior has been fixed meanwhile.
2018-11-09 16:16:43 +01:00
Rosen Penev
a3643c6076 openssl: Do not use engines when OpenSSL does not support
* src/openssl.c: Check for OPENSSL_NO_ENGINE before
 including openssl/engine.h and before calling ENGINE_load_builtin_engines()

Fixes compilation with no engines compiled.

Copyright-paperwork-exempt: Yes
Signed-off-by: Rosen Penev <rosenp@gmail.com>
2018-11-09 16:01:51 +01:00
Tim Rühsen
ed23e682da Fix HTTPS Perl tests
* tests/SSLTest.pm: Rename server cert and key file
* tests/Test-https*.px: Fix and remove OpenSSL hard-coding
* tests/certs/create-certs.sh: Script to generate test files
* tests/certs/*-template.txt: GnuTLS template files for certs and crl
* tests/certs/*.pem: Keys, certs, crls
* tests/certs/README: Removed commands, link to create-certs.sh
2018-11-09 15:47:53 +01:00
Kapus, Timotej
6d7cd9313c Replace some loops with string.h functions
* src/init.c: Replace loop with strspn
* src/url.c: Replace loop with strrchr

Copyright-paperwork-exempt: Yes
2018-10-28 10:36:46 +01:00
Luiz Angelo Daros de Luca
7903dbc9d8 * .gitmodules: Use https:// instead of git:// for gnulib
git:// does not work over http proxy

Signed-off-by: Luiz Angelo Daros de Luca <luizluca@gmail.com>
Copyright-paperwork-exempt: Yes
2018-10-26 22:55:44 +02:00
Luiz Angelo Daros de Luca
fd85ac9cc6 * src/host.c (sufmatch): Fix dot-prefixed domain matching
Current sufmatch does not match when domain is dot-prefixed.
The example of no_proxy in man (.mit.edu) does use a dot-prefixed
domain.

Signed-off-by: Luiz Angelo Daros de Luca <luizluca@gmail.com>
Copyright-paperwork-exempt: Yes
2018-10-26 22:54:26 +02:00
Tim Rühsen
21daa24e72 * src/convert.c (convert_links): Fix fallthrough 2018-10-26 22:52:41 +02:00
Darshit Shah
6d6679ba86 * bootstrap: Update script from gnulib 2018-10-22 16:45:58 +02:00
Darshit Shah
d18f78b86b * gnulib: Update library 2018-10-22 16:45:58 +02:00
Tim Rühsen
85310bfa58 * .lgtm.yml: New file to add LGTM to Gitlab.com CI 2018-10-19 12:32:01 +02:00
Tim Rühsen
b29854528a * configure.ac: Fix build issue with libgpgme 2018-10-16 10:31:54 +02:00
Tim Rühsen
3d518f125c * fuzz/*_fuzzer.in/*: Update fuzzer corpora 2018-10-14 15:15:49 +02:00
Nikos Mavrogiannopoulos
c11cc83d9e Enable post-handshake auth under gnutls on TLS1.3 2018-10-08 15:55:48 +02:00
Tim Rühsen
0727b8f3a9 * src/http.c (resp_new): Fix code to avoid false positive by clang 2018-09-20 14:59:06 +02:00
Tim Rühsen
88a49c1e41 * src/convert.c (convert_links): Fix code to avoid false positive by clang 2018-09-20 14:58:27 +02:00
Tim Rühsen
02afe1e41c Add support for PCRE2 pattern matching
* configure.ac: Check for libpcre2-8
* src/init.c (choices): Test for HAVE_LIBPCRE2
* src/main.c (main): Set regex compile and match functions
* src/options.h: Test for HAVE_LIBPCRE2
* src/utils.c: Include pcre2.h, add functions
  compile_pcre2_regex() and match_pcre2_regex()
* src/utils.h: Declare compile_pcre2_regex() and match_pcre2_regex()

Fixes #54677
Reported-by: Noël Köthe
2018-09-19 16:22:25 +02:00
Tim Rühsen
79be99aff4 Add . to perl path for all perl tests
* tests/*.px: Add -I . to the shebang

This allows perl test to be run from tests/ directory, e.g. via
  ./Test--post-file.px
2018-09-07 10:48:14 +02:00
Tomas Hozza
2bbdfd76da Add TLS 1.3 support for GnuTLS
* doc/wget.texi: Add "TLSv1_3" to --secure-protocol
* src/gnutls.c (set_prio_default): Use GNUTLS_TLS1_3 where needed

Wget currently allows specifying "TLSv1_3" as the parameter for
--secure-protocol option. However it is only implemented for OpenSSL
and in case wget is compiled with GnuTLS, it causes wget to abort with:
GnuTLS: unimplemented 'secure-protocol' option value 6

GnuTLS contains TLS 1.3 implementation since version 3.6.3 [1]. However
currently it must be enabled explicitly in the application of it to be
used. This will change after the draft is finalized. [2] However for
the time being, I enabled it explicitly in case "TLSv1_3" is used with
--secure-protocol.

I also fixed man page to contain "TLSv1_3" in all listings of available
parameters for --secure-protocol

[1] https://lists.gnupg.org/pipermail/gnutls-devel/2018-July/008584.html
[2] https://nikmav.blogspot.com/2018/05/gnutls-and-tls-13.html

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2018-09-07 09:56:02 +02:00
Tomas Korbar
7ddcebd61e Avoid creating empty wget-log when using -O and -q in background
* src/log.c (check_redirect_output): Check for quiet mode
2018-08-29 12:34:03 +02:00
Tomas Hozza
2f451dbf4e * src/warc.c (warc_write_cdx_record): Fix RESOURCE LEAK found by Coverity
Error: RESOURCE_LEAK (CWE-772): - REAL ERROR
wget-1.19.5/src/warc.c:1376: alloc_fn: Storage is returned from allocation function "url_escape".
wget-1.19.5/src/url.c:284:3: alloc_fn: Storage is returned from allocation function "url_escape_1".
wget-1.19.5/src/url.c:255:3: alloc_fn: Storage is returned from allocation function "xmalloc".
wget-1.19.5/lib/xmalloc.c:41:11: alloc_fn: Storage is returned from allocation function "malloc".
wget-1.19.5/lib/xmalloc.c:41:11: var_assign: Assigning: "p" = "malloc(n)".
wget-1.19.5/lib/xmalloc.c:44:3: return_alloc: Returning allocated memory "p".
wget-1.19.5/src/url.c:255:3: var_assign: Assigning: "newstr" = "xmalloc(newlen + 1)".
wget-1.19.5/src/url.c:258:3: var_assign: Assigning: "p2" = "newstr".
wget-1.19.5/src/url.c:275:3: return_alloc: Returning allocated memory "newstr".
wget-1.19.5/src/url.c:284:3: return_alloc_fn: Directly returning storage allocated by "url_escape_1".
wget-1.19.5/src/warc.c:1376: var_assign: Assigning: "redirect_location" = storage returned from "url_escape(redirect_location)".
wget-1.19.5/src/warc.c:1381: noescape: Resource "redirect_location" is not freed or pointed-to in "fprintf".
wget-1.19.5/src/warc.c:1387: leaked_storage: Returning without freeing "redirect_location" leaks the storage that it points to.
\# 1385|     fflush (warc_current_cdx_file);
\# 1386|
\# 1387|->   return true;
\# 1388|   }
\# 1389|

url_escape() really returns a newly allocated memory and it leaks when the warc_write_cdx_record() returns. The memory returned from url_escape() is usually stored in a temporary variable in other parts of the project and then freed. I took the same approach.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2018-08-27 13:25:34 +02:00
Tomas Hozza
8b451f9f21 * src/warc.c (warc_write_start_record): Fix potential RESOURCE LEAK
In warc_write_start_record() function, the reutrn value of dup() is
directly used in gzdopen() call and not stored anywhere. However the
zlib documentation says that "The duplicated descriptor should be saved
to avoid a leak, since gzdopen does not close fd if it fails." [1].
This change stores the FD in a variable and closes it in case gzopen()
fails.

[1] https://www.zlib.net/manual.html

Error: RESOURCE_LEAK (CWE-772):
wget-1.19.5/src/warc.c:217: open_fn: Returning handle opened by "dup".
wget-1.19.5/src/warc.c:217: leaked_handle: Failing to save or close handle opened by "dup(fileno(warc_current_file))" leaks it.
\#  215|
\#  216|         /* Start a new GZIP stream. */
\#  217|->       warc_current_gzfile = gzdopen (dup (fileno (warc_current_file)), "wb9");
\#  218|         warc_current_gzfile_uncompressed_size = 0;
\#  219|

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2018-08-27 13:25:07 +02:00
Tomas Hozza
c045cdded4 * src/utils.c (open_stat): Fix RESOURCE LEAK found by Coverity
Error: RESOURCE_LEAK (CWE-772):
wget-1.19.5/src/utils.c:914: open_fn: Returning handle opened by "open". [Note: The source code implementation of the function has been overridden by a user model.]
wget-1.19.5/src/utils.c:914: var_assign: Assigning: "fd" = handle returned from "open(fname, flags, mode)".
wget-1.19.5/src/utils.c:921: noescape: Resource "fd" is not freed or pointed-to in "fstat". [Note: The source code implementation of the function has been overridden by a builtin model.]
wget-1.19.5/src/utils.c:924: leaked_handle: Handle variable "fd" going out of scope leaks the handle.
\#  922|     {
\#  923|       logprintf (LOG_NOTQUIET, _("Failed to stat file %s, error: %s\n"), fname, strerror(errno));
\#  924|->     return -1;
\#  925|     }
\#  926|   #if !(defined(WINDOWS) || defined(__VMS))

This seems to be a real issue, since the opened file descriptor in "fd"
would leak. There is also additional check below the "fstat" call, which
closes the opened "fd".

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2018-08-27 13:24:46 +02:00
Tomas Hozza
dfef92bac3 * src/http.c (http_loop): Fix RESOURCE LEAK found by Coverity
Error: RESOURCE_LEAK (CWE-772):
wget-1.19.5/src/http.c:4486: alloc_fn: Storage is returned from allocation function "url_string".
wget-1.19.5/src/url.c:2248:3: alloc_fn: Storage is returned from allocation function "xmalloc".
wget-1.19.5/lib/xmalloc.c:41:11: alloc_fn: Storage is returned from allocation function "malloc".
wget-1.19.5/lib/xmalloc.c:41:11: var_assign: Assigning: "p" = "malloc(n)".
wget-1.19.5/lib/xmalloc.c:44:3: return_alloc: Returning allocated memory "p".
wget-1.19.5/src/url.c:2248:3: var_assign: Assigning: "result" = "xmalloc(size)".
wget-1.19.5/src/url.c:2248:3: var_assign: Assigning: "p" = "result".
wget-1.19.5/src/url.c:2250:3: noescape: Resource "p" is not freed or pointed-to in function "memcpy". [Note: The source code implementation of the function has been overridden by a builtin model.]
wget-1.19.5/src/url.c:2253:7: noescape: Resource "p" is not freed or pointed-to in function "memcpy". [Note: The source code implementation of the function has been overridden by a builtin model.]
wget-1.19.5/src/url.c:2257:11: noescape: Resource "p" is not freed or pointed-to in function "memcpy". [Note: The source code implementation of the function has been overridden by a builtin model.]
wget-1.19.5/src/url.c:2264:3: noescape: Resource "p" is not freed or pointed-to in function "memcpy". [Note: The source code implementation of the function has been overridden by a builtin model.]
wget-1.19.5/src/url.c:2270:7: identity_transfer: Passing "p" as argument 1 to function "number_to_string", which returns an offset off that argument.
wget-1.19.5/src/utils.c:1776:11: var_assign_parm: Assigning: "p" = "buffer".
wget-1.19.5/src/utils.c:1847:3: return_var: Returning "p", which is a copy of a parameter.
wget-1.19.5/src/url.c:2270:7: noescape: Resource "p" is not freed or pointed-to in function "number_to_string".
wget-1.19.5/src/utils.c:1774:25: noescape: "number_to_string(char *, wgint)" does not free or save its parameter "buffer".
wget-1.19.5/src/url.c:2270:7: var_assign: Assigning: "p" = "number_to_string(p, url->port)".
wget-1.19.5/src/url.c:2273:3: noescape: Resource "p" is not freed or pointed-to in function "full_path_write".
wget-1.19.5/src/url.c:1078:47: noescape: "full_path_write(struct url const *, char *)" does not free or save its parameter "where".
wget-1.19.5/src/url.c:2287:3: return_alloc: Returning allocated memory "result".
wget-1.19.5/src/http.c:4486: var_assign: Assigning: "hurl" = storage returned from "url_string(u, URL_AUTH_HIDE_PASSWD)".
wget-1.19.5/src/http.c:4487: noescape: Resource "hurl" is not freed or pointed-to in "logprintf".
wget-1.19.5/src/http.c:4513: leaked_storage: Variable "hurl" going out of scope leaks the storage it points to.
\# 4511|               {
\# 4512|                 printwhat (count, opt.ntry);
\# 4513|->               continue;
\# 4514|               }
\# 4515|             else

There are two conditional branches, which call continue, without freeing memory potentially allocated and pointed to by"hurl" pointer. In fase "!opt.verbose" is True and some of the appropriate conditions in the following if/else if construction, in which "continue" is called, are also true, then the memory allocated to "hurl" will leak.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2018-08-27 13:24:24 +02:00
Tomas Hozza
b8be904ac7 * src/http.c (check_auth): Fix RESOURCE LEAK found by Coverity
Error: RESOURCE_LEAK (CWE-772):
wget-1.19.5/src/http.c:2434: alloc_fn: Storage is returned from allocation function "xmalloc".
wget-1.19.5/lib/xmalloc.c:41:11: alloc_fn: Storage is returned from allocation function "malloc".
wget-1.19.5/lib/xmalloc.c:41:11: var_assign: Assigning: "p" = "malloc(n)".
wget-1.19.5/lib/xmalloc.c:44:3: return_alloc: Returning allocated memory "p".
wget-1.19.5/src/http.c:2434: var_assign: Assigning: "auth_stat" = storage returned from "xmalloc(4UL)".
wget-1.19.5/src/http.c:2446: noescape: Resource "auth_stat" is not freed or pointed-to in "create_authorization_line".
wget-1.19.5/src/http.c:5203:70: noescape: "create_authorization_line(char const *, char const *, char const *, char const *, char const *, _Bool *, uerr_t *)" does not free or save its parameter "auth_err".
wget-1.19.5/src/http.c:2476: leaked_storage: Variable "auth_stat" going out of scope leaks the storage it points to.
\# 2474|                 /* Creating the Authorization header went wrong */
\# 2475|               }
\# 2476|->         }
\# 2477|         else
\# 2478|           {

Error: RESOURCE_LEAK (CWE-772):
wget-1.19.5/src/http.c:2431: alloc_fn: Storage is returned from allocation function "url_full_path".
wget-1.19.5/src/url.c:1105:19: alloc_fn: Storage is returned from allocation function "xmalloc".
wget-1.19.5/lib/xmalloc.c:41:11: alloc_fn: Storage is returned from allocation function "malloc".
wget-1.19.5/lib/xmalloc.c:41:11: var_assign: Assigning: "p" = "malloc(n)".
wget-1.19.5/lib/xmalloc.c:44:3: return_alloc: Returning allocated memory "p".
wget-1.19.5/src/url.c:1105:19: var_assign: Assigning: "full_path" = "xmalloc(length + 1)".
wget-1.19.5/src/url.c:1107:3: noescape: Resource "full_path" is not freed or pointed-to in function "full_path_write".
wget-1.19.5/src/url.c:1078:47: noescape: "full_path_write(struct url const *, char *)" does not free or save its parameter "where".
wget-1.19.5/src/url.c:1110:3: return_alloc: Returning allocated memory "full_path".
wget-1.19.5/src/http.c:2431: var_assign: Assigning: "pth" = storage returned from "url_full_path(u)".
wget-1.19.5/src/http.c:2446: noescape: Resource "pth" is not freed or pointed-to in "create_authorization_line".
wget-1.19.5/src/http.c:5203:40: noescape: "create_authorization_line(char const *, char const *, char const *, char const *, char const *, _Bool *, uerr_t *)" does not free or save its parameter "path".
wget-1.19.5/src/http.c:2476: leaked_storage: Variable "pth" going out of scope leaks the storage it points to.
\# 2474|                 /* Creating the Authorization header went wrong */
\# 2475|               }
\# 2476|->         }
\# 2477|         else
\# 2478|           {

Both "pth" and "auth_stat" are allocated in "check_auth()" function. These are used for creating the HTTP Authorization Request header via "create_authorization_line()" function. In case the creation went OK (auth_err == RETROK), then the memory previously allocated to "pth" and "auth_stat" is freed. However if the creation failed, then the memory is never freed and it leaks.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2018-08-27 13:23:52 +02:00
Tomas Hozza
b24351183e * src/ftp.c (getftp): Fix RESOURCE LEAK found by Coverity
Error: RESOURCE_LEAK (CWE-772):
wget-1.19.5/src/ftp.c:1493: alloc_fn: Storage is returned from allocation function "fopen".
wget-1.19.5/src/ftp.c:1493: var_assign: Assigning: "fp" = storage returned from "fopen(con->target, "wb")".
wget-1.19.5/src/ftp.c:1811: leaked_storage: Variable "fp" going out of scope leaks the storage it points to.
\# 1809|     if (fp && !output_stream)
\# 1810|       fclose (fp);
\# 1811|->   return err;
\# 1812|   }
\# 1813|

It can happen, that "if (!output_stream || con->cmd & DO_LIST)" on line #1398 can be true, even though "output_stream != NULL". In this case a new file is opened to "fp". Later it may happen in the FTPS branch, that some error will occure and code will jump to label "exit_error". In "exit_error", the "fp" is closed only if "output_stream == NULL". However this may not be true as described earlier and "fp" leaks.

On line #1588, there is the following conditional free of "fp":

  /* Close the local file.  */
  if (!output_stream || con->cmd & DO_LIST)
    fclose (fp);

Therefore the conditional at the end of the function after "exit_error" label should be modified to:

  if (fp && (!output_stream || con->cmd & DO_LIST))
    fclose (fp);

This will ensure that "fp" does not leak in any case it sould be opened.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2018-08-27 13:20:48 +02:00
Tomas Hozza
2b2283d3e2 Don't limit the test suite HTTPS server to TLSv1
In Fedora, we are implementing crypto policies, in order to enhance the
security of user systems. This is done on the system level by global
configuration. It may happen that due to the active policy, only
TLSv1.2 or higher will be available in crypto libraries. While wget as
a client will by default determine the minimal TLS version supported by
both client and server, the HTTPS server implementation in testenv/
hardcodes use of TLSv1. As a result all HTTPS related tests fail in
case a more hardened crypto policy is set on the Fedora system.

This change removes the explicit TLS version setting and leaves the
determination of the minimal supported TLS version on the server and
client.

More information about Fedora change can be found here:
https://fedoraproject.org/wiki/Changes/StrongCryptoSettings

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2018-08-11 12:51:13 +02:00
Tim Rühsen
122a9f08a3 * src/gnutls.c (ssl_check_certificate): Fix grammar of error msg
Reported-by: Nicholas Sielicki
2018-06-13 20:34:24 +02:00
Tim Rühsen
333746f787 * fuzz/Makefile.am: Remove libtool LTLIB... from LDADD 2018-06-13 20:16:46 +02:00
Tim Rühsen
4fc69950da * src/http.c (http_loop): Fix --retry-on-host-error 2018-06-13 20:16:22 +02:00
ethus3h
e7979da9e8 Add new option --retry-on-host-error
* doc/wget.texi: Add docs for --retry-on-host-error
* src/http.c (http_loop): Add code for HOSTERR
* src/init.c: Add option --retry-on-host-error
* src/main.c: Likewise
* src/options.h: Add options.retry_on_host_error

Copyright-paperwork-exempt: Yes
2018-06-13 20:10:28 +02:00
Tim Rühsen
ad261f41ce Save original data to WARC file
* src/retr.c (write_data): Cleanup,
  (fd_read_body): Write to WARC before uncompressing

Fixes: #53968
2018-05-29 10:52:20 +02:00
Tim Rühsen
c88500fca8 * fuzz/get_ossfuzz_corpora: Speed up corpora download 2018-05-10 19:33:59 +02:00
Tim Rühsen
4188fcdced * src/main.c (print_version): Silence UBSAN message 2018-05-09 13:56:20 +02:00
Tim Rühsen
4bdb09d3a7 * src/utils.ci (file_exists_p): Fix stat(NULL,...) 2018-05-09 12:37:03 +02:00
Tim Rühsen
35f5f79ce1 * src/hsts.c (open_hsts_test_store): Fix unlink(NULL) 2018-05-09 12:29:39 +02:00
Tim Rühsen
3cbdc67c96 * src/hash.c: Silence UBSAN for hash functions 2018-05-09 12:16:51 +02:00
Tim Rühsen
cdaee00259 * fuzz/*_fuzzer.in: Update corpora from OSS-Fuzz 2018-05-09 11:45:22 +02:00
Tim Rühsen
ad2471425f * fuzz/get_ossfuzz_corpora: Fix path 2018-05-09 11:44:18 +02:00
Tim Rühsen
ace96e4412 * src/hsts.h: Fix header guard 2018-05-08 10:17:06 +02:00
Tim Rühsen
77286a2e03 * src/version.h: Add header guard 2018-05-08 10:10:44 +02:00
Tim Rühsen
7eff94e881 * src/host.c (wait_ares): Remove void assignment
Reported-by: Josef Moellers
2018-05-08 09:36:48 +02:00
20902 changed files with 60888 additions and 25036 deletions

21
.gitignore vendored
View File

@@ -1,6 +1,6 @@
/ABOUT-NLS~
# Project Root
/.sc-start-sc_bindtextdomain
/.sc-start-sc_prohibit_HAVE_MBRTOWC
/.sc-start-*
/.version
/ABOUT-NLS
/GNUmakefile
@@ -14,7 +14,10 @@
/config.status
/configure
/lib/
!/lib/Makefile.am
/maint.mk
/gnulib_po/
/aminclude_static.am
# Wildcard Ignores
*~
po/*.gmo*
@@ -24,6 +27,11 @@ tests/*.log
tests/*.trs
testenv/*.log
testenv/*.trs
fuzz/*
!fuzz/*.*
fuzz/*.o
fuzz/*.log
fuzz/*.trs
# build-aux/
build-aux/.gitignore
build-aux/ar-lib
@@ -46,6 +54,7 @@ doc/version.texi
doc/wget.1
doc/wget.info
doc/wget.pod
doc/.gitignore
# m4/
m4/.gitignore
m4/gnulib-cache.m4
@@ -59,6 +68,7 @@ po/Makevars
po/POTFILES
po/wget.pot
po/stamp-po
po/remove-potcdate.sed
# src/
src/.deps/
src/Makefile
@@ -87,9 +97,16 @@ testenv/Makefile.in
tests/Makefile
tests/Makefile.in
tests/unit-tests
tests/unit-tests.o
tests/.deps/
tests/certs/interca.conf
tests/certs/rootca.conf
# util/
util/Makefile
util/Makefile.in
# fuzz/
fuzz/.deps/
fuzz/Makefile.in
# others
/tags
/cscope.out

View File

@@ -1,124 +1,496 @@
# we utilize the images generated by the build-images project, to
# speed up CI runs. We also use ccache and store config.cache
# to speed up compilation.
stages:
- test-from-git
- test-from-tarball
- deploy
cache:
key: "$CI_JOB_NAME"
paths:
- cache/
before_script:
# CCache Config
- mkdir -p cache
- export CCACHE_BASEDIR=${PWD}
- export CCACHE_DIR=${PWD}/cache
- echo $CCACHE_DIR
- export CC="ccache gcc"
after_script:
# somehow after_script loses environment
- export CCACHE_BASEDIR=${PWD}
- export CCACHE_DIR=${PWD}/cache
- if type -p ccache >/dev/null; then ccache -s; fi
variables:
BUILD_IMAGES_PROJECT: gnuwget/build-images
DEBIAN_BUILD: buildenv-debian-stretch
FEDORA_BUILD: buildenv-f25
CENTOS7_BUILD: buildenv-centos7
CI_BASE_PATH: $CI_REGISTRY/$BUILD_IMAGES_PROJECT/wget
DEBIAN_OLDSTABLE_BUILD: buildenv-debian-oldstable
DEBIAN_TESTING_BUILD: buildenv-debian-testing
DEBIAN_STABLE_BUILD: buildenv-debian-stable
CENTOS_OLDLTS_BUILD: buildenv-centos7
FEDORA_BUILD: buildenv-fedora
MINGW_BUILD: buildenv-mingw
ARCH_BUILD: buildenv-arch
GET_SOURCES_ATTEMPTS: "3"
GIT_DEPTH: "5"
CONFIGURE_BASE_FLAGS: --enable-assert --cache-file cache/config.cache
CFLAGS_DEFAULT: -O0 -g -ggdb3
# In this build we combine
# * syntax-check
# * abi-check
# * build/valgrind-check
# * build/asan-check
# * build w/Werror and ubsan-check
Debian GNU/Linux build:
image: $CI_REGISTRY/$BUILD_IMAGES_PROJECT:$DEBIAN_BUILD
script:
- echo "127.0.0.1 wgettestingserver" >>/etc/hosts
- cat /etc/hosts
- touch .manywarnings
- alias make="make -j$(nproc)"
- ./bootstrap
- ./configure -C
- make syntax-check
- make check
# - make abi-check
- make distcheck
- make check-valgrind
- ./configure -C --with-ssl=openssl
- make clean
- make check
# - ./configure -C --enable-fsanitize-asan
# - make clean
# - make check
# - ./configure -C --enable-fsanitize-ubsan
# - make clean
# - make check
# - ./configure -C --enable-fsanitize-msan
# - make clean
# - make check
tags:
- shared
artifacts:
expire_in: 2 weeks
when: on_failure
paths:
- tests/*.log
# - compat_reports/
# Create the tarball in a separate build directory (VPATH).
Build-Tarball:
stage: test-from-git
image: $CI_REGISTRY/$BUILD_IMAGES_PROJECT:$DEBIAN_TESTING_BUILD
script:
- export CFLAGS=$CFLAGS_DEFAULT
- ./bootstrap
- autoreconf -fi
- mkdir vpath && cd vpath
- ../configure $CONFIGURE_BASE_FLAGS --cache-file ../cache/config.cache
--without-ssl --enable-ipv6 --without-zlib --without-libiconv-prefix
--disable-iri --disable-ntlm --disable-pcre --without-libpsl --without-libuuid
--without-libintl-prefix
- make -j$(nproc)
- make -j$(nproc) syntax-check
- make -j$(nproc) dist
- mv wget-*.gz ..
tags:
- shared
- linux
except:
- coverity-scan@gnuwget/wget
artifacts:
expire_in: 2 weeks
when: on_failure
paths:
- ./*.log
- vpath/*.log
- vpath/fuzz/*.log
- vpath/tests/*.log
- vpath/testenv/*.log
artifacts:
expire_in: 2 weeks
when: on_success
paths:
- wget-*.gz
#Centos7 build:
# image: $CI_REGISTRY/$BUILD_IMAGES_PROJECT:$CENTOS7_BUILD
# script:
# - sed -i 's/AM_GNU_GETTEXT_VERSION(0.19.3)/AM_GNU_GETTEXT_VERSION(0.18.2)/g' configure.ac
# - ./bootstrap && ./configure --enable-gcc-warnings --disable-doc && make -j$(nproc) check
# tags:
# - shared
# artifacts:
# expire_in: 2 weeks
# when: on_failure
# paths:
# - tests/*.log
# - compat_reports/
Valgrind:
stage: test-from-tarball
image: $CI_REGISTRY/$BUILD_IMAGES_PROJECT:$DEBIAN_TESTING_BUILD
variables:
GIT_STRATEGY: none
script:
- export CFLAGS=$CFLAGS_DEFAULT
- tar xvf wget-*.gz
- cd wget-*/
- ./configure $CONFIGURE_BASE_FLAGS --cache-file ../cache/config.cache
- make check-valgrind
tags:
- shared
- linux
except:
- coverity-scan@gnuwget/wget
dependencies:
- Build-Tarball
needs: ["Build-Tarball"]
artifacts:
expire_in: 2 weeks
when: on_failure
paths:
- wget-*/*.log
- wget-*/fuzz/*.log
- wget-*/tests/*.log
- wget-*/testenv/*.log
#clang-analyzer/Fedora:
# image: $CI_REGISTRY/$BUILD_IMAGES_PROJECT:$FEDORA_BUILD
# script:
# - ./bootstrap
# - scan-build ./configure --disable-doc --enable-gcc-warnings
# - make -j$(nproc) -C gl
# - scan-build --status-bugs -o scan-build-lib make -j$(nproc) -C lib
# tags:
# - shared
# except:
# - tags
# artifacts:
# expire_in: 2 weeks
# when: on_failure
# paths:
# - scan-build-lib/*
Minimal:
stage: test-from-tarball
image: $CI_BASE_PATH/debian:testing-minimal
variables:
GIT_STRATEGY: none
script:
- export CFLAGS=$CFLAGS_DEFAULT
- tar xvf wget-*.gz
- cd wget-*/
- ./configure $CONFIGURE_BASE_FLAGS --cache-file ../cache/config.cache
--disable-nls --without-ssl --enable-ipv6 --without-zlib --without-libiconv-prefix
--disable-iri --disable-ntlm --disable-pcre --without-libpsl --without-libuuid
--without-libintl-prefix
- make -j$(nproc) check-valgrind
tags:
- shared
- linux
except:
- coverity-scan@gnuwget/wget
dependencies:
- Build-Tarball
needs: ["Build-Tarball"]
artifacts:
expire_in: 2 weeks
when: on_failure
paths:
- wget-*/./*.log
- wget-*/fuzz/*.log
- wget-*/tests/*.log
- wget-*/testenv/*.log
#pages:
# image: $CI_REGISTRY/$BUILD_IMAGES_PROJECT:$DEBIAN_BUILD
# script:
# - ./bootstrap
# - CFLAGS="-g -O0" ./configure --enable-gtk-doc --enable-code-coverage --disable-gcc-warnings
# - make -j$(nproc) check
# - make local-code-coverage-output
# - mkdir -p public
# - rm -rf public/coverage
# - mv libidn2-*-coverage public/coverage
# - rm -rf public/reference
# - mv doc/reference/html public/reference
# - make -C doc manual
# - rm -rf public/manual
# - mv doc/manual public/manual
# tags:
# - shared
# artifacts:
# when: on_success
# paths:
# - public
# only:
# - master
Debian-OldStable:
stage: test-from-tarball
image: $CI_BASE_PATH/debian:oldstable
variables:
GIT_STRATEGY: none
script:
- export CFLAGS=$CFLAGS_DEFAULT
- tar xvf wget-*.gz
- cd wget-*/
- ./configure $CONFIGURE_BASE_FLAGS --cache-file ../cache/config.cache
- make
- make -j$(nproc) check
tags:
- shared
- linux
except:
- coverity-scan@gnuwget/wget
dependencies:
- Build-Tarball
needs: ["Build-Tarball"]
artifacts:
expire_in: 2 weeks
when: on_failure
paths:
- wget-*/./*.log
- wget-*/fuzz/*.log
- wget-*/tests/*.log
- wget-*/testenv/*.log
#MinGW64:
# image: $CI_REGISTRY/$BUILD_IMAGES_PROJECT:$FEDORA_BUILD
# before_script:
# - mount -t binfmt_misc binfmt_misc /proc/sys/fs/binfmt_misc &&
# echo ':DOSWin:M::MZ::/usr/bin/wine64:' > /proc/sys/fs/binfmt_misc/register
# script:
# - ./bootstrap
# - mingw64-configure --disable-valgrind-tests --disable-doc
# - mingw64-make -j$(nproc)
# - mingw64-make -j$(nproc) check
# tags:
# - shared
# - docker
# except:
# - tags
# artifacts:
# expire_in: 2 weeks
# when: on_failure
# paths:
# - ./*.log
# - tests/*.log
Debian-Stable:
stage: test-from-tarball
image: $CI_BASE_PATH/debian:stable
variables:
GIT_STRATEGY: none
script:
- export CFLAGS=$CFLAGS_DEFAULT
- tar xvf wget-*.gz
- cd wget-*/
- ./configure $CONFIGURE_BASE_FLAGS --cache-file ../cache/config.cache
- make -j$(nproc) check
tags:
- shared
- linux
except:
- coverity-scan@gnuwget/wget
dependencies:
- Build-Tarball
needs: ["Build-Tarball"]
artifacts:
expire_in: 2 weeks
when: on_failure
paths:
- wget-*/./*.log
- wget-*/fuzz/*.log
- wget-*/tests/*.log
- wget-*/testenv/*.log
CentOS-OldLTS:
stage: test-from-tarball
image: $CI_REGISTRY/$BUILD_IMAGES_PROJECT:$CENTOS_OLDLTS_BUILD
variables:
GIT_STRATEGY: none
script:
- export CFLAGS=$CFLAGS_DEFAULT
- tar xvf wget-*.gz
- cd wget-*/
- ./configure $CONFIGURE_BASE_FLAGS --cache-file ../cache/config.cache
- make -j$(nproc)
- make -j$(nproc) check
tags:
- shared
- linux
except:
- coverity-scan@gnuwget/wget
dependencies:
- Build-Tarball
needs: ["Build-Tarball"]
artifacts:
expire_in: 2 weeks
when: on_failure
paths:
- wget-*/./*.log
- wget-*/fuzz/*.log
- wget-*/tests/*.log
- wget-*/testenv/*.log
# The test suite is currently broken, so a temporary allowance
allow_failure: true
TLS/OpenSSL:
stage: test-from-tarball
image: $CI_REGISTRY/$BUILD_IMAGES_PROJECT:$DEBIAN_TESTING_BUILD
variables:
GIT_STRATEGY: none
script:
- export CFLAGS=$CFLAGS_DEFAULT
- tar xvf wget-*.gz
- cd wget-*/
- echo "127.0.0.1 wgettestingserver" >>/etc/hosts
- cat /etc/hosts
- ./configure $CONFIGURE_BASE_FLAGS --cache-file ../cache/config.cache --with-ssl=openssl
- make -j$(nproc)
- make -j$(nproc) check-valgrind
tags:
- shared
- linux
except:
- coverity-scan@gnuwget/wget
dependencies:
- Build-Tarball
needs: ["Build-Tarball"]
artifacts:
expire_in: 2 weeks
when: on_failure
paths:
- wget-*/./*.log
- wget-*/fuzz/*.log
- wget-*/tests/*.log
- wget-*/testenv/*.log
TLS/GnuTLS:
stage: test-from-tarball
image: $CI_REGISTRY/$BUILD_IMAGES_PROJECT:$DEBIAN_TESTING_BUILD
variables:
GIT_STRATEGY: none
script:
- export CFLAGS=$CFLAGS_DEFAULT
- tar xvf wget-*.gz
- cd wget-*/
- echo "127.0.0.1 wgettestingserver" >>/etc/hosts
- cat /etc/hosts
- ./configure $CONFIGURE_BASE_FLAGS --cache-file ../cache/config.cache --with-ssl=gnutls
- make -j$(nproc)
- make -j$(nproc) check-valgrind
tags:
- shared
- linux
except:
- coverity-scan@gnuwget/wget
dependencies:
- Build-Tarball
needs: ["Build-Tarball"]
artifacts:
expire_in: 2 weeks
when: on_failure
paths:
- wget-*/./*.log
- wget-*/fuzz/*.log
- wget-*/tests/*.log
- wget-*/testenv/*.log
Sanitizers:
stage: test-from-tarball
image: $CI_REGISTRY/$BUILD_IMAGES_PROJECT:$DEBIAN_TESTING_BUILD
variables:
GIT_STRATEGY: none
script:
- export CFLAGS=$CFLAGS_DEFAULT
- tar xvf wget-*.gz
- cd wget-*/
# - export CFLAGS="$CFLAGS_DEFAULT -Werror"
- export CFLAGS="$CFLAGS_DEFAULT"
- export CC="ccache clang"
- export UBSAN_OPTIONS=print_stacktrace=1
- export ASAN_SYMBOLIZER_PATH=/usr/bin/llvm-symbolizer
# - export LSAN_OPTIONS=suppressions="$PWD/tests/clang-asan-suppressions"
- ./configure $CONFIGURE_BASE_FLAGS --cache-file ../cache/config.cache
--enable-fsanitize-asan --enable-fsanitize-ubsan
- make -j$(nproc) check
tags:
- shared
- linux
except:
- coverity-scan@gnuwget/wget
dependencies:
- Build-Tarball
needs: ["Build-Tarball"]
artifacts:
expire_in: 2 weeks
when: on_failure
paths:
- wget-*/./*.log
- wget-*/fuzz/*.log
- wget-*/tests/*.log
- wget-*/testenv/*.log
Scan-Build:
stage: test-from-tarball
image: $CI_REGISTRY/$BUILD_IMAGES_PROJECT:$DEBIAN_TESTING_BUILD
variables:
GIT_STRATEGY: none
script:
- export CFLAGS=$CFLAGS_DEFAULT
- tar xvf wget-*.gz
- cd wget-*/
- scan-build ./configure $CONFIGURE_BASE_FLAGS --cache-file ../cache/config.cache
- make -C lib -j$(nproc)
- make -C src css_.o
- scan-build -v -enable-checker nullability --keep-empty --status-bugs -o ../scan-build make -j$(nproc)
tags:
- shared
- linux
except:
- tags
- coverity-scan@gnuwget/wget
dependencies:
- Build-Tarball
needs: ["Build-Tarball"]
artifacts:
expire_in: 2 weeks
when: on_failure
paths:
- scan-build/
# scan-build reports two false positives that we can't suppress
allow_failure: true
CoverageReports:
stage: test-from-tarball
image: $CI_REGISTRY/$BUILD_IMAGES_PROJECT:$DEBIAN_TESTING_BUILD
variables:
GIT_STRATEGY: none
script:
- export CFLAGS=$CFLAGS_DEFAULT
- tar xvf wget-*.gz
- cd wget-*/
- ./configure $CONFIGURE_BASE_FLAGS --cache-file ../cache/config.cache
- make -j$(nproc) check-coverage
- mv lcov ../test-coverage
- make -C doc html
- make -j$(nproc) fuzz-coverage
- mv lcov ../fuzz-coverage
- make dist
tags:
- shared
- linux
dependencies:
- Build-Tarball
needs: ["Build-Tarball"]
artifacts:
when: on_success
paths:
- test-coverage/
- fuzz-coverage/
only:
- master
MinGW64:
stage: test-from-tarball
image: $CI_REGISTRY/$BUILD_IMAGES_PROJECT:$MINGW_BUILD
variables:
GIT_STRATEGY: none
script:
- export CFLAGS=$CFLAGS_DEFAULT
- tar xvf wget-*.gz
- cd wget-*/
- mount -t binfmt_misc binfmt_misc /proc/sys/fs/binfmt_misc
- echo ':DOSWin:M::MZ::/usr/bin/wine:' > /proc/sys/fs/binfmt_misc/register
- export CC="ccache $PREFIX-gcc"
- export GCCLIB=$(dirname $(find /usr/lib/gcc/$PREFIX -name libgcc_s_seh-1.dll|grep posix))
- export WINEPATH="$WINEPATH;/usr/$PREFIX/bin;/usr/$PREFIX/lib;$PWD/libwget/.libs;$GCCLIB"
- echo "WINEPATH=$WINEPATH"
- LIBS="-ldl -lpsapi" ./configure $CONFIGURE_BASE_FLAGS --cache-file ../cache/config.cache
--build=x86_64-pc-linux-gnu --host=$PREFIX --enable-shared
- make -j$(nproc)
- cp -p src/wget.exe src/wget
- make check -j$(nproc) LOG_COMPILER=wine
tags:
- shared
- linux
except:
- tags
- coverity-scan@gnuwget/wget
dependencies:
- Build-Tarball
needs: ["Build-Tarball"]
artifacts:
expire_in: 2 weeks
when: on_failure
paths:
- wget-*/./config.h
- wget-*/./*.log
- wget-*/fuzz/*.log
- wget-*/tests/*.log
- wget-*/testenv/*.log
# The MinGW64 build breaks quite often, mostly due to dependencies.
allow_failure: true
# Build from git should work on Arch.
ArchLinux:
stage: test-from-git
image: $CI_BASE_PATH/archlinux
script:
- export CFLAGS="$CFLAGS_DEFAULT"
- ./bootstrap
- autoreconf -fi
- touch .manywarnings
- ./configure $CONFIGURE_BASE_FLAGS
- make -j$(nproc)
- make check -j$(nproc)
tags:
- shared
- linux
except:
- coverity-scan@gnuwget/wget
needs: []
artifacts:
expire_in: 2 weeks
when: on_failure
paths:
- ./*.log
- fuzz/*.log
- tests/*.log
- testenv/*.log
# Build from git should work on Fedora.
Fedora:
stage: test-from-git
image: $CI_REGISTRY/$BUILD_IMAGES_PROJECT:$FEDORA_BUILD
script:
- export CFLAGS="$CFLAGS_DEFAULT"
- ./bootstrap
- autoreconf -fi
- touch .manywarnings
- ./configure $CONFIGURE_BASE_FLAGS
- make -j$(nproc) check
tags:
- shared
- linux
except:
- coverity-scan@gnuwget/wget
needs: []
artifacts:
expire_in: 2 weeks
when: on_failure
paths:
- ./*.log
- fuzz/*.log
- tests/*.log
- testenv/*.log
pages:
stage: deploy
script:
- mkdir -p public
- rm -rf public/reference public/coverage public/fuzz-coverage
- mv wget-*.gz public/wget-latest.tar.gz
- mv test-coverage public/coverage
- mv fuzz-coverage public/fuzz-coverage
dependencies:
- CoverageReports
- Build-Tarball
artifacts:
when: on_success
paths:
- public
only:
- master

2
.gitmodules vendored
View File

@@ -1,3 +1,3 @@
[submodule "gnulib"]
path = gnulib
url = git://git.sv.gnu.org/gnulib.git
url = https://git.savannah.gnu.org/git/gnulib.git

8
.lgtm.yml Normal file
View File

@@ -0,0 +1,8 @@
extraction:
cpp:
prepare:
packages:
- libgpgme11-dev
- autoconf-archive
configure:
command: ./bootstrap && autoreconf -ivf && ./configure

13
.mailmap Normal file
View File

@@ -0,0 +1,13 @@
Tim Rühsen <tim.ruehsen@gmx.de> Tim Ruehsen <tim.ruehsen@gmx.de>
Tim Rühsen <tim.ruehsen@gmx.de> rockdaboot <tim.ruehsen@gmx.de>
Darshit Shah <darnir@gnu.org> <darnir@gmail.com>
Darshit Shah <darnir@gnu.org> <darnir@reniac.com>
Ander Juaristi <ajuaristi@gmx.es> <a@juaristi.eus>
Ander Juaristi <ajuaristi@gmx.es> Ander Juaristi Alamos <ajuaristi@gmx.es>
Micah Cowan <micah@cowan.name> micah <devnull@localhost>
Steven Schubiger <stsc@members.fsf.org> Steven Schubiger <stsc@member.fsf.org>
Steven Schubiger <stsc@members.fsf.org> <schubiger@gmail.com>
Giuseppe Scrivano <gscrivano@gnu.org> <gscrivan@redhat.com>
Gisle Vanem <gisle.vanem@gmail.com> <gvanem@yahoo.no>
Gisle Vanem <gisle.vanem@gmail.com> <gvanem@broadpark.no>
Gisle Vanem <gisle.vanem@gmail.com> <giva@bgnett.no>

1
.prev-version Normal file
View File

@@ -0,0 +1 @@
1.21.3

View File

@@ -10,7 +10,7 @@ notifications:
email:
recipients:
- darnir@gmail.com
- bug-wget@gnu.org
- wget-dev@gnu.org
on_success: change
on_failure: always

View File

@@ -55,4 +55,8 @@ Ted Mielczarek. Support for parsing links from CSS.
Saint Xavier. Support for IRIs (RFC 3987).
Giuseppe Scrivano. Added support for HTTP/1.1. Current Wget maintainer.
Giuseppe Scrivano. Added support for HTTP/1.1. Active Wget Maintainer from 2010-2015.
Tim Ruehsen. Active Wget Maintainer / Developer from 2012-2020.
Darshit Shah. Active Wget Maintainer / Developer from 2013-2020.

65
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,65 @@
# Wget Contribution Guidelines
1. [Guidelines For Patch
Submissions](#Guidelines_For_Patch_Submissions)
1. [What is a patch ?](#What_is_a_patch.3F)
2. [Where to send the patches](#Where_to_send_the_patches)
3. [Standards and coding style](#Standards_and_coding_style)
Guidelines For Patch Submissions {#Guidelines_For_Patch_Submissions}
================================
What is a patch ? {#What_is_a_patch.3F}
-----------------
A patch file, also known as a \"diff\", is a textual representation of
changes to source code. Patches are readable enough
to be reviewed by []{#line-9 .anchor}humans and at the same time regular
enough to be processed by programs. The `patch`
utility is used to change the source code in the
manner that the patch describes, this being called \"applying\" the
patch. Patches work even on files that have been
modified independently of the modifications in the
patch, as long as those other changes do not
conflict with the patch.
Because of these properties, patches are the preferred means of
distributing the changes to a free software project.
If you have made a change to Wget and would like to
contribute it, you will need to create a patch and
send it to the developers; please read on.
Where to send the patches {#Where_to_send_the_patches}
-------------------------
Patches intended to be applied to Wget should be mailed to
bug-wget@gnu.org. Each patch will be reviewed by the
developers, and will be acked and added to the
distribution, or rejected with an explanation.
Unfortunately, the developers are often busy with
their day jobs, so the review process can take a while.
*Every* patch should be accompanied by an explanation of what the patch
changes, and why the change is desirable or
necessary. The explanation need not be long, but
please don\'t just send a patch without any
accompanying text.
Please send your patches as email attachments. It is important that the
patch survives the travel unchanged so that we can feed it to
the `patch` utility or `git am/apply` after or while reviewing it.
Standards and coding style {#Standards_and_coding_style}
--------------------------
Wget abides by the GNU coding standards, available at:
- https://www.gnu.org/prep/standards.html
But maybe even better is to stay with the surrounding coding style.
When working with git: The commit messages needs a certain GNU style
format. Please check `git log` for examples. The maintainers might
correct small issues here before pushing / applying without asking.

View File

@@ -1,7 +1,7 @@
GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Copyright (C) 2007, 2023 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.

View File

@@ -1,6 +1,5 @@
# Makefile for `Wget' utility
# Copyright (C) 1995-1997, 2006-2018 Free Software Foundation,
# Inc.
# Copyright (C) 1995-1997, 2006-2023 Free Software Foundation, Inc.
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -41,7 +40,7 @@ distuninstallcheck_listfiles = find . -type f | \
ACLOCAL_AMFLAGS = -I m4
# subdirectories in the distribution
SUBDIRS = lib src doc po util fuzz tests testenv
SUBDIRS = lib src doc po gnulib_po util fuzz tests testenv
EXTRA_DIST = MAILING-LIST \
msdos/config.h msdos/Makefile.DJ \
@@ -49,6 +48,7 @@ EXTRA_DIST = MAILING-LIST \
build-aux/build_info.pl build-aux/git-version-gen .version
CLEANFILES = *~ *.bak $(DISTNAME).tar.gz
DISTCLEANFILES = po/stamp-po gnulib_po/stamp-po
BUILT_SOURCES = .version
@@ -80,3 +80,30 @@ gen-ChangeLog:
check-valgrind:
TESTS_ENVIRONMENT="VALGRIND_TESTS=1" $(MAKE) check
LCOV_INFO=wget.info
clean-lcov:
rm -rf $(LCOV_INFO) */*.gc?? */.libs/*.gc?? lcov/
lcov --zerocounters --directory src/
check-coverage: clean clean-lcov
$(MAKE) CFLAGS="$(CFLAGS) --coverage" LDFLAGS="$(LDFLAGS) --coverage"
lcov --capture --initial --directory src/ --output-file $(LCOV_INFO)
$(MAKE) CFLAGS="$(CFLAGS) --coverage" LDFLAGS="$(LDFLAGS) --coverage" VALGRIND_TESTS=0 check
lcov --capture --directory src/ --output-file $(LCOV_INFO)
lcov --remove $(LCOV_INFO) '/usr/include/*' '*/lib/*' -o $(LCOV_INFO)
genhtml --prefix . --ignore-errors source $(LCOV_INFO) --legend --title "Wget" --output-directory=lcov
@echo
@echo "You can now view the coverage report with 'xdg-open lcov/index.html'"
fuzz-coverage: clean clean-lcov
$(MAKE) -C lib
$(MAKE) -C src CFLAGS="$(CFLAGS) --coverage" LDFLAGS="$(LDFLAGS) --coverage"
$(MAKE) -C fuzz check CFLAGS="$(CFLAGS) --coverage" LDFLAGS="$(LDFLAGS) --coverage"
lcov --capture --initial --directory fuzz --directory src --output-file $(LCOV_INFO)
lcov --capture --directory fuzz --directory src --output-file $(LCOV_INFO)
lcov --remove $(LCOV_INFO) '/usr/include/*' '*/lib/*' -o $(LCOV_INFO)
genhtml --prefix . --ignore-errors source $(LCOV_INFO) --legend --title "Wget-fuzz" --output-directory=lcov
@echo
@echo "You can now view the coverage report with 'xdg-open lcov/index.html'"

124
NEWS
View File

@@ -1,9 +1,121 @@
GNU Wget NEWS -- history of user-visible changes.
Copyright (C) 1997-2018 Free Software Foundation, Inc.
See the end for copying conditions.
* Noteworthy changes in release ?.? (????-??-??) [?]
* Noteworthy changes in release 1.21.4 (2023-05-11)
** Document --retry-on-host-error in help text
** Increase read buffer size to 64k. This should speed up downloads on gigabit and faster connections
** Update deprecated option '--html-extension' to '--adjust-extension' in documentation
** Update gnulib compatibility layer.
Fixes HSTS test failures on i686. (Thanks to Andreas Enge for ponting it out)
* Noteworthy changes in release 1.21.3 (2022-02-26)
** Fix computation of total bytes downloaded during FTP trasnfers (#61277)
** Add option to select TLS 1.3 on the command line
** Fix HSTS build issues on some 64-bit big-endian systems
** Hide password during status report in --no-verbose
** Remove a sprurious print statement that showed up even during --quiet
** Some more cleanups and bug-fixes
* Noteworthy changes in release 1.21.2 (2021-09-07)
** Support for autoconf 2.71
** Fix a double free in FTP when using an absolute path
** Release tarballs no longer have a dependency on Python.
** --page-requisites will now also download links marked as "alternate
stylesheet" or "icon"
* Noteworthy changes in release 1.21.1 (2021-01-09)
** Fix compilation on MacOS and Solaris 9
** Resove bashism from configure.ac
** Fix a compilation warning on 32-bit systems
* Changes in Wget 1.21
** Improve the number of translated strings
** Remove all uses of alloca
In some places the length of untrusted strings has been used, e.g.
strings from the command line or from remote.
** Fix buffer overflows in progress bar code in some locales
** Fix two null pointer accesses
** Amend cookie file header to be recognized by the 'file' command
** Post Handshake Authentication for OpenSSL
** Require gettext version 0.19.3+
** Add configure flags --enable-fsanitize-ubsan, --enable-fsanitize-asan
and --enable-fsanitize-msan for gcc and clang
** Make several smaller fixes, enhance fuzzing, enhance building
* Changes in Wget 1.20.3
** Fixed a buffer overflow vulnerability
* Changes in Wget 1.20.2
** NTLM authentication will retry under certain cases
* Changes in Wget 1.20.1
** --xattr is no longer default since it introduces privacy issues.
** --xattr saves the Referer as scheme/host/port, user/pw/path/query/fragment
are no longer saved to prevent privacy issues.
** --xattr saves the Original URL without user/password to prevent
privacy issues.
* Changes in Wget 1.20
** Add new option `--retry-on-host-error` to treat local errors as transient
and hence Wget will retry to download the file after a brief waiting period.
** Fixed multiple potential resource leaks as found by static analysis
** Wget will now not create an empty wget-log file when running with -q and -b
switches together
** When compiled using the GnuTLS >= 3.6.3, Wget now has support for TLSv1.3
** Now there is support for using libpcre2 for regex pattern matching
** When downloading over FTP recursively, one can now use the
--{accept,reject}-regex switches to fine-tune the downloaded files
** Building Wget from the git sources now requires autoconf 2.63 or above.
Building from the Tarballs works as it used to.
Please send GNU Wget bug reports to <bug-wget@gnu.org>.
* Changes in Wget 1.19.5
@@ -349,7 +461,7 @@ Please send GNU Wget bug reports to <bug-wget@gnu.org>.
backward-incompatibility; any script that relies on the old
behaviour must use --trust-server-names.
** Fix a problem when -k is used and some URLs are specified trough
** Fix a problem when -k is used and some URLs are specified through
CSS.
** Convert correctly URLs that need to be encoded to local files when following
@@ -359,7 +471,7 @@ Please send GNU Wget bug reports to <bug-wget@gnu.org>.
** Print the total download time as part of the summary for recursive downloads.
** Now it is possible to specify a different startup configuration file trough
** Now it is possible to specify a different startup configuration file through
the --config option.
** Fix an infinite loop with the error '<filename> has sprung into existence'
@@ -1087,7 +1199,7 @@ geturl -vo log http://fly.cc.fer.hr/
----------------------------------------------------------------------
Copyright information:
Copyright (C) 1997-2005 Free Software Foundation, Inc.
Copyright (C) 1997-2023 Free Software Foundation, Inc.
Permission is granted to anyone to make or distribute verbatim
copies of this document as received, in any medium, provided that

7
README
View File

@@ -14,7 +14,7 @@ While doing that, Wget respects the Robot Exclusion Standard
(/robots.txt). Wget can be instructed to convert the links in
downloaded HTML files to the local files for offline viewing.
Recursive downloading also works with FTP, where Wget can retrieves a
Recursive downloading also works with FTP, where Wget can retrieve a
hierarchy of directories and files.
With both HTTP and FTP, Wget can check whether a remote file has
@@ -53,15 +53,14 @@ See the file `MAILING-LIST' for information about Wget mailing lists.
Wget's home page is at <https://www.gnu.org/software/wget/>.
If you would like to contribute code for Wget, please read
https://wget.addictivecode.org/PatchGuidelines.
CONTRIBUTING.md.
Wget was originally written and mainained by Hrvoje Niksic. Please see
the file AUTHORS for a list of major contributors, and the ChangeLogs
for a detailed listing of all contributions.
Copyright (C) 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004
2005, 2006, 2007, 2008, 2009 Free Software Foundation, Inc.
Copyright (C) 1995-2023 Free Software Foundation, Inc.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by

View File

@@ -8,7 +8,7 @@ Compiling From Repository Sources
libiconv, libpsl, libpcre, pkg-config, libmetalink and GnuPG are not
absolutely required):
* [20]autoconf (currently, GNU Wget requires version 2.61). This is
* [20]autoconf (currently, GNU Wget requires version 2.64). This is
needed to generate the configure script from configure.in. This is
not required when building from a tarball distribution; only when
building from repository sources.
@@ -76,7 +76,7 @@ Compiling From Repository Sources
On GNU systems, the functionality provided by libiconv is already
present in the system libraries.
* [33]git is used to fetch gnulib files trough the bootstrap.sh script.
* [33]git is used to fetch gnulib files through the bootstrap.sh script.
* [34]libpsl is required for using a public suffix list to check for valid
cookie domains. You need the header files and the library installed.
@@ -104,8 +104,8 @@ Compiling From Repository Sources
The command line options --dns-servers and --bind-dns-address are
only available when configured with --with-cares.
* [49]Gpef is required by some gnulib modules, most notably "unicase". You
need the application installed on the machine to compile Wget from git.
* [49]gperf is required by some gnulib modules, most notably "unicase". You
need the application installed on the machine to compile Wget from git.
For those who might be confused as to what to do once they check out
@@ -161,8 +161,8 @@ Testing and development
that bug, which might be tedious. Tests should only be run through GDB for that purpose.
Copyright (C) 2008, 2010, 2014-2015, 2018 Free Software Foundation,
Inc.
Copyright (C) 2008, 2010, 2014-2015, 2018-2023 Free Software
Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<https://www.gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.

1581
bootstrap

File diff suppressed because it is too large Load Diff

View File

@@ -1,5 +1,5 @@
# bootstrap.conf - Bootstrap configuration.
# Copyright (C) 2007-2012, 2018 Free Software Foundation, Inc.
# Copyright (C) 2007-2012, 2018-2023 Free Software Foundation, Inc.
#
# This file is part of GNU Wget.
#
@@ -17,15 +17,16 @@
# along with this program. If not, see `https://www.gnu.org/licenses/'.
# gnulib library name
source_base=lib
tests_base=lib/tests
gnulib_name=libgnu
gnulib_tool_option_extras=
gnulib_tool_option_extras="--makefile-name=gnulib.mk --po-base=gnulib_po --po-domain=wget"
use_libtool=0
checkout_only_file=
# gnulib modules used by this package.
gnulib_modules="
accept
alloca
announce-gen
base32
bind
@@ -38,10 +39,11 @@ connect
dirname
fcntl
flock
fnmatch
fnmatch-gnu
fopen
futimens
ftello
gendocs
getaddrinfo
getline
getopt-gnu
@@ -52,6 +54,7 @@ gettext-h
gitlog-to-changelog
git-version-gen
gnupload
gnu-web-doc-update
group-member
hostent
ioctl
@@ -85,6 +88,7 @@ quote
quotearg
recv
regex
rename
select
send
setsockopt
@@ -115,39 +119,27 @@ libunistring-optional
unistr/u8-strlen
unicase/u8-tolower
utime
utime-h
utimens
vasprintf
vsnprintf
warnings
wcwidth
write
xmemdup0
xstrndup
"
gnulib_extra_files="
$build_aux/install-sh
$build_aux/mdate-sh
$build_aux/texinfo.tex
$build_aux/depcomp
$build_aux/config.guess
$build_aux/config.sub
$build_aux/config.rpath
doc/INSTALL
"
# Build prerequisites
buildreq="\
autoconf 2.62
autoconf 2.64
automake 1.11.1
autopoint -
gettext 0.17
gettext -
git 1.4.4
makeinfo -
perl 5.5
rsync -
tar -
xz -
gzip -
flex -
gperf -
"
@@ -166,4 +158,10 @@ bootstrap_post_import_hook ()
sed -e "s/gl_unicase_special_lookup.*/gl_unicase_special_lookup\ \(const char \*str, size_t len\)/g" \
> lib/unicase/special-casing-table.h
fi
# ./bootstrap calls 'autopoint --force' which will overwrite
# po/Makevars.template. So here we add what we need for a VPATH
# build. ./bootstrap will create po/Makevars from po/Makevars.template
# after our change.
echo 'PACKAGE_GNU = yes' >> po/Makevars.template
}

View File

@@ -2,7 +2,7 @@
# Generate build_info.c.
# Copyright (C) 2009-2011, 2018 Free Software Foundation, Inc.
# Copyright (C) 2009-2011, 2018-2023 Free Software Foundation, Inc.
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by

26
cfg.mk
View File

@@ -1,37 +1,37 @@
# Explicit syntax-check exceptions.
exclude_file_name_regexp--sc_trailing_blank = ^doc/annou.*$$
exclude_file_name_regexp--sc_bindtextdomain = ^fuzz/.*.c$$
exclude_file_name_regexp--sc_require_config_h = fuzz/main.c$$
exclude_file_name_regexp--sc_require_config_h_first = fuzz/main.c$$
exclude_file_name_regexp--sc_require_config_h = (fuzz/.*.c|src/decc_ver.c|util/trunc.c)$$
exclude_file_name_regexp--sc_require_config_h_first = (fuzz/.*.c|src/decc_ver.c|src/vms.c|util/trunc.c)$$
exclude_file_name_regexp--sc_file_system = ^(NEWS|testenv/server/ftp/ftp_server.py)$$
export VC_LIST_EXCEPT_DEFAULT=^(lib/.*|m4/.*|md5/.*|build-aux/.*|src/gettext\.h|.*ChangeLog|tests/certs/.*)$$
config_h_header = "wget\.h"
local-checks-to-skip = \
sc_const_long_option \
sc_error_message_uppercase \
sc_file_system \
sc_immutable_NEWS \
sc_copyright_check \
sc_makefile_path_separator_check \
sc_require_config_h \
sc_require_config_h_first \
sc_two_space_separator_in_usage \
sc_useless_cpp_parens \
\
sc_prohibit_atoi_atof \
sc_prohibit_strcmp \
sc_cast_of_alloca_return_value \
sc_cast_of_x_alloc_return_value \
sc_prohibit_S_IS_definition \
\
sc_program_name \
sc_makefile_at_at_check \
sc_cast_of_argument_to_free \
sc_prohibit_undesirable_word_seq
sc_prohibit_gnu_make_extensions \
\
sc_indent
#SHELL=bash -x
show-vc-list-except:
@$(VC_LIST_EXCEPT)
VC_LIST_ALWAYS_EXCLUDE_REGEX = ^ABOUT-NLS|((msdos|vms)/.*)|ChangeLog-2014-12-10|/.*\.der|^tests/certs/.*|^fuzz/.*\.(in|repro)/.*$$
update-copyright-env = UPDATE_COPYRIGHT_USE_INTERVALS=1
generated_files =
old_NEWS_hash = 39db0b0f2cce0f250b4f5b8891990f17
manual_title = $(VERSION)

View File

@@ -1,5 +1,5 @@
dnl Template file for GNU Autoconf
dnl Copyright (C) 1995-1997, 2001, 2007-2014, 2018 Free Software
dnl Copyright (C) 1995-1997, 2001, 2007-2014, 2018-2023 Free Software
dnl Foundation, Inc.
dnl This program is free software; you can redistribute it and/or modify
@@ -30,10 +30,13 @@ dnl
dnl Process this file with autoconf to produce a configure script.
dnl
AC_INIT([wget],
m4_esyscmd([build-aux/git-version-gen .tarball-version]),
[bug-wget@gnu.org])
AC_PREREQ(2.61)
dnl Error if AX_CODE_COVERAGE if missing (like on the 1.21 release).
m4_pattern_forbid([^_?AX_])
m4_pattern_allow([AX_CHECK_GNU_MAKE_HEADLINE])
m4_pattern_allow([_AX_CODE_COVERAGE_GCOV_PROG_WITH])
AC_INIT([wget],[m4_esyscmd(build-aux/git-version-gen .tarball-version)],[bug-wget@gnu.org])
AC_PREREQ([2.64])
dnl
dnl What version of Wget are we building?
@@ -48,10 +51,13 @@ AC_CONFIG_SRCDIR([src/wget.h])
dnl
dnl Automake setup
dnl
AM_INIT_AUTOMAKE([1.9])
AM_INIT_AUTOMAKE([subdir-objects dist-lzip])
AC_USE_SYSTEM_EXTENSIONS
AC_PROG_CC
dnl
dnl Get cannonical host
dnl Get canonical host
dnl
AC_CANONICAL_HOST
AC_DEFINE_UNQUOTED([OS_TYPE], "$host_os",
@@ -70,7 +76,7 @@ AC_ARG_WITH([libpsl],
dnl SSL: Configure SSL backend to use
AC_ARG_WITH([ssl],
[AS_HELP_STRING([--with-ssl={gnutls,openssl}], [specify SSL backend. GNU TLS is the default.])])
[AS_HELP_STRING([--with-ssl={gnutls,openssl,no}], [specify SSL backend. GNU TLS is the default.])])
dnl Zlib: Configure use of zlib for compression
AC_ARG_WITH([zlib],
@@ -93,16 +99,69 @@ AC_ARG_ENABLE([fuzzing],
AC_SUBST([LIB_FUZZING_ENGINE])
AC_DEFINE([FUZZING], 1, [Define to 1 if this is a fuzzing build])
], [enable_fuzzing=no; LIB_FUZZING_ENGINE=""])
#FUZZ_LIBS=$LIBS
#if test $enable_fuzzing = "yes"; then
# OLD_LIBS=$LIBS
# AC_SEARCH_LIBS([dlsym], [dl dld])
# FUZZ_LIBS=$LIBS
# LIBS=$OLD_LIBS
#fi
#AC_SUBST([FUZZ_LIBS])
AM_CONDITIONAL([FUZZING], [test "$enable_fuzzing" = "yes"])
dnl needed for some fuzzers
OLD_LIBS="$LIBS"
AC_SEARCH_LIBS([dlopen], [dl dld])
if [[ "${ac_cv_search_dlopen#-l}" != "${ac_cv_search_dlopen}" ]]; then
FUZZ_LIBS="${ac_cv_search_dlopen}"
AC_SUBST([FUZZ_LIBS])
fi
LIBS="$OLD_LIBS"
AC_ARG_ENABLE([fsanitize-ubsan],
[AS_HELP_STRING([--enable-fsanitize-ubsan], [Turn on Undefined Behavior Sanitizer (for developers)])],
[gl_cc_sanitize_ubsan=yes], [gl_cc_sanitize_ubsan=no])
AC_ARG_ENABLE([fsanitize-asan],
[AS_HELP_STRING([--enable-fsanitize-asan], [Turn on Address Sanitizer (for developers) (mutually exclusive with Memory/Thread sanitizer or Valgrind tests)])],
[gl_cc_sanitize_asan=yes], [gl_cc_sanitize_asan=no])
AC_ARG_ENABLE([fsanitize-msan],
[AS_HELP_STRING([--enable-fsanitize-msan], [Turn on Memory Sanitizer (for developers) (mutually exclusive with Address/Thread sanitizer or Valgrind tests)])],
[gl_cc_sanitize_msan=yes], [gl_cc_sanitize_msan=no])
if test "$gl_cc_sanitize_asan" = yes; then
if test "$gl_cc_sanitize_msan" = yes; then
AC_MSG_ERROR([Address Sanitizer and Memory Sanitizer are mutually exclusive])
fi
fi
if test "$gl_cc_sanitize_ubsan" = yes; then
gl_WARN_ADD([-fsanitize=undefined])
gl_WARN_ADD([-fno-sanitize-recover=undefined])
# additional clang options
gl_WARN_ADD([-fsanitize=integer])
gl_WARN_ADD([-fsanitize=nullability])
gl_WARN_ADD([-fno-sanitize-recover=nullability])
fi
if test "$gl_cc_sanitize_asan" = yes; then
gl_WARN_ADD([-fsanitize=address])
gl_WARN_ADD([-fno-omit-frame-pointer])
# additional clang option
gl_WARN_ADD([-fsanitize-address-use-after-scope])
fi
if test "$gl_cc_sanitize_msan" = yes; then
# clang options
gl_WARN_ADD([-fsanitize=memory])
gl_WARN_ADD([-fsanitize-memory-track-origins])
# gcc options
if test "$CCNAME" = "gcc"; then
gl_WARN_ADD([-fsanitize=leak])
fi
gl_WARN_ADD([-fno-omit-frame-pointer])
gl_WARN_ADD([-fPIE])
fi
dnl Opie: Support for opie s/key FTP logins
AC_ARG_ENABLE([opie],
[AS_HELP_STRING([--disable-opie], [disable support for opie or s/key FTP login])],
@@ -155,6 +214,12 @@ AC_ARG_ENABLE(valgrind-tests,
AS_IF([test "x$ENABLE_VALGRIND" != xno], [
AC_CHECK_PROG(HAVE_VALGRIND, valgrind, yes, no)
AS_IF([test "x$HAVE_VALGRIND" = xyes], [
if test "$gl_cc_sanitize_asan" = yes; then
AC_MSG_ERROR([Valgrind and Address Sanitizer are mutually exclusive])
elif test "$gl_cc_sanitize_msan" = yes; then
AC_MSG_ERROR([Valgrind and Memory Sanitizer are mutually exclusive])
fi
VALGRIND_TESTS="1"
AC_SUBST(VALGRIND_TESTS)
VALGRIND_INFO="Test suite will be run under Valgrind"
@@ -184,10 +249,6 @@ dnl We want these before the checks, so the checks can modify their values.
test -z "$CFLAGS" && CFLAGS= auto_cflags=1
test -z "$CC" && cc_specified=yes
AC_PROG_CC
AM_PROG_CC_C_O
AC_AIX
gl_EARLY
dnl
@@ -200,11 +261,12 @@ dnl
dnl Gettext
dnl
AM_GNU_GETTEXT([external],[need-ngettext])
AM_GNU_GETTEXT_VERSION([0.17])
AM_GNU_GETTEXT_VERSION([0.19.3])
AM_GNU_GETTEXT_REQUIRE_VERSION([0.19.3])
AC_PROG_RANLIB
AC_PROG_LEX
AC_PROG_LEX([noyywrap])
dnl Turn on optimization by default. Specifically:
dnl
@@ -234,13 +296,6 @@ AC_C_CONST
AC_C_INLINE
AC_C_VOLATILE
dnl Check for basic headers, even though we expect them to exist and
dnl #include them unconditionally in the code. Their detection is
dnl still needed because test programs used by Autoconf macros check.
dnl Without the checks they will fail to be included in test programs,
dnl which will subsequently fail.
AC_HEADER_STDC
dnl Check for large file support. This check needs to come fairly
dnl early because it could (in principle) affect whether functions and
dnl headers are available, whether they work, etc.
@@ -252,7 +307,7 @@ dnl Checks for system header files that might be missing.
dnl
AC_HEADER_STDBOOL
AC_CHECK_HEADERS(unistd.h sys/time.h)
AC_CHECK_HEADERS(termios.h sys/ioctl.h sys/select.h utime.h sys/utime.h)
AC_CHECK_HEADERS(termios.h sys/ioctl.h sys/select.h)
AC_CHECK_HEADERS(stdint.h inttypes.h pwd.h wchar.h dlfcn.h)
AC_CHECK_DECLS(h_errno,,,[#include <netdb.h>])
@@ -262,11 +317,7 @@ dnl Check sizes of integer types. These are used to find n-bit
dnl integral types on older systems that fail to provide intN_t and
dnl uintN_t typedefs.
dnl
AC_CHECK_SIZEOF([short])
AC_CHECK_SIZEOF([int])
AC_CHECK_SIZEOF([long])
AC_CHECK_SIZEOF([long long])
AC_CHECK_SIZEOF([void *])
dnl
dnl Checks for non-universal or system-specific types.
@@ -281,9 +332,12 @@ AC_CHECK_TYPES(sig_atomic_t, [], [], [
#include <signal.h>
])
# gnulib
# gnulib, AM_ICONV should be placed before gl_INIT
AM_ICONV
gl_INIT
AX_CODE_COVERAGE
dnl
dnl Checks for library functions.
dnl
@@ -293,26 +347,137 @@ AC_CHECK_FUNCS(strptime timegm vsnprintf vasprintf drand48 pathconf)
AC_CHECK_FUNCS(strtoll usleep ftello sigblock sigsetjmp memrchr wcwidth mbtowc)
AC_CHECK_FUNCS(sleep symlink utime strlcpy random fmemopen)
if test x"$ENABLE_OPIE" = xyes; then
AC_LIBOBJ([ftp-opie])
fi
dnl We expect to have these functions on Unix-like systems configure
dnl runs on. The defines are provided to get them in config.h.in so
dnl Wget can still be ported to non-Unix systems (such as Windows)
dnl that lack some of these functions.
AC_DEFINE([HAVE_STRCASECMP], 1, [Define to 1 if you have the `strcasecmp' function.])
AC_DEFINE([HAVE_STRNCASECMP], 1, [Define to 1 if you have the `strncasecmp' function.])
AC_DEFINE([HAVE_STRDUP], 1, [Define to 1 if you have the `strdup' function.])
AC_DEFINE([HAVE_ISATTY], 1, [Define to 1 if you have the `isatty' function.])
AC_DEFINE([HAVE_STRDUP], 1, [Define to 1 if you have the 'strdup' function.])
AC_DEFINE([HAVE_ISATTY], 1, [Define to 1 if you have the 'isatty' function.])
dnl Deal with specific hosts
case $host_os in
*mingw32* )
LIBS+=' -lws2_32'
AC_LIBOBJ([mswindows])
OS_USED="mswindows"
;;
esac
AM_CONDITIONAL([OS_MSWINDOWS], [test x"$OS_USED" = x"mswindows"])
# enable all possible compiler warnings in WARN_FLAGS
#
# to enable by default: create a file '.manywarnings'
# enable explicitly : ./configure --enable-manywarnings
# disable explicitly: ./configure --disable-manywarnings
sanitizer_flags="$WARN_CFLAGS"
wget_MANYWARNINGS(WARN_CFLAGS, C)
WARN_CFLAGS="$WARN_CFLAGS $sanitizer_flags"
if test -n "$WARN_CFLAGS"; then
if test "$CCNAME" = "gcc"; then
# Set up list of unwanted warnings
nw=
nw="$nw -Wsystem-headers" # System headers may trigger lot's of useless warnings
nw="$nw -Wvla" # This project is C99
nw="$nw -Wstack-protector"
nw="$nw -Wmissing-field-initializer"
nw="$nw -Wtraditional"
nw="$nw -Wtraditional-conversion"
nw="$nw -Wc++-compat"
nw="$nw -Wcast-qual"
nw="$nw -Wconversion"
nw="$nw -Wsign-conversion"
nw="$nw -Wunsuffixed-float-constants"
nw="$nw -Wdeclaration-after-statement" # C89 only, messing up gcc < 5
nw="$nw -Wcast-function-type" # gcc 8, very noisy
nw="$nw -Wabi" # gcc 8, very noisy
nw="$nw -Wunused-macros" # triggers in auto-generated lex css parser, #pragma doesn't work, conflicts with -Werror
nw="$nw -Wchkp" # Deprecated option
nw="$nw -Wswitch-enum" # Too noisy
nw="$nw -Wswitch-default" # TODO: Enable someday. Too noisy for now
nw="$nw -Wpedantic" # GCC 11. Too noisy, will never be supported
nw="$nw -Wnested-externs" # Stylistic choice and we do use it in hsts code
nw="$nw -Wc90-c99-compat" # gcc-12, we use C99
nw="$nw -Wlong-long" # gcc-12, we use C99
if test "$cross_compiling" = yes; then
nw="$nw -Wformat"
fi
# remove unwanted warn flags
wget_WORD_REMOVE([WARN_CFLAGS], [$WARN_CFLAGS], [$nw])
WARN_CFLAGS="$WARN_CFLAGS -Wno-undef -Wno-float-equal"
# add more flags as you like
if test $GCC_VERSION -ge 5; then
WARN_CFLAGS="$WARN_CFLAGS -fdiagnostics-color=always"
fi
if test "$cross_compiling" = yes; then
WARN_CFLAGS="$WARN_CFLAGS -Wno-format"
fi
if test $GCC_VERSION -ge 8; then
WARN_CFLAGS="$WARN_CFLAGS -Wno-cast-function-type"
fi
# WARN_CFLAGS="$WARN_CFLAGS -Werror"
# We use a slightly smaller set of warning options for lib/.
# Remove the following and save the result in GNULIB_WARN_CFLAGS.
# Removing is not enough if these switches are implicitly set by other
# flags like -Wall or -Wextra. We have to explicitly unset them
# with -Wno-....
nw=
nw="$nw -Wpedantic"
nw="$nw -Wsign-compare"
nw="$nw -Wunused-parameter"
nw="$nw -Wswitch-default"
nw="$nw -Wformat-nonliteral"
nw="$nw -Wsuggest-attribute=pure"
nw="$nw -Wunsafe-loop-optimizations"
nw="$nw -Wundef"
nw="$nw -Wswitch-enum"
nw="$nw -Wbad-function-cast"
nw="$nw -Wredundant-decls"
nw="$nw -Werror"
wget_WORD_REMOVE([GNULIB_WARN_CFLAGS], [$WARN_CFLAGS], [$nw])
wget_WORD_REMOVE([CFLAGS], [$CFLAGS], [-Werror])
# disable options implicitly set by other options
GNULIB_WARN_CFLAGS="-Wno-error $GNULIB_WARN_CFLAGS"
GNULIB_WARN_CFLAGS="$GNULIB_WARN_CFLAGS -Wno-sign-compare -Wno-unused-parameter -Wno-alloca"
GNULIB_WARN_CFLAGS="$GNULIB_WARN_CFLAGS -Wno-float-conversion -Wno-cast-function-type"
if test "$cross_compiling" = yes; then
GNULIB_WARN_CFLAGS="$GNULIB_WARN_CFLAGS -Wno-incompatible-pointer-types"
fi
elif test "$CCNAME" = "clang"; then
# setup flags for this project
WARN_CFLAGS="$WARN_CFLAGS -Wno-system-headers -Wno-vla -Wno-cast-qual -Wno-padded"
WARN_CFLAGS="$WARN_CFLAGS -Wno-reserved-id-macro -Wno-sign-conversion -Wno-disabled-macro-expansion"
WARN_CFLAGS="$WARN_CFLAGS -Wno-documentation -Wno-documentation-unknown-command"
WARN_CFLAGS="$WARN_CFLAGS -Wno-covered-switch-default -Wno-unused-macros"
WARN_CFLAGS="$WARN_CFLAGS -Wno-missing-field-initializers"
# remove all flags from WARN_FLAGS that are already in CFLAGS
# wget_WORD_REMOVE([WARN_CFLAGS], [$WARN_CFLAGS], [$CFLAGS])
wget_WORD_REMOVE([CFLAGS], [$CFLAGS], [-Werror])
# disable verbose options
GNULIB_WARN_CFLAGS="-Wno-error $GNULIB_WARN_CFLAGS"
GNULIB_WARN_CFLAGS="$WARN_CFLAGS -Wno-sign-compare -Wno-unused-parameter -Wno-undef -Wno-format-nonliteral"
GNULIB_WARN_CFLAGS="$GNULIB_WARN_CFLAGS -Wno-conversion -Wno-disabled-macro-expansion -Wno-c++98-compat"
GNULIB_WARN_CFLAGS="$GNULIB_WARN_CFLAGS -Wno-gnu-statement-expression -Wno-shorten-64-to-32 -Wno-switch-enum"
GNULIB_WARN_CFLAGS="$GNULIB_WARN_CFLAGS -Wno-unused-macros -Wno-missing-field-initializers"
GNULIB_WARN_CFLAGS="$GNULIB_WARN_CFLAGS -Wno-gnu-zero-variadic-macro-arguments -Wno-conditional-uninitialized"
GNULIB_WARN_CFLAGS="$GNULIB_WARN_CFLAGS -Wno-comma -Wno-assign-enum -Wno-unreachable-code -Wno-error"
GNULIB_WARN_CFLAGS="$GNULIB_WARN_CFLAGS -Wno-missing-field-initializers"
# remove all flags from GNULIB_WARN_FLAGS that are already in CFLAGS
# wget_WORD_REMOVE([GNULIB_WARN_CFLAGS], [$GNULIB_WARN_CFLAGS], [$CFLAGS])
fi
AC_SUBST([WARN_CFLAGS])
AC_SUBST([GNULIB_WARN_CFLAGS])
fi
dnl
@@ -355,7 +520,7 @@ AS_IF([test x"$with_ssl" = xopenssl], [
if [test x"$with_libssl_prefix" = x]; then
PKG_CHECK_MODULES([OPENSSL], [openssl], [
AC_MSG_NOTICE([compiling in support for SSL via OpenSSL])
AC_LIBOBJ([openssl])
ssl_library="openssl"
LIBS="$OPENSSL_LIBS $LIBS"
CFLAGS="$OPENSSL_CFLAGS -DHAVE_LIBSSL $CFLAGS"
LIBSSL=" " # ntlm check below wants this
@@ -384,9 +549,9 @@ AS_IF([test x"$with_ssl" = xopenssl], [
AC_CHECK_LIB(ssl32, SSL_connect, [
ssl_found=yes
AC_MSG_NOTICE([Enabling support for SSL via OpenSSL (shared)])
AC_LIBOBJ([openssl])
ssl_library="openssl"
LIBS="${LIBS} -lssl32"
AC_DEFINE([HAVE_LIBSSL32], [1], [Define to 1 if you have the `ssl32' library (-lssl32).])
AC_DEFINE([HAVE_LIBSSL32], [1], [Define to 1 if you have the 'ssl32' library (-lssl32).])
],
AC_MSG_ERROR([openssl not found: shared lib eay32 found but ssl32 not found]))
@@ -413,7 +578,7 @@ AS_IF([test x"$with_ssl" = xopenssl], [
then
ssl_found=yes
AC_MSG_NOTICE([compiling in support for SSL via OpenSSL])
AC_LIBOBJ([openssl])
ssl_library="openssl"
LIBS="$LIBSSL $LIBS"
AC_CHECK_FUNCS([RAND_egd])
elif test x"$with_ssl" != x
@@ -432,7 +597,7 @@ AS_IF([test x"$with_ssl" = xopenssl], [
if [test x"$with_libgnutls_prefix" = x]; then
PKG_CHECK_MODULES([GNUTLS], [gnutls], [
AC_MSG_NOTICE([compiling in support for SSL via GnuTLS])
AC_LIBOBJ([gnutls])
ssl_library="gnutls"
LIBS="$GNUTLS_LIBS $LIBS"
CFLAGS="$GNUTLS_CFLAGS -DHAVE_LIBGNUTLS $CFLAGS"
AC_DEFINE([HAVE_LIBGNUTLS], [1], [Define if using gnutls.])
@@ -447,7 +612,7 @@ AS_IF([test x"$with_ssl" = xopenssl], [
then
ssl_found=yes
AC_MSG_NOTICE([compiling in support for SSL via GnuTLS])
AC_LIBOBJ([gnutls])
ssl_library="gnutls"
LIBS="$LIBGNUTLS $LIBS"
else
AC_MSG_ERROR([GnuTLS has not been found. Use --with-ssl=openssl if you explicitly want OpenSSL.])
@@ -459,69 +624,37 @@ AS_IF([test x"$with_ssl" = xopenssl], [
]) # endif: --with-ssl == openssl?
dnl Enable NTLM if requested and if SSL is available.
if test x"$LIBSSL" != x || test "$ac_cv_lib_ssl32_SSL_connect" = yes
if test x"$ENABLE_NTLM" != xno
then
if test x"$ENABLE_NTLM" != xno
if test x"$LIBSSL" != x || test "$ac_cv_lib_ssl32_SSL_connect" = yes
then
ENABLE_NTLM=yes
AC_DEFINE([ENABLE_NTLM], 1,
[Define if you want the NTLM authorization support compiled in.])
AC_LIBOBJ([http-ntlm])
fi
else
AC_CHECK_LIB(nettle, nettle_md4_init, [HAVE_NETTLE=yes], [HAVE_NETTLE=no; AC_MSG_WARN(*** libnettle was not found. You will not be able to use NTLM)])
if test x"$HAVE_NETTLE" = xyes
then
AC_SUBST(NETTLE_LIBS, "-lnettle")
AC_DEFINE([HAVE_NETTLE], [1], [Use libnettle])
if test x"$ENABLE_NTLM" != xno
then
ENABLE_NTLM=yes
AC_DEFINE([ENABLE_NTLM], 1,
[Define if you want the NTLM authorization support compiled in.])
AC_LIBOBJ([http-ntlm])
LIBS="$NETTLE_LIBS $LIBS"
fi
AC_DEFINE([ENABLE_NTLM], 1, [Define if you want the NTLM authorization support compiled in.])
else
dnl If SSL is unavailable and the user explicitly requested NTLM,
dnl abort.
if test x"$ENABLE_NTLM" = xyes
then
AC_MSG_ERROR([NTLM authorization requested and SSL not enabled; aborting])
PKG_CHECK_MODULES([NETTLE], nettle, [
HAVE_NETTLE=yes
LIBS="$NETTLE_LIBS $LIBS"
CFLAGS="$NETTLE_CFLAGS $CFLAGS"
], [
AC_CHECK_LIB(nettle, nettle_md4_init, [HAVE_NETTLE=yes], [HAVE_NETTLE=no; AC_MSG_WARN(*** libnettle was not found. You will not be able to use NTLM)])
if test x"$HAVE_NETTLE" != xyes; then
if test x"$ENABLE_NTLM" = xyes; then
AC_MSG_ERROR([NTLM authorization requested and SSL not enabled; aborting])
fi
else
AC_SUBST(NETTLE_LIBS, "-lnettle")
LIBS="$NETTLE_LIBS $LIBS"
fi
])
if test x"$HAVE_NETTLE" = xyes; then
AC_DEFINE([HAVE_NETTLE], [1], [Use libnettle])
ENABLE_NTLM=yes
AC_DEFINE([ENABLE_NTLM], 1, [Define if you want the NTLM authorization support compiled in.])
fi
fi
fi
dnl
dnl Check for libmetalink
dnl
AS_IF([test x"$with_metalink" != xno], [
PKG_CHECK_MODULES([METALINK], libmetalink, [
LIBS="$METALINK_LIBS $LIBS"
CFLAGS="$METALINK_CFLAGS $CFLAGS"
AC_DEFINE([HAVE_METALINK], [1], [Define if using metalink.])
with_metalink=yes
], [
with_metalink=no
])
have_gpg=no
AS_IF([test x"$with_metalink" = xyes], [
dnl
dnl Check for GPGME
dnl
m4_ifdef([AM_PATH_GPGME], [
AM_PATH_GPGME([], [
LIBS="$GPGME_LIBS $LIBS"
CFLAGS="$GPGME_CFLAGS $CFLAGS"
AC_DEFINE([HAVE_GPGME], [1], [Define if GPGME is available.])
have_gpg=yes
])
])
])
])
dnl **********************************************************************
dnl Checks for IPv6
dnl **********************************************************************
@@ -539,7 +672,7 @@ dnl is to autodetect IPv6 and use it where available.
dnl
AC_ARG_ENABLE(ipv6,
AC_HELP_STRING([--disable-ipv6],[disable IPv6 support]),
AS_HELP_STRING([--disable-ipv6],[disable IPv6 support]),
[case "${enable_ipv6}" in
no)
AC_MSG_NOTICE([disabling IPv6 at user request])
@@ -556,7 +689,7 @@ AC_ARG_ENABLE(ipv6,
ipv6=yes
;;
*)
AC_MSG_ERROR([Invalid --enable-ipv6 argument \`$enable_ipv6'])
AC_MSG_ERROR([Invalid --enable-ipv6 argument '$enable_ipv6'])
;;
esac
], [
@@ -574,7 +707,7 @@ fi
if test "X$ipv6" = "Xyes"; then
TYPE_STRUCT_SOCKADDR_IN6([],[
AC_MSG_NOTICE([Disabling IPv6 support: your system does not support \`struct sockaddr_in6'])
AC_MSG_NOTICE([Disabling IPv6 support: your system does not support 'struct sockaddr_in6'])
ipv6=no
])
if test "X$ipv6" = "Xyes"; then
@@ -592,9 +725,9 @@ fi
dnl
dnl Find makeinfo. We used to provide support for Emacs processing
dnl Texinfo using `emacs -batch -eval ...' where makeinfo is
dnl Texinfo using 'emacs -batch -eval ...' where makeinfo is
dnl unavailable, but that broke with the addition of makeinfo-specific
dnl command-line options, such as `-I'. Now we depend on makeinfo to
dnl command-line options, such as '-I'. Now we depend on makeinfo to
dnl build the Info documentation.
dnl
@@ -620,7 +753,7 @@ dnl Check for IDN/IRIs
dnl
AC_ARG_ENABLE(iri,
AC_HELP_STRING([--disable-iri],[disable IDN/IRIs support]),
AS_HELP_STRING([--disable-iri],[disable IDN/IRIs support]),
[case "${enable_iri}" in
no)
dnl Disable IRIs checking
@@ -637,7 +770,7 @@ AC_ARG_ENABLE(iri,
iri=yes
;;
*)
AC_MSG_ERROR([Invalid --enable-iri argument \`$enable_iri'])
AC_MSG_ERROR([Invalid --enable-iri argument '$enable_iri'])
;;
esac
], [
@@ -647,8 +780,6 @@ AC_ARG_ENABLE(iri,
)
AS_IF([test "X$iri" != "Xno"],[
AM_ICONV
if test "X$am_cv_func_iconv" != "Xyes"; then
iri=no
if test "X$force_iri" = "Xyes"; then
@@ -659,20 +790,18 @@ AS_IF([test "X$iri" != "Xno"],[
fi
])
AC_ARG_WITH(libidn, AC_HELP_STRING([--with-libidn=[DIR]],
[Support IDN2008/IRIs (needs GNU libidn2 + libunicode)]),
libidn=$withval, libidn="")
if test "X$iri" != "Xno"; then
AS_IF([test "x$with_libidn2" != xno], [
AC_SEARCH_LIBS(idn2_lookup_u8, idn2,
[with_libidn2=yes; AC_DEFINE([ENABLE_IRI], 1, [Define if IRI support is enabled.])],
[with_libidn2=no; iri=no; AC_MSG_WARN(*** LIBIDN2 was not found. You will not be able to use IDN2008 support)])
# AS_IF([test "x$with_libidn2" = xyes], [
# AC_SEARCH_LIBS(u8_tolower, unistring,
# [AC_DEFINE([ENABLE_IRI], 1, [Define if IRI support is enabled.])],
# [iri=no; AC_MSG_WARN(*** LIBUNISTRING was not found. You will not be able to use IDN2008 support)])
# ])
PKG_CHECK_MODULES([LIBIDN2], [libidn2 >= 0.14.0], [
with_libidn2=yes
LIBS="$LIBIDN2_LIBS $LIBS"
CFLAGS="$LIBIDN2_CFLAGS $CFLAGS"
AC_DEFINE([ENABLE_IRI], 1, [Define if IRI support is enabled.])
], [
AC_SEARCH_LIBS(idn2_lookup_u8, idn2,
[with_libidn2=yes; AC_DEFINE([ENABLE_IRI], 1, [Define if IRI support is enabled.])],
[with_libidn2=no; iri=no; AC_MSG_WARN(*** LIBIDN2 was not found. You will not be able to use IDN2008 support)])
])
])
fi
@@ -680,8 +809,7 @@ dnl
dnl Check for UUID
dnl
AC_ARG_WITH(libuuid, AC_HELP_STRING([--without-libuuid],
[Generate UUIDs for WARC files via libuuid]))
AC_ARG_WITH(libuuid, AS_HELP_STRING([--without-libuuid],[Generate UUIDs for WARC files via libuuid]))
AS_IF([test "x$with_libuuid" = xyes], [
# libuuid was explicitly requested
@@ -715,18 +843,37 @@ AS_IF([test "x$uuid_mode" = x1], [
])
dnl
dnl Check for PCRE
dnl Check for PCRE2 / PCRE
dnl
AC_ARG_ENABLE(pcre, AC_HELP_STRING([--disable-pcre],
[Disable PCRE style regular expressions]))
AC_ARG_ENABLE(pcre2, AS_HELP_STRING([--disable-pcre2],[Disable PCRE2 style regular expressions]))
AC_ARG_ENABLE(pcre, AS_HELP_STRING([--disable-pcre],[Disable PCRE style regular expressions]))
AS_IF([test "X$enable_pcre" != "Xno"],[
AS_IF([test "X$enable_pcre2" != "Xno"],[
enable_pcre2=no
PKG_CHECK_MODULES([PCRE2], libpcre2-8, [
CFLAGS="$PCRE2_CFLAGS $CFLAGS"
LIBS="$PCRE2_LIBS $LIBS"
AC_DEFINE([HAVE_LIBPCRE2], [1], [Define if libpcre2 is available.])
enable_pcre2=yes
], [
AC_CHECK_HEADER(pcre2.h, [
AC_CHECK_LIB(pcre2-8, pcre2_compile_8, [
LIBS="${LIBS} -lpcre2-8"
AC_DEFINE([HAVE_LIBPCRE2], 1, [Define if libpcre2 is available.])
enable_pcre2=yes
])
])
])
])
AS_IF([test "X$enable_pcre" != "Xno" && test "X$enable_pcre2" != "Xyes"],[
PKG_CHECK_MODULES([PCRE], libpcre, [
CFLAGS="$PCRE_CFLAGS $CFLAGS"
AC_CHECK_HEADER(pcre.h, [
LIBS="$PCRE_LIBS $LIBS"
AC_DEFINE([HAVE_LIBPCRE], [1], [Define if libpcre is available.])
enable_pcre=yes
])
], [
AC_CHECK_HEADER(pcre.h, [
@@ -738,6 +885,8 @@ AS_IF([test "X$enable_pcre" != "Xno"],[
])
])
AS_IF([test "X$enable_pcre2" = Xyes], [PCRE_INFO="yes, via libpcre2"], [test "X$enable_pcre" = Xyes], [PCRE_INFO="yes, via libpcre"], [PCRE_INFO=no])
dnl
dnl Check for libcares (resolver library)
dnl
@@ -763,6 +912,45 @@ AS_IF([test "X$with_cares" = "Xyes"],[
RESOLVER_INFO="libc, --bind-dns-address and --dns-servers not available"
])
dnl
dnl Check for libmetalink
dnl
AS_IF([test x"$with_metalink" != xno], [
PKG_CHECK_MODULES([METALINK], libmetalink, [
LIBS="$METALINK_LIBS $LIBS"
CFLAGS="$METALINK_CFLAGS $CFLAGS"
AC_DEFINE([HAVE_METALINK], [1], [Define if using metalink.])
with_metalink=yes
], [
with_metalink=no
])
have_gpg=no
AS_IF([test x"$with_metalink" = xyes], [
dnl
dnl Check for GPGME
dnl
PKG_CHECK_MODULES([GPGME], gpgme, [
LIBS="$LIBS $GPGME_LIBS"
CFLAGS="$GPGME_CFLAGS $CFLAGS"
AC_DEFINE([HAVE_GPGME], [1], [Define if GPGME is available.])
have_gpg=yes
], [
m4_ifdef([AM_PATH_GPGME], [
AM_PATH_GPGME([], [
# Put libgpgme to the end of the library list since it introduces a -L linker flags.
# That -L might break the build if there are two different version of
# a library (e.g. GnuTLS) in /usr/local and in the system directory.
LIBS="$LIBS $GPGME_LIBS"
CFLAGS="$GPGME_CFLAGS $CFLAGS"
AC_DEFINE([HAVE_GPGME], [1], [Define if GPGME is available.])
have_gpg=yes
])
])
])
])
])
dnl
dnl Extended Attribute support
dnl
@@ -790,17 +978,21 @@ test "X${ENABLE_XATTR}" = "Xyes" && AC_DEFINE([ENABLE_XATTR], 1,
[Define if you want file meta-data storing into POSIX Extended Attributes compiled in.])
dnl Needed by src/Makefile.am
AM_CONDITIONAL([IRI_IS_ENABLED], [test "X$iri" != "Xno"])
AM_CONDITIONAL([WITH_IRI], [test "X$iri" != "Xno"])
AM_CONDITIONAL([WITH_SSL], [test "X$with_ssl" != "Xno"])
AM_CONDITIONAL([METALINK_IS_ENABLED], [test "X$with_metalink" != "Xno"])
AM_CONDITIONAL([WITH_METALINK], [test "X$with_metalink" != "Xno"])
AM_CONDITIONAL([WITH_XATTR], [test "X$ENABLE_XATTR" != "Xno"])
AM_CONDITIONAL([WITH_NTLM], [test "X$ENABLE_NTLM" = "Xyes"])
AM_CONDITIONAL([WITH_OPIE], [test x"$ENABLE_OPIE" = x"yes"])
AM_CONDITIONAL([WITH_OPENSSL], [test x"$ssl_library" = x"openssl"])
AM_CONDITIONAL([WITH_GNUTLS], [test x"$ssl_library" = x"gnutls"])
dnl
dnl Create output
dnl
AC_CONFIG_FILES([Makefile src/Makefile doc/Makefile util/Makefile
po/Makefile.in tests/Makefile fuzz/Makefile
lib/Makefile testenv/Makefile
po/Makefile.in gnulib_po/Makefile.in tests/Makefile
fuzz/Makefile lib/Makefile testenv/Makefile
tests/certs/interca.conf tests/certs/rootca.conf])
AC_CONFIG_HEADERS([src/config.h])
AC_OUTPUT
@@ -811,12 +1003,13 @@ AC_MSG_NOTICE([Summary of build options:
Host OS: $host_os
Install prefix: $prefix
Compiler: $CC
CFlags: $CFLAGS $CPPFLAGS
CFlags: $CFLAGS $CPPFLAGS $WARN_CFLAGS
LDFlags: $LDFLAGS
Libs: $LIBS
SSL: $with_ssl
Zlib: $with_zlib
PSL: $with_libpsl
PCRE: $PCRE_INFO
Digest: $ENABLE_DIGEST
NTLM: $ENABLE_NTLM
OPIE: $ENABLE_OPIE

View File

@@ -1,28 +1,37 @@
#!/usr/bin/env sh
#!/usr/bin/env bash
# GNU Wget: Make a new release
#
# This is a helper script to make a new release of GNU Wget
#
# Make sure that
# Author: Darshit Shah <darnir@gnu.org>
set -e
set -o pipefail
set -u
export CFLAGS="-g -O2 -Wall"
export LC_ALL=C
EXTENSIONS="gzip lzip"
ask_continue() {
printf "Continue? [y/N]: "
read inp
read -r inp
[ "$inp" != "y" ] && [ "$inp" != "Y" ] && exit 1
# For some completely weird reason, this set +x is required. Else, the test
# *ALWAYS* fails
set +x
}
REQUIRED_PROGRAMS="ncftpput sponge"
for prog in $REQUIRED_PROGRAMS; do
if ! command -v "$prog" >/dev/null 2>&1; then
2>&1 echo "Required program $prog not found. Please install before proceeding"
exit 1
fi
done
if [ ! -d ".git" ]; then
echo "Please run this script from the root of the git repository"
fi
@@ -36,6 +45,7 @@ git clean -dxfq
echo "=> Bootstrap"
./bootstrap > /dev/null 2>&1
autoreconf -ivf
echo "=> Configure"
./configure -q
@@ -46,42 +56,33 @@ APP_VER=$(grep "^PACKAGE_VERSION='" configure | cut -d "'" -f2)
echo "Making release for $APP_NAME $APP_VER"
ask_continue
NEWS_VERSION=$(grep -i "Changes in $APP_NAME" NEWS | head -1 | awk '{print $5}')
NEWS_VERSION=$(grep -i "Noteworthy changes in release" NEWS | head -2 | tail -1 | awk '{print $6}')
if [ "$NEWS_VERSION" != "$APP_VER" ]; then
echo "Latest version in NEWS file does not match latest tag"
exit 1
fi
CUR_VER=$(git describe | sed 's/^v//')
PREV_VER=$(git describe --abbrev=0 --tags $(git rev-list --tags --skip=1 --max-count=1) | sed 's/^v//')
LNO_CVER=$(grep -in "Changes in $APP_NAME" NEWS | head -1 | cut -f1 -d:)
LNO_PVER=$(grep -in "Changes in $APP_NAME" NEWS | head -2 | tail -1 | cut -f1 -d:)
PREV_VER=$(cat .prev-version)
LNO_CVER=$(grep -in "Noteworthy changes in release" NEWS | head -2 | tail -1 | cut -f1 -d:)
LNO_PVER=$(grep -in "Noteworthy changes in release" NEWS | head -3 | tail -1 | cut -f1 -d:)
sed -n "${LNO_CVER},${LNO_PVER}p" NEWS
echo "This is the current contents of your NEWS"
ask_continue
echo "=> Make distcheck"
make -s distcheck > /dev/null 2>&1
for ext in $EXTENSIONS; do
[ "$ext" = 'gz' ] && ext="gzip"
[ "$ext" = 'lz' ] && ext="lzip"
echo "=> Making $APP_NAME-$CUR_VER.tar.$ext"
make -s "dist-$ext" > /dev/null
done
echo ""
REL_TYPE=
REL_LOC=
while [ -z $REL_TYPE ]; do
printf "Enter release type (alpha, beta, stable): "
read x
case $x in
a|A|alpha|Alpha) REL_TYPE=alpha && REL_LOC="alpha.gnu.org";;
b|B|beta|Beta) REL_TYPE=beta && REL_LOC="alpha.gnu.org";;
s|S|stable|Stable) REL_TYPE=stable && REL_LOC="ftp.gnu.org";;
a|A|alpha|Alpha) REL_TYPE=alpha;;
b|B|beta|Beta) REL_TYPE=beta;;
s|S|stable|Stable) REL_TYPE=stable;;
esac
done
echo "=> Make release"
make -s release RELEASE_TYPE=$REL_TYPE
GPG_KEY=
while [ -z $GPG_KEY ]; do
printf "Enter GPG Key ID for Signing: "
@@ -94,15 +95,30 @@ done
echo ""
APP_LOWER="$(echo "$APP_NAME" | tr '[:upper:]' '[:lower:]')"
./build-aux/gnupload --to "$REL_LOC:$APP_LOWER" --user "$GPG_KEY" --symlink-regex "$APP_LOWER-$CUR_VER".tar.*
upload_command=$(make -s emit_upload_commands RELEASE_TYPE=stable \
| sponge \
| head -n3 \
| tail -1 \
| sed "s/--to/--user $GPG_KEY --symlink-regex --to/g"
)
echo "=> Uploading"
echo "Upload tarballs?"
echo "$upload_command"
ask_continue
$upload_command
echo ""
build-aux/announce-gen --release-type "$REL_TYPE" --package-name "$APP_LOWER" --previous-version="$PREV_VER" --current-version "$CUR_VER" --gpg-key-id="$GPG_KEY" --url-directory "https://${REL_LOC}/gnu/${APP_LOWER}" --news=NEWS > "$APP_NAME-$APP_VER.announce"
echo "Remaining Release Checklist:
1. Send a mail to coordinator@translationproject.org with the subject
\"Requesting Translations for $APP_NAME-$APP_VER.POT\"
2. Announce email template placed at ~/announce-$APP_NAME-$APP_VER
Send an announce mail to info-gnu@gnu.org
3. Run \`make web-manual\` and \`make web-manual-update\` to update manual online
(Only Stable)
"
echo ""
echo "Announce email template placed at $APP_NAME-$APP_VER.announce"
prev_tag=$(git tag | tail -2 | head -1)
echo ""

25
contrib/mk_authors Executable file
View File

@@ -0,0 +1,25 @@
#!/usr/bin/env bash
prev_tag=`git tag --sort=taggerdate|tail -1`
echo -e "The authors list is autogenerated from the git history; sorted by number of commits\n"
(
if [ -z $prev_tag ]; then
git shortlog -sen
else
git shortlog -sen ${prev_tag}..
fi
) | cut -f 2 | sed 's/@/ at /g'
#(
#if [ -z $prev_tag ]; then
# git log --reverse|grep -i reported|cut -d':' -f2|cut -d'<' -f1|cut -d' ' -f2-|tr -d '"'
#else
# git log ${prev_tag}.. --reverse|grep -i reported|cut -d':' -f2|cut -d'<' -f1|cut -d' ' -f2-|tr -d '"'
#fi
#) | sort -u
echo -e "\n\nThe translators list is autogenerated from po file history\n"
sed -n 's/.*Last-Translator: *\(.*\) *<.*/\1/p' po/*.po | sort -u

14
contrib/spell-checker Executable file
View File

@@ -0,0 +1,14 @@
#!/usr/bin/env bash
#
# see https://github.com/codespell-project/codespell
#
# in short:
# - pip install codespell
# - add ~/.local/bin to PATH
set -e
codespell $* -L "tim,gonna,cas,referer,fpr,vas,msdos,ifset,ba,tage,tolen,files',hist,lightening,wither,parm" `git ls-files|\
egrep -v '_fuzzer.in|_fuzzer.repro|\.der$|\.pem$|gnulib|ChangeLog|tests/gpg|\.png$|\.dat$|contrib/spell-checker|tests/certs/.*\.crt'`
#codespell $* -L "tim,gonna,cas,te,referer,ist,fpr,vas" src/wget.c contrib/spell-checker

View File

@@ -1,5 +1,5 @@
# Makefile for `wget' utility
# Copyright (C) 1995-1997, 2007-2011, 2015, 2018 Free Software
# Copyright (C) 1995-1997, 2007-2011, 2015, 2018-2023 Free Software
# Foundation, Inc.
# This program is free software; you can redistribute it and/or modify

View File

@@ -5,8 +5,8 @@
@c hence no sectioning command or @node.
@display
Copyright @copyright{} 2000-2002, 2007-2008, 2015, 2018 Free Software
Foundation, Inc.
Copyright @copyright{} 2000--2002, 2007--2008, 2015, 2018--2023 Free
Software Foundation, Inc.
@uref{http://fsf.org/}
Everyone is permitted to copy and distribute verbatim copies

View File

@@ -106,7 +106,7 @@ found unsafe too, the file is not downloaded.
See [1 #section-1.1].
cat > bugus.meta4 << EOF
cat > bogus.meta4 << EOF
<?xml version="1.0" encoding="UTF-8"?>
<metalink xmlns="urn:ietf:params:xml:ns:metalink">
<file name="/dir/A/File1">

View File

@@ -1,6 +1,6 @@
#! /usr/bin/env perl
# Copyright (C) 1999-2001, 2003, 2007, 2009-2011, 2015, 2018 Free
# Copyright (C) 1999-2001, 2003, 2007, 2009-2011, 2015, 2018-2023 Free
# Software Foundation, Inc.
# This file is part of GCC.

View File

@@ -31,8 +31,8 @@ This file documents the GNU Wget utility for downloading network
data.
@c man begin COPYRIGHT
Copyright @copyright{} 1996-2011, 2015, 2018 Free Software Foundation,
Inc.
Copyright @copyright{} 1996--2011, 2015, 2018--2023 Free Software
Foundation, Inc.
@iftex
Permission is granted to make and distribute verbatim copies of
@@ -64,6 +64,8 @@ Texts. A copy of the license is included in the section entitled
@ignore
@c man begin AUTHOR
Originally written by Hrvoje Nikšić <hniksic@xemacs.org>.
Currently maintained by Darshit Shah <darnir@gnu.org> and
Tim Rühsen <tim.ruehsen@gmx.de>.
@c man end
@c man begin SEEALSO
This is @strong{not} the complete manual for GNU Wget.
@@ -71,6 +73,9 @@ For more complete information, including more detailed explanations of
some of the options, and a number of commands available
for use with @file{.wgetrc} files and the @samp{-e} option, see the GNU
Info entry for @file{wget}.
Also see wget2(1), the updated version of GNU Wget with even better
support for recursive downloading and modern protocols like HTTP/2.
@c man end
@end ignore
@@ -379,7 +384,7 @@ and @file{/~somebody}. You can also clear the lists in @file{.wgetrc}
(@pxref{Wgetrc Syntax}).
@example
wget -X '' -X /~nobody,/~somebody
wget -X "" -X /~nobody,/~somebody
@end example
Most options that do not accept arguments are @dfn{boolean} options,
@@ -540,6 +545,14 @@ right NUMBER.
Set preferred location for Metalink resources. This has effect if multiple
resources with same priority are available.
@cindex xattr
@item --xattr
Enable use of file system's extended attributes to save the
original URL and the Referer HTTP header value if used.
Be aware that the URL might contain private information like
access tokens or credentials.
@cindex force html
@item -F
@@ -855,7 +868,7 @@ files. In such a case, wget could simply be invoked with this parameter to get
a much cleaner output on the screen.
This option will also force the progress bar to be printed to @file{stderr} when
used alongside the @samp{--logfile} option.
used alongside the @samp{--output-file} option.
@item -N
@itemx --timestamping
@@ -1030,10 +1043,11 @@ megabytes (with @samp{m} suffix).
Note that quota will never affect downloading a single file. So if you
specify @samp{wget -Q10k https://example.com/ls-lR.gz}, all of the
@file{ls-lR.gz} will be downloaded. The same goes even when several
@sc{url}s are specified on the command-line. However, quota is
respected when retrieving either recursively, or from an input file.
Thus you may safely type @samp{wget -Q2m -i sites}---download will be
aborted when the quota is exceeded.
@sc{url}s are specified on the command-line. The quota is checked only
at the end of each downloaded file, so it will never result in a partially
downloaded file. Thus you may safely type @samp{wget -Q2m -i sites}---download
will be aborted after the file that exhausts the quota is completely
downloaded.
Setting quota to 0 or to @samp{inf} unlimits the download quota.
@@ -1394,10 +1408,10 @@ to the inability of server-side scripts to cope with the connections.
@cindex cache
@item --no-cache
Disable server-side cache. In this case, Wget will send the remote
server an appropriate directive (@samp{Pragma: no-cache}) to get the
file from the remote service, rather than returning the cached version.
This is especially useful for retrieving and flushing out-of-date
documents on proxy servers.
server appropriate directives (@samp{Cache-Control: no-cache} and
@samp{Pragma: no-cache}) to get the file from the remote service,
rather than returning the cached version. This is especially useful
for retrieving and flushing out-of-date documents on proxy servers.
Caching is allowed by default.
@@ -1750,6 +1764,10 @@ some few obscure servers, which never send HTTP authentication
challenges, but accept unsolicited auth info, say, in addition to
form-based authentication.
@item --retry-on-host-error
Consider host errors, such as ``Temporary failure in name resolution'',
as non-fatal, transient errors.
@item --retry-on-http-error=@var{code[,code,...]}
Consider given HTTP response codes as non-fatal, transient errors.
Supply a comma-separated list of 3-digit HTTP response codes as
@@ -1780,9 +1798,9 @@ If Wget is compiled without SSL support, none of these options are available.
@cindex SSL protocol, choose
@item --secure-protocol=@var{protocol}
Choose the secure protocol to be used. Legal values are @samp{auto},
@samp{SSLv2}, @samp{SSLv3}, @samp{TLSv1}, @samp{TLSv1_1}, @samp{TLSv1_2}
and @samp{PFS}. If @samp{auto} is used, the SSL library is given the
liberty of choosing the appropriate protocol automatically, which is
@samp{SSLv2}, @samp{SSLv3}, @samp{TLSv1}, @samp{TLSv1_1}, @samp{TLSv1_2},
@samp{TLSv1_3} and @samp{PFS}. If @samp{auto} is used, the SSL library is
given the liberty of choosing the appropriate protocol automatically, which is
achieved by sending a TLSv1 greeting. This is the default.
Specifying @samp{SSLv2}, @samp{SSLv3}, @samp{TLSv1}, @samp{TLSv1_1},
@@ -1805,7 +1823,7 @@ Set the cipher list string. Typically this string sets the
cipher suites and other SSL/TLS options that the user wish should be used, in a
set order of preference (GnuTLS calls it 'priority string'). This string
will be fed verbatim to the SSL/TLS engine (OpenSSL or GnuTLS) and hence
its format and syntax is dependant on that. Wget will not process or manipulate it
its format and syntax is dependent on that. Wget will not process or manipulate it
in any way. Refer to the OpenSSL or GnuTLS documentation for more information.
@cindex SSL certificate, check
@@ -1979,7 +1997,7 @@ from the database.
If you supply your own HSTS database via @samp{--hsts-file}, be aware that Wget
may modify the provided file if any change occurs between the HSTS policies
requested by the remote servers and those in the file. When Wget exists,
requested by the remote servers and those in the file. When Wget exits,
it effectively updates the HSTS database by rewriting the database file with the new entries.
If the supplied file does not exist, Wget will create one. This file will contain the new HSTS
@@ -1993,7 +2011,7 @@ the same time over the HSTS database. Before dumping the updated HSTS entries
on the file, Wget will re-read it and merge the changes.
Using a custom HSTS database and/or modifying an existing one is discouraged.
For more information about the potential security threats arised from such practice,
For more information about the potential security threats arose from such practice,
see section 14 "Security Considerations" of RFC 6797, specially section 14.9
"Creative Manipulation of HSTS Policy Store".
@end table
@@ -2126,7 +2144,7 @@ download them recursively, though this feature may be added in the future.
When @samp{--retr-symlinks=no} is specified, the linked-to file is not
downloaded. Instead, a matching symbolic link is created on the local
filesystem. The pointed-to file will not be retrieved unless this recursive
file system. The pointed-to file will not be retrieved unless this recursive
retrieval would have encountered it separately and downloaded it anyway. This
option poses a security risk where a malicious FTP Server may cause Wget to
write to files outside of the intended directories through a specially crafted
@@ -2186,8 +2204,21 @@ details. The default maximum depth is 5.
@item -l @var{depth}
@itemx --level=@var{depth}
Specify recursion maximum depth level @var{depth} (@pxref{Recursive
Download}).
Set the maximum number of subdirectories that Wget will recurse into to @var{depth}.
In order to prevent one from accidentally downloading very large websites when using recursion
this is limited to a depth of 5 by default, i.e., it will traverse at most 5 directories deep
starting from the provided URL.
Set @samp{-l 0} or @samp{-l inf} for infinite recursion depth.
@example
wget -r -l 0 http://@var{site}/1.html
@end example
Ideally, one would expect this to download just @file{1.html}.
but unfortunately this is not the case, because @samp{-l 0} is equivalent to
@samp{-l inf}---that is, infinite recursion. To download a single @sc{html}
page (or a handful of them), specify them all on the command line and leave away @samp{-r}
and @samp{-l}. To download the essential items to view a single @sc{html} page, see @samp{page requisites}.
@cindex proxy filling
@cindex delete after retrieval
@@ -3015,7 +3046,7 @@ file so that it keeps its date of modification.
wget -S http://www.gnu.ai.mit.edu/
@end example
A simple @code{ls -l} shows that the time stamp on the local file equals
A simple @code{ls -l} shows that the timestamp on the local file equals
the state of the @code{Last-Modified} header, as returned by the server.
As you can see, the time-stamping info is preserved locally, even
without @samp{-N} (at least for @sc{http}).
@@ -3951,7 +3982,7 @@ or @samp{application/xhtml+xml} to @file{@var{name}.html}.
@example
wget --mirror --convert-links --backup-converted \
--html-extension -o /home/me/weeklog \
--adjust-extension -o /home/me/weeklog \
https://www.gnu.org/
@end example
@@ -4186,7 +4217,7 @@ from the debug log before sending it to the bug address. The
@code{-d} won't go out of its way to collect sensitive information,
but the log @emph{will} contain a fairly complete transcript of Wget's
communication with the server, which may include passwords and pieces
of downloaded data. Since the bug address is publically archived, you
of downloaded data. Since the bug address is publicly archived, you
may assume that all bug reports are visible to the public.
@item
@@ -4435,7 +4466,7 @@ Tomislav Petrović, Mario Mikočević---many bug reports and
suggestions.
@item
François Pinard---many thorough bug reports and discussions.
Françis Pinard---many thorough bug reports and discussions.
@item
Karl Eichwalder---lots of help with internationalization, Makefile
@@ -4473,6 +4504,14 @@ Ted Mielczarek---donated support for CSS.
@item
Saint Xavier---Support for IRIs (RFC 3987).
@item
Tim Rühsen---Loads of helpful patches, especially fuzzing support and
Continuous Integration. Maintainer since 2014.
@item
Darshit Shah---Many helpful patches. Community support on various platforms.
Maintainer since 2014.
@item
People who provided donations for development---including Brian Gough.
@end itemize

View File

@@ -1,23 +1,32 @@
AM_CFLAGS = $(WERROR_CFLAGS) $(WARN_CFLAGS) -Wno-unused-parameter -Wno-pedantic
AM_CFLAGS = $(WERROR_CFLAGS) $(WARN_CFLAGS) $(CODE_COVERAGE_CFLAGS)
AM_CPPFLAGS = -I$(top_srcdir)/src -I$(srcdir) -I$(top_builddir)/lib -I$(top_srcdir)/lib \
$(CODE_COVERAGE_CPPFLAGS) \
-DSRCDIR=\"$(abs_srcdir)\"
LDADD = ../lib/libgnu.a \
$(GETADDRINFO_LIB) $(HOSTENT_LIB) $(INET_NTOP_LIB) $(INET_PTON_LIB) \
$(LIBSOCKET) $(LIB_CLOCK_GETTIME) $(LIB_CRYPTO) $(LIB_GETLOGIN) $(LIB_NANOSLEEP) $(LIB_POLL) \
$(LIB_POSIX_SPAWN) $(LIB_PTHREAD_SIGMASK) $(LIB_SELECT) $(LTLIBICONV) $(LTLIBINTL) \
$(LTLIBMULTITHREAD) $(LTLIBTHREAD) $(SERVENT_LIB) @INTL_MACOSX_LIBS@ \
-ldl
$(LIB_POSIX_SPAWN) $(LIB_PTHREAD_SIGMASK) $(LIB_SELECT) $(LIBICONV) $(LIBINTL) \
$(LIBMULTITHREAD) $(LIBTHREAD) $(SERVENT_LIB) @INTL_MACOSX_LIBS@ \
$(FUZZ_LIBS) $(CODE_COVERAGE_LIBS)
WGET_TESTS = \
wget_css_fuzzer$(EXEEXT) \
wget_cookie_fuzzer$(EXEEXT) \
wget_css_fuzzer$(EXEEXT) \
wget_ftpls_fuzzer$(EXEEXT) \
wget_html_fuzzer$(EXEEXT) \
wget_netrc_fuzzer$(EXEEXT) \
wget_options_fuzzer$(EXEEXT) \
wget_progress_fuzzer$(EXEEXT) \
wget_read_hunk_fuzzer$(EXEEXT) \
wget_robots_fuzzer$(EXEEXT) \
wget_url_fuzzer$(EXEEXT)
EXTRA_DIST = test-runner.sh
if WITH_NTLM
WGET_TESTS += wget_ntlm_fuzzer$(EXEEXT)
endif
if FUZZING
bin_PROGRAMS = $(WGET_TESTS)
LDADD += $(LIB_FUZZING_ENGINE)
@@ -26,6 +35,7 @@ if FUZZING
else
AM_CPPFLAGS += -DTEST_RUN
AM_TESTS_ENVIRONMENT = export VALGRIND_TESTS"=@VALGRIND_TESTS@";
LOG_COMPILER = $(top_srcdir)/fuzz/test-runner.sh
TESTS = $(WGET_TESTS)
check_PROGRAMS = $(WGET_TESTS)
MAIN = main.c fuzzer.h
@@ -36,12 +46,12 @@ endif
../src/libunittest.a:
$(MAKE) $(AM_MAKEFLAGS) -C ../src libunittest.a
wget_css_fuzzer_SOURCES = wget_css_fuzzer.c $(MAIN)
wget_css_fuzzer_LDADD = ../src/libunittest.a $(LDADD)
wget_cookie_fuzzer_SOURCES = wget_cookie_fuzzer.c $(MAIN)
wget_cookie_fuzzer_LDADD = ../src/libunittest.a $(LDADD)
wget_css_fuzzer_SOURCES = wget_css_fuzzer.c $(MAIN)
wget_css_fuzzer_LDADD = ../src/libunittest.a $(LDADD)
wget_ftpls_fuzzer_SOURCES = wget_ftpls_fuzzer.c $(MAIN)
wget_ftpls_fuzzer_LDADD = ../src/libunittest.a $(LDADD)
@@ -51,9 +61,18 @@ wget_html_fuzzer_LDADD = ../src/libunittest.a $(LDADD)
wget_netrc_fuzzer_SOURCES = wget_netrc_fuzzer.c $(MAIN)
wget_netrc_fuzzer_LDADD = ../src/libunittest.a $(LDADD)
wget_ntlm_fuzzer_SOURCES = wget_ntlm_fuzzer.c $(MAIN)
wget_ntlm_fuzzer_LDADD = ../src/libunittest.a $(LDADD)
wget_options_fuzzer_SOURCES = wget_options_fuzzer.c $(MAIN)
wget_options_fuzzer_LDADD = ../src/libunittest.a $(LDADD)
wget_progress_fuzzer_SOURCES = wget_progress_fuzzer.c $(MAIN)
wget_progress_fuzzer_LDADD = ../src/libunittest.a $(LDADD)
wget_read_hunk_fuzzer_SOURCES = wget_read_hunk_fuzzer.c $(MAIN)
wget_read_hunk_fuzzer_LDADD = ../src/libunittest.a $(LDADD)
wget_robots_fuzzer_SOURCES = wget_robots_fuzzer.c $(MAIN)
wget_robots_fuzzer_LDADD = ../src/libunittest.a $(LDADD)
@@ -74,10 +93,10 @@ clean-local:
oss-fuzz:
if test "$$OUT" != ""; then \
XLIBS="-lpsl -lgnutls -lnettle -lhogweed -lidn2 -lunistring"; \
XLIBS="-lpsl -lgnutls -lhogweed -lnettle -lidn2 -lunistring -lz"; \
for ccfile in wget*_fuzzer.c; do \
fuzzer=$$(basename $$ccfile .c); \
$$CXX $$CXXFLAGS -I$(top_srcdir)/src -I$(top_srcdir) -I$(top_srcdir)/lib \
$$CC $$CFLAGS -I$(top_srcdir)/src -I$(top_srcdir) -I$(top_srcdir)/lib \
"$${fuzzer}.c" -o "$${fuzzer}" \
../src/libunittest.a ../lib/libgnu.a $${LIB_FUZZING_ENGINE} \
-Wl,-Bstatic $${XLIBS} -Wl,-Bdynamic; \

View File

@@ -14,21 +14,20 @@ regression testing with top dir 'make check' or 'make check-valgrind'.
Use the following commands on top dir:
```
export CC=clang-6.0
export CXX=clang++-6.0
export CC=clang
# address sanitizer:
export CFLAGS="-O1 -g -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=undefined,integer,nullability -fsanitize=address -fsanitize-address-use-after-scope -fsanitize-coverage=trace-pc-guard,trace-cmp"
#export CFLAGS="-O1 -g -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=undefined,integer,nullability -fsanitize=address -fsanitize-address-use-after-scope -fsanitize-coverage=trace-pc-guard,trace-cmp"
export CFLAGS="-O1 -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=undefined -fsanitize=address -fsanitize-address-use-after-scope -fsanitize=fuzzer-no-link"
# undefined sanitizer;
export CFLAGS="-O1 -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=bool,array-bounds,float-divide-by-zero,function,integer-divide-by-zero,return,shift,signed-integer-overflow,vla-bound,vptr -fno-sanitize-recover=bool,array-bounds,float-divide-by-zero,function,integer-divide-by-zero,return,shift,signed-integer-overflow,vla-bound,vptr -fsanitize=fuzzer-no-link"
export CXXFLAGS="$CFLAGS -stdlib=libc++"
export LIB_FUZZING_ENGINE="-lFuzzer -lstdc++"
./configure --enable-fuzzing --without-metalink --without-zlib --disable-pcre --without-libuuid
export LIB_FUZZING_ENGINE="-lFuzzer -lstdc++"
./configure --enable-fuzzing --without-metalink --without-zlib --disable-pcre --without-libuuid --enable-assert
make clean
make -j$(nproc)
cd fuzz
# run wget_options_fuzzer
UBSAN_OPTIONS=print_stacktrace=1 ASAN_SYMBOLIZER_PATH=/usr/lib/llvm-6.0/bin/llvm-symbolizer \
UBSAN_OPTIONS=print_stacktrace=1 ASAN_SYMBOLIZER_PATH=/usr/bin/llvm-symbolizer \
./run-clang.sh wget_options_fuzzer
```

View File

@@ -1,5 +1,5 @@
/*
* Copyright(c) 2017-2018 Free Software Foundation, Inc.
* Copyright (c) 2017-2023 Free Software Foundation, Inc.
*
* This file is part of GNU Wget.
*
@@ -20,6 +20,22 @@
#include <stddef.h> // size_t
#include <stdint.h> // uint8_t
#if defined __clang__ || __GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 5)
#pragma GCC diagnostic ignored "-Wunused"
#pragma GCC diagnostic ignored "-Wunused-parameter"
#pragma GCC diagnostic ignored "-Wpedantic"
#endif
#define CLOSE_STDERR \
int bak = dup(STDERR_FILENO); \
int fd = open("/dev/null", O_WRONLY); \
dup2(fd, STDERR_FILENO); \
close(fd);
#define RESTORE_STDERR \
dup2(bak, STDERR_FILENO); \
close(bak);
#ifdef __cplusplus
extern "C"
#endif

View File

@@ -1,6 +1,6 @@
#!/bin/sh -eu
if ! grep -q FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION Makefile; then
if ! grep FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION Makefile >/dev/null; then
echo "The fuzzers haven't been built for fuzzing (maybe for regression testing !?)"
echo "Please built regarding README.md and try again."
exit 1

View File

@@ -13,7 +13,7 @@ if test -z "$1"; then
exit 1
fi
if ! grep -q FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION Makefile; then
if ! grep FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION Makefile >/dev/null; then
echo "The fuzzers haven't been built for fuzzing (maybe for regression testing !?)"
echo "Please built regarding README.md and try again."
exit 1
@@ -23,9 +23,10 @@ fuzzer=$1
project=wget
# sync/copy the OSS-Fuzz corpora into the .new directory
mkdir -p ${fuzzer}.new
cp -fp ${fuzzer}.in/* ${fuzzer}.new
gsutil -m rsync gs://${project}-corpus.clusterfuzz-external.appspot.com/libFuzzer/wget_${fuzzer} ${fuzzer}.new
mkdir -p ${fuzzer}.in ${fuzzer}.new
cp -fp ${fuzzer}.in/* ${fuzzer}.new 2>/dev/null || true
gsutil cp $(gsutil ls gs://${project}-backup.clusterfuzz-external.appspot.com/corpus/libFuzzer/${fuzzer}|tail -n 1) ${fuzzer}.new/
(cd ${fuzzer}.new && unzip -q -o *.zip && rm *.zip)
# create fuzzer target
BUILD_ONLY=1 ./run-clang.sh ${fuzzer}

View File

@@ -1,4 +1,22 @@
/*
* Copyright (c) 2017-2023 Free Software Foundation, Inc.
*
* This file is part of GNU Wget.
*
* GNU Wget is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 3 of the License, or
* (at your option) any later version.
*
* GNU Wget is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Wget. If not, see <https://www.gnu.org/licenses/>.
* Created 19.10.2017 by Tim Rühsen
*
* Call glob() using data from fuzzer crash file

View File

@@ -1,5 +1,5 @@
/*
* Copyright(c) 2017-2018 Free Software Foundation, Inc.
* Copyright (c) 2017-2023 Free Software Foundation, Inc.
*
* This file is part of GNU Wget.
*
@@ -51,11 +51,12 @@ static int test_all_from(const char *dirname)
char fname[strlen(dirname) + strlen(dp->d_name) + 2];
snprintf(fname, sizeof(fname), "%s/%s", dirname, dp->d_name);
struct file_memory *fmem;
struct file_memory *fmem;
if ((fmem = wget_read_file(fname))) {
printf("testing %ld bytes from '%s'\n", fmem->length, fname);
fflush(stdout);
LLVMFuzzerTestOneInput((uint8_t *)fmem->content, fmem->length);
wget_read_file_free(fmem);
wget_read_file_free(fmem);
}
}
closedir(dirp);
@@ -67,30 +68,15 @@ static int test_all_from(const char *dirname)
int main(int argc, char **argv)
{
// if VALGRIND testing is enabled, we have to call ourselves with valgrind checking
const char *valgrind = getenv("VALGRIND_TESTS");
const char *target;
size_t target_len;
if (!valgrind || !*valgrind || !strcmp(valgrind, "0")) {
// fallthrough
}
else if (!strcmp(valgrind, "1")) {
char cmd[strlen(argv[0]) + 256];
snprintf(cmd, sizeof(cmd), "VALGRIND_TESTS=\"\" valgrind --error-exitcode=301 --leak-check=yes --show-reachable=yes --track-origins=yes %s", argv[0]);
return system(cmd) != 0;
} else {
char cmd[strlen(valgrind) + strlen(argv[0]) + 32];
snprintf(cmd, sizeof(cmd), "VALGRIND_TESTS="" %s %s", valgrind, argv[0]);
return system(cmd) != 0;
}
if ((target = strrchr(argv[0], SLASH)))
target = strrchr(target, '/');
else
if ((target = strrchr(argv[0], SLASH))) {
if (strrchr(target, '/'))
target = strrchr(target, '/');
} else
target = strrchr(argv[0], '/');
target = target ? target + 1 : argv[0];
if (strncmp(target, "lt-", 3) == 0)

View File

@@ -1,6 +1,6 @@
#!/bin/sh -eu
# Copyright(c) 2017 Free Software Foundation, Inc.
# Copyright (c) 2017, 2019-2023 Free Software Foundation, Inc.
#
# This file is part of GNU Wget.
#
@@ -17,7 +17,7 @@
# You should have received a copy of the GNU Lesser General Public License
# along with GNU Wget. If not, see <https://www.gnu.org/licenses/>.
if ! grep -q '^CC=.afl-clang-fast' ../config.log; then
if ! grep '^CC=.afl-clang-fast' ../config.log >/dev/null; then
echo "compile first library as:"
echo "CC=afl-clang-fast ./configure"
exit 1

View File

@@ -1,6 +1,6 @@
#!/bin/bash -e
#
# Copyright(c) 2017 Free Software Foundation, Inc.
# Copyright (c) 2017, 2019-2023 Free Software Foundation, Inc.
#
# This file is part of GNU Wget.
#
@@ -30,7 +30,7 @@ if test -z "$1"; then
exit 1
fi
if ! grep -q FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION Makefile; then
if ! grep FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION Makefile >/dev/null; then
echo "The fuzzers haven't been built for fuzzing (maybe for regression testing !?)"
echo "Please built regarding README.md and try again."
exit 1

9
fuzz/test-runner.sh Executable file
View File

@@ -0,0 +1,9 @@
#!/usr/bin/env sh
WRAPPER=""
if [ -n "$VALGRIND_TESTS" ]; then
WRAPPER="valgrind --error-exitcode=301 --leak-check=yes --show-reachable=yes --track-origins=yes"
fi
exec $WRAPPER "$@"

View File

@@ -1,5 +1,5 @@
/*
* Copyright(c) 2017-2018 Free Software Foundation, Inc.
* Copyright (c) 2017-2023 Free Software Foundation, Inc.
*
* This file is part of GNU Wget.
*
@@ -25,6 +25,8 @@
#include <stdio.h> // fmemopen
#include <string.h> // strncmp
#include <stdlib.h> // free
#include <fcntl.h> // open flags
#include <unistd.h> // close
#include "wget.h"
#undef fopen_wgetrc
@@ -60,15 +62,10 @@ FILE *fopen_wgetrc(const char *pathname, const char *mode)
void exit_wget(int status)
{
}
#else
void exit(int status)
{
}
#endif
int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
FILE *bak;
struct cookie_jar *cookie_jar;
char *set_cookie;
@@ -79,8 +76,7 @@ int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
memcpy(set_cookie, data, size);
set_cookie[size] = 0;
bak = stderr;
stderr = fopen("/dev/null", "w");
CLOSE_STDERR
cookie_jar = cookie_jar_new();
cookie_handle_set_cookie(cookie_jar, "x", 81, "p", set_cookie);
@@ -88,10 +84,9 @@ int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
cookie_handle_set_cookie(cookie_jar, "x", 80, "p/d/", set_cookie);
cookie_jar_delete(cookie_jar);
fclose(stderr);
stderr = bak;
RESTORE_STDERR
free(set_cookie);
free(set_cookie);
return 0;
}

View File

@@ -0,0 +1 @@
??????

View File

@@ -1 +0,0 @@
__Host-ID=x; expires=MoHos__Host-ID=x; expires=Mon, 29 Feb 2016 07:48:97 GMT; mh=/; domain=x.y; httponly; sec__Host-ID=x; expires=Mon, 29 Feb 2016 07:48:97 GMT; mh=/; domain=x.y; htut-ID=x; expires=Mon, 29 Feb 2016 07:48:97 GMT; mh=/; domain=x.y; hture

View File

@@ -0,0 +1 @@
AAAAAAAAAAA

View File

@@ -0,0 +1 @@
]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]

View File

@@ -0,0 +1 @@
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

View File

@@ -0,0 +1 @@
¦=;e;x;Õ;e;e;r;s;„;A;r;y

View File

@@ -1 +0,0 @@
__Hos_o=;;yõ=m;tmID=m;=;y=m;y=m;<;yõ=m;tmID=m;

View File

@@ -0,0 +1 @@
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

View File

@@ -1 +0,0 @@
__expires=x; expires=Mon, 14 4032 07:48Mon, 14 4032 07:48:5$ GMT; stmID=x; expires=Mon, 14 4000 07:48:54 GMT; matp=x; expires=Mon, 43 4032 07:48:5$ G032 07:48:27 GMT; stm 14 4000 07ÅË8:54 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%GMT; m:48:54 GMT; stmI»D=x; e|pires=Mon, 14 4000 07ID=x; expires=Mon, 07 4032 07:48:5$ GMT; stmID=x; expin, 14 4032 07:48:5$ GMT; stmID=x; expires=Mon, 14 4000 07:48:54 GMT; matp=x; expires=Mon, 14 4032 07:48:5$ G!32 07:48:54 GMT; stm 14 4000 07ÅË8:54 GMT; m:48:54 GMT; stmI»n4 1, 4032 07:48:5$ GMT; stmID=x; expcure

View File

@@ -0,0 +1 @@
]=;max-age=3;max-age=4;max-age=8;max-age=3;max-age=4;max-age=8

View File

@@ -1,2 +0,0 @@
__H@=x qqqqqqqqqqqqqqqqq*qqqqqqFpath=; secure
x-age; secure

View File

@@ -1 +0,0 @@
_dict=libwget_cookie_fuzzer.dmutate_depth _e=;Ä;Ø;;;;Ø;;;;Ä;=;;Ø;;;;Ä;=;__Host-I@=x; expires=Mon, 29 Feb@=x; expir st-016 07:48es=Mon, 28 Feb 2016 07::54 GMT;"max-48:

View File

@@ -0,0 +1 @@
xxxxxx

View File

@@ -0,0 +1 @@
```````````````````````````````````````````

View File

@@ -0,0 +1 @@
***************************************************************************************************************************************************************************

View File

@@ -1 +0,0 @@
_use¡<>šÂÊà pa<70>h=_Host-ID=xGMT; max-age=5; pa<70>h=ÑÄß<C39F>ž•n=x.y; httpomax-agecure

View File

@@ -1 +0,0 @@
__Host-ID=x; expires=29 Feb 2016 07:48:54 GMT;t-ID=x; expires=29 Feb 2016 10:48:54 GMT; max-age=5; p; expires=29 Feb 2016 07:48:54 GMT;t-ID=x; expires=58 Feb 2016 10:48:54 GMT; max-age=5; pathomt-ID=x; expires=29 Feb 2016 07:48:54 GMT;t-IDBx; exp=x.y; athomt-ID=x; expires=29 Feb 2016 07:48:54 GMT;t-IDBx; expires=29 Feb 2016 07:48:54 GMT; max-age=5; pathomai;ost-ID=x; expires=29 Feb 2016 07:48:54 GMT;t-ID=x; expires=2; Feb 2016 10:48:5: GMT; max-age=5; pathomt-ID=x; expires=29 Feb 2016 07:48:54 GMT;t-ID=x; expires=29 Feb 2096 07:48:54 GMT; max-age=5; pathomai;=x.y; httponly

View File

@@ -0,0 +1 @@
OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO

View File

@@ -0,0 +1 @@
*=;*=;*!=;*#=;*_=;*_=;*!=;*#=;*_=;*ヒ=;*_=;*_=;*ヒ=;*ェ=;*_=;*!=;*#=;*#=;*_=;*=;*ヒ=;*_=;*!=;*#=;*_=;*@=;*W=;*!=;*#=;*_=;*ヒ=;*_=;*_=;*p=;*_=;*レ=;*@=;*_=;*!=;*#=;*_=;*ヒ=;*_=;*_=;*p=;*_=;*レ=;*_=;*!=;*#=;*_=;*ヒ=;*_=;*_=;*p=;*_=;*_=;*ヒ=;*_=;*p=;*_=;*_=;*ヒ=;*_=;*ヒ=;*_=;*!=;*_=;*ヒ=;*_=;*!=;*#=;*_=;*レ=;*_=;*!=;*#=;*_=;*ヒ=;*_=;*_=;*p=;*_=;*_=;*#=;*ス=

View File

@@ -0,0 +1 @@
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

View File

@@ -1 +0,0 @@
xexpires= domain==/; domain==/; domain=th=/; domain=th=/; domain=th=/; domain==/; domain=th=/; domain==/; domain=th=/; domain==/; domain=ht=/; domain==/; domain=th=/; domain==/; domain=th=/expires=; domain==/; domain=th=/;omain==/;*=/; >domain=h=/; domain==/; doamin==/; doiain=th=/; domain=th=/:; domain=th=/; domain==/; domain=th=/; dom'ain==/; domain=th=/; domain==/; domain=th=/; ƒƒƒƒƒƒƒdomain==/; domain=th=/; domai=th=/; domain ==/; domain=th=/expires=; domain==/; domain=th=/;omain==/; domain=th=/; domain==/; domain=th=/expires=; domainy=

View File

@@ -0,0 +1 @@
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB

View File

@@ -1 +0,0 @@
_httpominimize_crash16 07:48??¿??????????????????????????????????,,,,,,,,,,6:48??¿????????????????????????expires=??????????,,,,,,,,,,,,,,ŠŠŠŠŠŠŠŠŠŠŠ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,???????54 GMT; max-age=5; paagominimize_crash16 07:48??¿????????????;?????????????????????,,,,,,,,,,7:48??¿????????????????????????expires=??????????,,,,,,,,,,,,,,ŠŠŠŠŠŠŠŠŠŠŠ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,???????5path=2 GMT; max-age=5; paagŠŠŠŠŠŠŠŠŠŠ,,,,,,,,,,,,,,,,,,,,,,,,,l,,,???????54 GMT; ŠŠŠ,,,,,,,,,,,,,,,,,,,,,,,,,l,,,???????54 GMT;ge=5=x.ÿÿÿ ly; sñcure

View File

@@ -1,2 +0,0 @@
_ÿÿÿÿÿÿ
ä=x; expi!res=-86148:54 =x; expires=-43855 Feb 6119 07:48:0Tx; expires=-437708 Feb 6120 07:48:54 8:54 =x; expires=-86148:54 =x; expires=-43855 Feb 6119 07:48:5Tx; expires=-437708 Feb 6120 07:48:54 8:54 =x; expires=-43855 Feb 6119 07:48:5TxC; expires=-437708 Feb 120 07:48:54 =b 6119 07:48:5Tx; expires=-218854 Feb 6120 07:48:54 =_axost-ID=x; 6120 07:48:54 =st-ID=x; expires=-87708 Feb 6119 07:48:54 =x; expires=3-4855 Feb 6119 07:48:5Tx; emax-age43855 Feb 6119 07:48:5Tx; expires=-437708 Feb 6120 07:48:54 =b 6119 07:48:5Tx; expires=-437708 Feb 6120 07:48:54 48:5Tx; expires=-437708 Feb 6120 07:48:54 =_axost-ID=x; ex88888888888888888888888888888888888888888pires=-c87708 Feb 6119 07:48:54 =x; expires=-43855 Feb 6119 07:48:5Tx; expires=-43855 Feb 611907:48:5T

View File

@@ -0,0 +1 @@
(=;expires==;expires=i;expires=M;expires=*;expires=*;expires==;expires=M;expires=M;expires=3;expires==;expires=M;expires=*;expires=*;expires==;expires=M;expires=M;expires=MonFeB1 7:4:83¹;expires==;expires=M;expires=*;expires=*;expires=x;expires=M;expires=ª;expires=*;expires==;expires=M;expires=M;expires=3;expires==;expires=M;expires=*;expires=*;expires==;expires=*;expires=M;expires=M;expires=4;expires=<;expires=M;expires=M;expires=E;expires=e

View File

@@ -0,0 +1 @@
PPP

View File

@@ -0,0 +1 @@
4444444444444444444444444444444444444444444

View File

@@ -0,0 +1 @@
22222222222222222222222222222222222222222222222222222222222222222222222222222222222222

View File

@@ -1 +0,0 @@
_ÿÿÿÿÿÿ ¤seedANShttponly.;domain=.;_;D=_;domain=.;domain=.;_;;domain=.;domain=.;;;doma=o;id;n.main=.;;d(o'main=.;domainn=.;domain=.;_;Domain=.;(;domain=.=_;domain=.;domain=.;_;;domain=.;domaiQQQQQQQQQQQQQQQQQQQQQQQQQQQQomain=.;;domain=.;domain=.;domain=.;(;domain=.;;;domain=.;domain=.;;;domain=.;d_D=_;domain=.;domain=.;_;n=.;domain=.;;;domain=.;;domain=.;;domain=.;domainain=.;domain=.;_;;domain=.;domain=.;;;domain=.;;domain=.;;domain=.;domain=.;domain=.;(;domain=.;;;domain=.;domain=.;om

View File

@@ -1,3 +0,0 @@
__Host-ID=x; 4032 age=2;HHHHHHdomain=HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHh=l; domain=x.y; httpohttponly; sñcurnly;ýly; seceur
e
securd

View File

@@ -0,0 +1 @@
:=;domain=';path=/p/

View File

@@ -0,0 +1 @@
RRRRRRRRRRR

View File

@@ -1 +0,0 @@
_o=;=;=;y=;;=;=;;<3B>y=_e=';˙˙ qa<71>h=_Host-ID=xGMT; <20>h=_Host-ID=xGMT; domain=˙˙˙˙˙˙˙httponly; securSSSSSSSSSSSSSSSSnly; securSSSSSÓSSSSSSSSSSSSSSSSSSSSSSSSSSSS;=;=;=SSSSSSSSSS;;=;=;=;y=;=;=;=;;y=;=;=;=;=;=;=;==;=SSSS;=;==SSSSSSSSSSSSSSSSSSSS:SSSe

View File

@@ -1 +0,0 @@
_D=_;domain=.;domain=.;domain=.;;;do“ž“=.;;domain=.;;domain=.;domTTT(TTTTTTTTTTTTTTT*************************************************************************************TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTainn=.;domainin=.;domain=.;;;doman;=o;i.dmain=.;;domain=.;domain=.;domain=.;(;domain=.;;;domain=.;domain=.;om

View File

@@ -1 +0,0 @@
__Host-KDFeec)ure

View File

@@ -0,0 +1 @@
IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII

View File

@@ -0,0 +1 @@
.=;max-age=˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙=-˛

View File

@@ -0,0 +1,24 @@
J="/ path=pires=
Jpath=

View File

@@ -0,0 +1 @@
wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww

View File

@@ -0,0 +1 @@
kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk

View File

@@ -1,2 +0,0 @@
__Hos__Host-ID__Host-ID=x; expires=Mon, 5=x.y; httpontsecure-IDmain=ontsecureontsecure-ID=x; expires=Mon, 29 Feb 2016UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU«ªªªªªªªUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUax-age=2; pat$=/; x.y; httponly; ly; secure
secure

View File

@@ -1,2 +0,0 @@
__H@=x @=x; expires=Mon, 29 Femmmmmmmmmmmqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq*qqqqqqFpath=; secure
x-age; secure

View File

@@ -0,0 +1 @@
ggg

View File

@@ -1 +0,0 @@
__Host-ID=x; expires=29 Feb 2016 07:48:54 GMT;t-ID=x; expiMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM³¨MMMMMMMMMMMMMMMMMMMMMMMMMres=29 Feb 2expires=016 10:48:54 GMT; max-ageGMT; max-age=5; pathomt-ID=x; expires=29 Feb 2016 07:48:54 GMT;t-IDBx; exp=x.y; athomt-ID=x; expires=58 Feb 2016 07:48:54 GMT;t-IDBx; expires=29 Feb 2016 07:48:54 GM;t-IDBx; exp=x.y; athomt-ID=x; expires=29 Fÿÿÿÿÿÿÿÿÿÿeb 2016 07:48:54 GMT;t-IDBx; expires=29 Feb 2016 07:48:54 GMT; max-age=5; pathomai;ost-ID=x; expires=29 FT; max-age=5; pathomai;ost-ID=x; expires=29 Feb 2016 07:48:54 GMT;t-ID=x; expires=2; Feb 2016 10:48:5: GMT; max-age=5; pathomt-ID=x; max-age=5; pathomai;=x.y; httponly

View File

@@ -1 +0,0 @@
__Host-ID=x; e httponjjjjjjjjjjjjjjjjjjjjzjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjzjjjjjjjjjjjjjjjjjjjjjjjjGMT;Žexpires=n=x.; sec}re

View File

@@ -1 +0,0 @@
_D=_;domain=.;domain=.;_;D=_;domain=.;domain=.;_;;domain=.;domain=.;;;domain=.;;domain=.;;domain=.;domainn=.;domain=.;_;Domain=.;(;domain=.=_;domain=.;domain=.;_;;domain=.;domain=.;;;domain=.;;domain=.;;domain=.;domain=.;domain=.;(;domain=.;;;domain=.;domain=.;;;domain=.;d_D=_;domain=.;domain=.;_;D=_;dom_;D=_;domain=.;domain=.;_;;domain=.;domain==_;dom_;D=_;domain=.;domain=.;_;;domain=.;domain=.;;;domain=.;_NNN)NNNNires=M/n,çË4 F; dNNNNNNNNNNNNNNNNy; ;domain=.;;domain=.;domainn=.;domain=.;_;D=_;domain=.;domain=.;domain=.;;;domain=.;domain=.;;;domain=.;;domain=..;;;domain=.;;domain=.;;domain=.;domainn=.;domain=.;_;D=_;domain=.;domain=.;domain=.;;;domain=.;domain=.;;;domain=.;;domain=.;;domain=.;domainhttp=.;domain=.;;domain=.;;;domain=.;domain=.;domain=.;(;domain=.;m.=ndin=.;;;domain=.;d_D=_;domain=.;domain=.;_;D=_;domain=.;domain=.;_;;doin=.;;;domain=.;domain=.;;;domain=.;;domainin=.;domain=.;domain=.;(;domain=.;;;domain=.;domain=onlyre.;om

Some files were not shown because too many files have changed in this diff Show More