183 Commits

Author SHA1 Message Date
Darshit Shah
e55916e5de Prepare NEWS for new release 2018-11-13 16:24:17 +01:00
Darshit Shah
c4a6b89b2e * configure.ac: gnulib now expects autoconf >=2.63 2018-11-13 16:20:19 +01:00
Darshit Shah
3d4e1f47a1 * gnulib: Update library 2018-11-13 16:20:19 +01:00
Jay Satiro
61271d87f6 * src/init.c: Stop freeing the pointer returned by ws_mypath()
.. since ws_mypath() saves the address it returns in a static pointer
for reuse, to also be returned in later calls.
2018-11-13 15:51:51 +01:00
Darshit Shah
2bc2d2f803 * src/ftp.c(ftp_retrieve_glob): Honor {accept,reject}-regex switches as well 2018-11-13 15:51:51 +01:00
Darshit Shah
8c741da256 * src/ftp.c (ftp_retrieve_glob): Refactor to prevent looping over listing multiple times 2018-11-13 15:51:51 +01:00
Tim Rühsen
d3383d619b * .gitlab-ci.yml: Split into GnuTLS and OpenSSL build 2018-11-11 18:43:46 +01:00
Tim Rühsen
62dced2f43 * Makefile.am: dist clean po/stamp-po 2018-11-11 17:51:37 +01:00
Tim Rühsen
04116409a8 Remove auto-generated files from po/ 2018-11-11 16:58:09 +01:00
Tim Rühsen
7b8325b416 Add VPATH build 2018-11-11 16:57:41 +01:00
Tim Rühsen
11fad3fa72 Revert "Bail out on unexpected 416 server errors"
This reverts commit 6f3b995993.

The code is obviously wrong, see https://savannah.gnu.org/bugs/?54963
Also, the example from the original post doesn't work any more.
With other words, the broken server behavior has been fixed meanwhile.
2018-11-09 16:16:43 +01:00
Rosen Penev
a3643c6076 openssl: Do not use engines when OpenSSL does not support
* src/openssl.c: Check for OPENSSL_NO_ENGINE before
 including openssl/engine.h and before calling ENGINE_load_builtin_engines()

Fixes compilation with no engines compiled.

Copyright-paperwork-exempt: Yes
Signed-off-by: Rosen Penev <rosenp@gmail.com>
2018-11-09 16:01:51 +01:00
Tim Rühsen
ed23e682da Fix HTTPS Perl tests
* tests/SSLTest.pm: Rename server cert and key file
* tests/Test-https*.px: Fix and remove OpenSSL hard-coding
* tests/certs/create-certs.sh: Script to generate test files
* tests/certs/*-template.txt: GnuTLS template files for certs and crl
* tests/certs/*.pem: Keys, certs, crls
* tests/certs/README: Removed commands, link to create-certs.sh
2018-11-09 15:47:53 +01:00
Kapus, Timotej
6d7cd9313c Replace some loops with string.h functions
* src/init.c: Replace loop with strspn
* src/url.c: Replace loop with strrchr

Copyright-paperwork-exempt: Yes
2018-10-28 10:36:46 +01:00
Luiz Angelo Daros de Luca
7903dbc9d8 * .gitmodules: Use https:// instead of git:// for gnulib
git:// does not work over http proxy

Signed-off-by: Luiz Angelo Daros de Luca <luizluca@gmail.com>
Copyright-paperwork-exempt: Yes
2018-10-26 22:55:44 +02:00
Luiz Angelo Daros de Luca
fd85ac9cc6 * src/host.c (sufmatch): Fix dot-prefixed domain matching
Current sufmatch does not match when domain is dot-prefixed.
The example of no_proxy in man (.mit.edu) does use a dot-prefixed
domain.

Signed-off-by: Luiz Angelo Daros de Luca <luizluca@gmail.com>
Copyright-paperwork-exempt: Yes
2018-10-26 22:54:26 +02:00
Tim Rühsen
21daa24e72 * src/convert.c (convert_links): Fix fallthrough 2018-10-26 22:52:41 +02:00
Darshit Shah
6d6679ba86 * bootstrap: Update script from gnulib 2018-10-22 16:45:58 +02:00
Darshit Shah
d18f78b86b * gnulib: Update library 2018-10-22 16:45:58 +02:00
Tim Rühsen
85310bfa58 * .lgtm.yml: New file to add LGTM to Gitlab.com CI 2018-10-19 12:32:01 +02:00
Tim Rühsen
b29854528a * configure.ac: Fix build issue with libgpgme 2018-10-16 10:31:54 +02:00
Tim Rühsen
3d518f125c * fuzz/*_fuzzer.in/*: Update fuzzer corpora 2018-10-14 15:15:49 +02:00
Nikos Mavrogiannopoulos
c11cc83d9e Enable post-handshake auth under gnutls on TLS1.3 2018-10-08 15:55:48 +02:00
Tim Rühsen
0727b8f3a9 * src/http.c (resp_new): Fix code to avoid false positive by clang 2018-09-20 14:59:06 +02:00
Tim Rühsen
88a49c1e41 * src/convert.c (convert_links): Fix code to avoid false positive by clang 2018-09-20 14:58:27 +02:00
Tim Rühsen
02afe1e41c Add support for PCRE2 pattern matching
* configure.ac: Check for libpcre2-8
* src/init.c (choices): Test for HAVE_LIBPCRE2
* src/main.c (main): Set regex compile and match functions
* src/options.h: Test for HAVE_LIBPCRE2
* src/utils.c: Include pcre2.h, add functions
  compile_pcre2_regex() and match_pcre2_regex()
* src/utils.h: Declare compile_pcre2_regex() and match_pcre2_regex()

Fixes #54677
Reported-by: Noël Köthe
2018-09-19 16:22:25 +02:00
Tim Rühsen
79be99aff4 Add . to perl path for all perl tests
* tests/*.px: Add -I . to the shebang

This allows perl test to be run from tests/ directory, e.g. via
  ./Test--post-file.px
2018-09-07 10:48:14 +02:00
Tomas Hozza
2bbdfd76da Add TLS 1.3 support for GnuTLS
* doc/wget.texi: Add "TLSv1_3" to --secure-protocol
* src/gnutls.c (set_prio_default): Use GNUTLS_TLS1_3 where needed

Wget currently allows specifying "TLSv1_3" as the parameter for
--secure-protocol option. However it is only implemented for OpenSSL
and in case wget is compiled with GnuTLS, it causes wget to abort with:
GnuTLS: unimplemented 'secure-protocol' option value 6

GnuTLS contains TLS 1.3 implementation since version 3.6.3 [1]. However
currently it must be enabled explicitly in the application of it to be
used. This will change after the draft is finalized. [2] However for
the time being, I enabled it explicitly in case "TLSv1_3" is used with
--secure-protocol.

I also fixed man page to contain "TLSv1_3" in all listings of available
parameters for --secure-protocol

[1] https://lists.gnupg.org/pipermail/gnutls-devel/2018-July/008584.html
[2] https://nikmav.blogspot.com/2018/05/gnutls-and-tls-13.html

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2018-09-07 09:56:02 +02:00
Tomas Korbar
7ddcebd61e Avoid creating empty wget-log when using -O and -q in background
* src/log.c (check_redirect_output): Check for quiet mode
2018-08-29 12:34:03 +02:00
Tomas Hozza
2f451dbf4e * src/warc.c (warc_write_cdx_record): Fix RESOURCE LEAK found by Coverity
Error: RESOURCE_LEAK (CWE-772): - REAL ERROR
wget-1.19.5/src/warc.c:1376: alloc_fn: Storage is returned from allocation function "url_escape".
wget-1.19.5/src/url.c:284:3: alloc_fn: Storage is returned from allocation function "url_escape_1".
wget-1.19.5/src/url.c:255:3: alloc_fn: Storage is returned from allocation function "xmalloc".
wget-1.19.5/lib/xmalloc.c:41:11: alloc_fn: Storage is returned from allocation function "malloc".
wget-1.19.5/lib/xmalloc.c:41:11: var_assign: Assigning: "p" = "malloc(n)".
wget-1.19.5/lib/xmalloc.c:44:3: return_alloc: Returning allocated memory "p".
wget-1.19.5/src/url.c:255:3: var_assign: Assigning: "newstr" = "xmalloc(newlen + 1)".
wget-1.19.5/src/url.c:258:3: var_assign: Assigning: "p2" = "newstr".
wget-1.19.5/src/url.c:275:3: return_alloc: Returning allocated memory "newstr".
wget-1.19.5/src/url.c:284:3: return_alloc_fn: Directly returning storage allocated by "url_escape_1".
wget-1.19.5/src/warc.c:1376: var_assign: Assigning: "redirect_location" = storage returned from "url_escape(redirect_location)".
wget-1.19.5/src/warc.c:1381: noescape: Resource "redirect_location" is not freed or pointed-to in "fprintf".
wget-1.19.5/src/warc.c:1387: leaked_storage: Returning without freeing "redirect_location" leaks the storage that it points to.
\# 1385|     fflush (warc_current_cdx_file);
\# 1386|
\# 1387|->   return true;
\# 1388|   }
\# 1389|

url_escape() really returns a newly allocated memory and it leaks when the warc_write_cdx_record() returns. The memory returned from url_escape() is usually stored in a temporary variable in other parts of the project and then freed. I took the same approach.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2018-08-27 13:25:34 +02:00
Tomas Hozza
8b451f9f21 * src/warc.c (warc_write_start_record): Fix potential RESOURCE LEAK
In warc_write_start_record() function, the reutrn value of dup() is
directly used in gzdopen() call and not stored anywhere. However the
zlib documentation says that "The duplicated descriptor should be saved
to avoid a leak, since gzdopen does not close fd if it fails." [1].
This change stores the FD in a variable and closes it in case gzopen()
fails.

[1] https://www.zlib.net/manual.html

Error: RESOURCE_LEAK (CWE-772):
wget-1.19.5/src/warc.c:217: open_fn: Returning handle opened by "dup".
wget-1.19.5/src/warc.c:217: leaked_handle: Failing to save or close handle opened by "dup(fileno(warc_current_file))" leaks it.
\#  215|
\#  216|         /* Start a new GZIP stream. */
\#  217|->       warc_current_gzfile = gzdopen (dup (fileno (warc_current_file)), "wb9");
\#  218|         warc_current_gzfile_uncompressed_size = 0;
\#  219|

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2018-08-27 13:25:07 +02:00
Tomas Hozza
c045cdded4 * src/utils.c (open_stat): Fix RESOURCE LEAK found by Coverity
Error: RESOURCE_LEAK (CWE-772):
wget-1.19.5/src/utils.c:914: open_fn: Returning handle opened by "open". [Note: The source code implementation of the function has been overridden by a user model.]
wget-1.19.5/src/utils.c:914: var_assign: Assigning: "fd" = handle returned from "open(fname, flags, mode)".
wget-1.19.5/src/utils.c:921: noescape: Resource "fd" is not freed or pointed-to in "fstat". [Note: The source code implementation of the function has been overridden by a builtin model.]
wget-1.19.5/src/utils.c:924: leaked_handle: Handle variable "fd" going out of scope leaks the handle.
\#  922|     {
\#  923|       logprintf (LOG_NOTQUIET, _("Failed to stat file %s, error: %s\n"), fname, strerror(errno));
\#  924|->     return -1;
\#  925|     }
\#  926|   #if !(defined(WINDOWS) || defined(__VMS))

This seems to be a real issue, since the opened file descriptor in "fd"
would leak. There is also additional check below the "fstat" call, which
closes the opened "fd".

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2018-08-27 13:24:46 +02:00
Tomas Hozza
dfef92bac3 * src/http.c (http_loop): Fix RESOURCE LEAK found by Coverity
Error: RESOURCE_LEAK (CWE-772):
wget-1.19.5/src/http.c:4486: alloc_fn: Storage is returned from allocation function "url_string".
wget-1.19.5/src/url.c:2248:3: alloc_fn: Storage is returned from allocation function "xmalloc".
wget-1.19.5/lib/xmalloc.c:41:11: alloc_fn: Storage is returned from allocation function "malloc".
wget-1.19.5/lib/xmalloc.c:41:11: var_assign: Assigning: "p" = "malloc(n)".
wget-1.19.5/lib/xmalloc.c:44:3: return_alloc: Returning allocated memory "p".
wget-1.19.5/src/url.c:2248:3: var_assign: Assigning: "result" = "xmalloc(size)".
wget-1.19.5/src/url.c:2248:3: var_assign: Assigning: "p" = "result".
wget-1.19.5/src/url.c:2250:3: noescape: Resource "p" is not freed or pointed-to in function "memcpy". [Note: The source code implementation of the function has been overridden by a builtin model.]
wget-1.19.5/src/url.c:2253:7: noescape: Resource "p" is not freed or pointed-to in function "memcpy". [Note: The source code implementation of the function has been overridden by a builtin model.]
wget-1.19.5/src/url.c:2257:11: noescape: Resource "p" is not freed or pointed-to in function "memcpy". [Note: The source code implementation of the function has been overridden by a builtin model.]
wget-1.19.5/src/url.c:2264:3: noescape: Resource "p" is not freed or pointed-to in function "memcpy". [Note: The source code implementation of the function has been overridden by a builtin model.]
wget-1.19.5/src/url.c:2270:7: identity_transfer: Passing "p" as argument 1 to function "number_to_string", which returns an offset off that argument.
wget-1.19.5/src/utils.c:1776:11: var_assign_parm: Assigning: "p" = "buffer".
wget-1.19.5/src/utils.c:1847:3: return_var: Returning "p", which is a copy of a parameter.
wget-1.19.5/src/url.c:2270:7: noescape: Resource "p" is not freed or pointed-to in function "number_to_string".
wget-1.19.5/src/utils.c:1774:25: noescape: "number_to_string(char *, wgint)" does not free or save its parameter "buffer".
wget-1.19.5/src/url.c:2270:7: var_assign: Assigning: "p" = "number_to_string(p, url->port)".
wget-1.19.5/src/url.c:2273:3: noescape: Resource "p" is not freed or pointed-to in function "full_path_write".
wget-1.19.5/src/url.c:1078:47: noescape: "full_path_write(struct url const *, char *)" does not free or save its parameter "where".
wget-1.19.5/src/url.c:2287:3: return_alloc: Returning allocated memory "result".
wget-1.19.5/src/http.c:4486: var_assign: Assigning: "hurl" = storage returned from "url_string(u, URL_AUTH_HIDE_PASSWD)".
wget-1.19.5/src/http.c:4487: noescape: Resource "hurl" is not freed or pointed-to in "logprintf".
wget-1.19.5/src/http.c:4513: leaked_storage: Variable "hurl" going out of scope leaks the storage it points to.
\# 4511|               {
\# 4512|                 printwhat (count, opt.ntry);
\# 4513|->               continue;
\# 4514|               }
\# 4515|             else

There are two conditional branches, which call continue, without freeing memory potentially allocated and pointed to by"hurl" pointer. In fase "!opt.verbose" is True and some of the appropriate conditions in the following if/else if construction, in which "continue" is called, are also true, then the memory allocated to "hurl" will leak.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2018-08-27 13:24:24 +02:00
Tomas Hozza
b8be904ac7 * src/http.c (check_auth): Fix RESOURCE LEAK found by Coverity
Error: RESOURCE_LEAK (CWE-772):
wget-1.19.5/src/http.c:2434: alloc_fn: Storage is returned from allocation function "xmalloc".
wget-1.19.5/lib/xmalloc.c:41:11: alloc_fn: Storage is returned from allocation function "malloc".
wget-1.19.5/lib/xmalloc.c:41:11: var_assign: Assigning: "p" = "malloc(n)".
wget-1.19.5/lib/xmalloc.c:44:3: return_alloc: Returning allocated memory "p".
wget-1.19.5/src/http.c:2434: var_assign: Assigning: "auth_stat" = storage returned from "xmalloc(4UL)".
wget-1.19.5/src/http.c:2446: noescape: Resource "auth_stat" is not freed or pointed-to in "create_authorization_line".
wget-1.19.5/src/http.c:5203:70: noescape: "create_authorization_line(char const *, char const *, char const *, char const *, char const *, _Bool *, uerr_t *)" does not free or save its parameter "auth_err".
wget-1.19.5/src/http.c:2476: leaked_storage: Variable "auth_stat" going out of scope leaks the storage it points to.
\# 2474|                 /* Creating the Authorization header went wrong */
\# 2475|               }
\# 2476|->         }
\# 2477|         else
\# 2478|           {

Error: RESOURCE_LEAK (CWE-772):
wget-1.19.5/src/http.c:2431: alloc_fn: Storage is returned from allocation function "url_full_path".
wget-1.19.5/src/url.c:1105:19: alloc_fn: Storage is returned from allocation function "xmalloc".
wget-1.19.5/lib/xmalloc.c:41:11: alloc_fn: Storage is returned from allocation function "malloc".
wget-1.19.5/lib/xmalloc.c:41:11: var_assign: Assigning: "p" = "malloc(n)".
wget-1.19.5/lib/xmalloc.c:44:3: return_alloc: Returning allocated memory "p".
wget-1.19.5/src/url.c:1105:19: var_assign: Assigning: "full_path" = "xmalloc(length + 1)".
wget-1.19.5/src/url.c:1107:3: noescape: Resource "full_path" is not freed or pointed-to in function "full_path_write".
wget-1.19.5/src/url.c:1078:47: noescape: "full_path_write(struct url const *, char *)" does not free or save its parameter "where".
wget-1.19.5/src/url.c:1110:3: return_alloc: Returning allocated memory "full_path".
wget-1.19.5/src/http.c:2431: var_assign: Assigning: "pth" = storage returned from "url_full_path(u)".
wget-1.19.5/src/http.c:2446: noescape: Resource "pth" is not freed or pointed-to in "create_authorization_line".
wget-1.19.5/src/http.c:5203:40: noescape: "create_authorization_line(char const *, char const *, char const *, char const *, char const *, _Bool *, uerr_t *)" does not free or save its parameter "path".
wget-1.19.5/src/http.c:2476: leaked_storage: Variable "pth" going out of scope leaks the storage it points to.
\# 2474|                 /* Creating the Authorization header went wrong */
\# 2475|               }
\# 2476|->         }
\# 2477|         else
\# 2478|           {

Both "pth" and "auth_stat" are allocated in "check_auth()" function. These are used for creating the HTTP Authorization Request header via "create_authorization_line()" function. In case the creation went OK (auth_err == RETROK), then the memory previously allocated to "pth" and "auth_stat" is freed. However if the creation failed, then the memory is never freed and it leaks.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2018-08-27 13:23:52 +02:00
Tomas Hozza
b24351183e * src/ftp.c (getftp): Fix RESOURCE LEAK found by Coverity
Error: RESOURCE_LEAK (CWE-772):
wget-1.19.5/src/ftp.c:1493: alloc_fn: Storage is returned from allocation function "fopen".
wget-1.19.5/src/ftp.c:1493: var_assign: Assigning: "fp" = storage returned from "fopen(con->target, "wb")".
wget-1.19.5/src/ftp.c:1811: leaked_storage: Variable "fp" going out of scope leaks the storage it points to.
\# 1809|     if (fp && !output_stream)
\# 1810|       fclose (fp);
\# 1811|->   return err;
\# 1812|   }
\# 1813|

It can happen, that "if (!output_stream || con->cmd & DO_LIST)" on line #1398 can be true, even though "output_stream != NULL". In this case a new file is opened to "fp". Later it may happen in the FTPS branch, that some error will occure and code will jump to label "exit_error". In "exit_error", the "fp" is closed only if "output_stream == NULL". However this may not be true as described earlier and "fp" leaks.

On line #1588, there is the following conditional free of "fp":

  /* Close the local file.  */
  if (!output_stream || con->cmd & DO_LIST)
    fclose (fp);

Therefore the conditional at the end of the function after "exit_error" label should be modified to:

  if (fp && (!output_stream || con->cmd & DO_LIST))
    fclose (fp);

This will ensure that "fp" does not leak in any case it sould be opened.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2018-08-27 13:20:48 +02:00
Tomas Hozza
2b2283d3e2 Don't limit the test suite HTTPS server to TLSv1
In Fedora, we are implementing crypto policies, in order to enhance the
security of user systems. This is done on the system level by global
configuration. It may happen that due to the active policy, only
TLSv1.2 or higher will be available in crypto libraries. While wget as
a client will by default determine the minimal TLS version supported by
both client and server, the HTTPS server implementation in testenv/
hardcodes use of TLSv1. As a result all HTTPS related tests fail in
case a more hardened crypto policy is set on the Fedora system.

This change removes the explicit TLS version setting and leaves the
determination of the minimal supported TLS version on the server and
client.

More information about Fedora change can be found here:
https://fedoraproject.org/wiki/Changes/StrongCryptoSettings

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2018-08-11 12:51:13 +02:00
Tim Rühsen
122a9f08a3 * src/gnutls.c (ssl_check_certificate): Fix grammar of error msg
Reported-by: Nicholas Sielicki
2018-06-13 20:34:24 +02:00
Tim Rühsen
333746f787 * fuzz/Makefile.am: Remove libtool LTLIB... from LDADD 2018-06-13 20:16:46 +02:00
Tim Rühsen
4fc69950da * src/http.c (http_loop): Fix --retry-on-host-error 2018-06-13 20:16:22 +02:00
ethus3h
e7979da9e8 Add new option --retry-on-host-error
* doc/wget.texi: Add docs for --retry-on-host-error
* src/http.c (http_loop): Add code for HOSTERR
* src/init.c: Add option --retry-on-host-error
* src/main.c: Likewise
* src/options.h: Add options.retry_on_host_error

Copyright-paperwork-exempt: Yes
2018-06-13 20:10:28 +02:00
Tim Rühsen
ad261f41ce Save original data to WARC file
* src/retr.c (write_data): Cleanup,
  (fd_read_body): Write to WARC before uncompressing

Fixes: #53968
2018-05-29 10:52:20 +02:00
Tim Rühsen
c88500fca8 * fuzz/get_ossfuzz_corpora: Speed up corpora download 2018-05-10 19:33:59 +02:00
Tim Rühsen
4188fcdced * src/main.c (print_version): Silence UBSAN message 2018-05-09 13:56:20 +02:00
Tim Rühsen
4bdb09d3a7 * src/utils.ci (file_exists_p): Fix stat(NULL,...) 2018-05-09 12:37:03 +02:00
Tim Rühsen
35f5f79ce1 * src/hsts.c (open_hsts_test_store): Fix unlink(NULL) 2018-05-09 12:29:39 +02:00
Tim Rühsen
3cbdc67c96 * src/hash.c: Silence UBSAN for hash functions 2018-05-09 12:16:51 +02:00
Tim Rühsen
cdaee00259 * fuzz/*_fuzzer.in: Update corpora from OSS-Fuzz 2018-05-09 11:45:22 +02:00
Tim Rühsen
ad2471425f * fuzz/get_ossfuzz_corpora: Fix path 2018-05-09 11:44:18 +02:00
Tim Rühsen
ace96e4412 * src/hsts.h: Fix header guard 2018-05-08 10:17:06 +02:00
Tim Rühsen
77286a2e03 * src/version.h: Add header guard 2018-05-08 10:10:44 +02:00
Tim Rühsen
7eff94e881 * src/host.c (wait_ares): Remove void assignment
Reported-by: Josef Moellers
2018-05-08 09:36:48 +02:00
Tim Rühsen
15a39093b8 Update NEWS file for new release 2018-05-06 18:38:29 +02:00
Tim Rühsen
1fc9c95ec1 Fix cookie injection (CVE-2018-0494)
* src/http.c (resp_new): Replace \r\n by space in continuation lines

Fixes #53763
 "Malicious website can write arbitrary cookie entries to cookie jar"

HTTP header parsing left the \r\n from continuation line intact.
The Set-Cookie code didn't check and could be tricked to write
\r\n into the cookie jar, allowing a server to generate cookies at will.
2018-05-06 18:24:58 +02:00
Tim Rühsen
f51936745a * tests/Test-https-weboftrust.px: Skip test, needs cert regen 2018-05-06 18:19:50 +02:00
Tim Rühsen
491c6914cb Fix make syntax-check
* cfg.mk: Add fuzzer reproducers to exception list
* po/POTFILES.in: Add src/spider.c
2018-05-06 17:44:37 +02:00
Tim Rühsen
a6452061f8 Fix HTTPS tests
* tests/Test-https-badcerts.px: Fix test return value
* tests/Test-https-crl.px: Likewise
* README: How to create certs with GnuTLS's certtool
* tests/certs/revokedcrl.pem: Recreated revocation
* tests/certs/server.crt: Recreated server cert with no expiry
* tests/certs/test-ca-cert.pem: Recreated CA cert with no expiry
2018-05-06 17:30:42 +02:00
Tim Rühsen
77cf701416 * src/init.c: Bring new --ciphers into right order in options array 2018-05-06 12:49:46 +02:00
Ander Juaristi
c4eb863299 * doc/wget.texi: Add description for --ciphers 2018-05-05 22:50:23 +02:00
Ander Juarist
b9c4cadd84 OpenSSL: Better seeding of PRNG
* src/openssl.c (init_prng): keep gathering entropy even though we
                              already have enough
   (ssl_connect_with_timeout_callback): reseed PRNG again just before
                                        the handshake

Reported-by: Jeffrey Walton <noloader@gmail.com>
2018-05-05 22:49:06 +02:00
Ander Juaristi
744671aac6 Enhance SSL/TLS security
This commit hardens SSL/TLS a bit more in the following ways:

 * Explicitly exclude NULL authentication and the 'MEDIUM' cipher list
   category. Ciphers in the 'HIGH' level are only considered - this
   includes all symmetric ciphers with key lengths larger than 128 bits,
   and some ('modern') 128-bit ciphers, such as AES in GCM mode.
 * Allow RSA key exchange by default, but exclude it when
   Perfect Forward Secrecy is desired (with --secure-protocol=PFS).
 * Introduce new option --ciphers to set the cipher list that the SSL/TLS
   engine will favor. This string is fed directly to the underlying TLS
   library (GnuTLS or OpenSSL) without further processing, and hence its
   format and syntax are directly dependent on the specific library.

Reported-by: Jeffrey Walton <noloader@gmail.com>
2018-05-05 22:49:06 +02:00
Tim Rühsen
26a50942d8 * src/netrc.c (parse_netrc_fp): Fix two memleaks 2018-04-28 20:50:30 +02:00
Tim Rühsen
a1c9018797 Add new fuzzer for the .netrc parser
* fuzz/wget_netrc_fuzzer.c: New fuzzer
* fuzz/wget_netrc_fuzzer.dict: Fuzzer dictionary
* fuzz/wget_netrc_fuzzer.in: Initial corpora
* src/ftp.c (getftp): Amend call to search_netrc()
* src/http.c (initialize_request): Likewise
* src/netrc.c: Cleanup, prepare code for fuzzing
* src/netrc.h: Cleanup
2018-04-28 20:49:57 +02:00
Tim Rühsen
734d0aee15 * src/utils.c (match_tail): Fix unsigned integer overflow 2018-04-27 12:56:25 +02:00
Tim Rühsen
7de006bade Add new fuzzer for the Set-Cookie parser
* fuzz/Makefile.am: Add wget_cookie_fuzzer
* fuzz/wget_cookie_fuzzer.c: New fuzzer
* fuzz/wget_cookie_fuzzer.dict: Fuzzers dictionary
* fuzz/wget_cookie_fuzzer.in: Initial corpora
2018-04-27 12:56:25 +02:00
Tim Rühsen
78838d761f Fix buffer overflow in CSS parser
* src/css-url.c (get_uri_string): Check input length
* fuzz/wget_css_fuzzer.repro/buffer-overflow-6600180399865856:
  Add reproducer corpus

Fixes OSS-Fuzz issue #8033.
This is a long standing bug affecting all versions <= 1.19.4.
2018-04-26 22:40:28 +02:00
Tim Rühsen
cb47f3aaa4 Fix buffer overflow in CSS parser
* src/css-url.c (get_urls_css): Check input string length
* fuzz/wget_css_fuzzer.repro/negative-size-param-5724866467594240:
  Add reproducer corpus

Fixes OSS-Fuzz issue #8032.
This is a long standing bug affecting all versions <= 1.19.4.
2018-04-26 21:25:28 +02:00
Tim Rühsen
acfd9b4d56 Exclude fuzz corpora from tarball
* fuzz/Makefile.am: Do not include corpora in tarball
* fuzz/main.c: SKIP if corpora directory isn't found (make check)

The fuzz corpora are thousands of files, not needed for a standard build
from a distribution tarball. The reproducers of former issues are being
included for regression testing.
2018-04-26 16:18:01 +02:00
Tim Rühsen
ceb5d2d794 * tests/Makefile.am: Add -I/src to AM_CPPFLAGS 2018-04-26 16:17:10 +02:00
Tim Rühsen
939dbb0ebb Add CSS slowness reproducer (fixed)
* fuzz/wget_css_fuzzer.repro/slowness-6275836549267456: New file

This file created an extreme CPU usage with the old CSS parser.
2018-04-26 16:07:46 +02:00
Tim Rühsen
caa08d7470 Update CSS grammar from 1.x to 2.2
* src/css-tokens.h: Add enums and fixate values
* src/css.l: Include config.h,
  ignore several compiler warnings,
  update the grammar to CSS 2.2

Fixes OSS-Fuzz issue #8010 (slowness issue).
This is a long standing bug affecting all versions <= 1.19.4.

Some crafted CSS input was extremely slow / CPU wasting, so it could
be used as a DOS attack against website scanning.

The code/grammar changes were backported from Wget2.x.
2018-04-26 13:10:39 +02:00
Tim Rühsen
76fb1fe6f6 * src/res.c (add_path): Fix memleak (parsing robots.txt)
Fixes OSS-Fuzz issue #8005.
This is a long standing bug affecting all versions <= 1.19.4.
2018-04-25 11:33:38 +02:00
Tim Rühsen
fe6d1247ad * src/ftp-ls.c (ftp_parse_winnt_ls): Fix integer overflow
Fixes OSS-Fuzz issue #7999.
This is a long standing bug affecting all versions <= 1.19.4.
2018-04-25 09:37:29 +02:00
Tim Rühsen
02325168ca Add new fuzzer for the URL parser
* fuzz/Makefile.am: Add wget_url_fuzzer
* fuzz/wget_url_fuzzer.c: New fuzzer
* fuzz/wget_url_fuzzer.in: Initial corpora
2018-04-24 21:36:06 +02:00
Tim Rühsen
93e5a97f25 Add new fuzzer for robots.txt parsing
* fuzz/Makefile.am: Add wget_robots_fuzzer
* fuzz/wget_robots_fuzzer.c: New fuzzer
* fuzz/wget_robots_fuzzer.in: Initial corpora
2018-04-24 11:47:49 +02:00
Tim Rühsen
36482a21ea * fuzz/README.md: Add CFLAGS for undefined sanitizer 2018-04-24 11:30:06 +02:00
Tim Rühsen
7ee3ad1c48 * src/ftp-ls.c (ftp_parse_winnt_ls): Fix integer overflow 2018-04-24 11:11:47 +02:00
Tim Rühsen
79c1f333dc * src/ftp-ls.c (ftp_parse_vms_ls): Fix integer overflow by left shift 2018-04-24 11:05:52 +02:00
Tim Rühsen
d8365b0607 * src/ftp-ls.c (ftp_parse_unix_ls): Fix integer overflow in date parsing 2018-04-24 10:55:29 +02:00
Tim Rühsen
b0f802c46c * src/ftp-ls.c (ftp_parse_winnt_ls): Fix heap-buffer-overflow
Fixes OSS-Fuzz issue #7931.
This is a long standing bug affecting all versions <= 1.19.4.
2018-04-22 12:45:51 +02:00
Tim Rühsen
96c64a859d * src/ftp-ls.c (ftp_parse_winnt_ls): Fix heap-buffer-overflow
Fixes OSS-Fuzz issue #7930.
This is a long standing bug affecting all versions <= 1.19.4.
2018-04-22 11:33:35 +02:00
Tim Rühsen
2269cc2f1b * fuzz/wget_ftpls_fuzzer.in: Update corpora 2018-04-22 00:29:47 +02:00
Tim Rühsen
7d3da08537 * src/ftp-ls.c (eat_carets): Fix heap-buffer-overflow 2018-04-21 23:48:01 +02:00
Tim Rühsen
2b61c46183 * src/ftp-ls.c (ftp_parse_winnt_ls): Fix memleak 2018-04-21 22:52:01 +02:00
Tim Rühsen
f0d715b264 * src/ftp-ls.c (ftp_parse_vms_ls): Fix heap-buffer-overflow 2018-04-21 22:47:17 +02:00
Tim Rühsen
b3ff8ce3d5 * src/ftp-ls.c (ftp_parse_vms_ls): Fix heap-buffer-overflow 2018-04-21 22:45:03 +02:00
Tim Rühsen
c7014fbaea * src/ftp-ls.c (ftp_parse_vms_ls): Fix memleak 2018-04-21 22:42:30 +02:00
Tim Rühsen
407cd5f23b Add new fuzzer for the FTP listing parsers
* fuzz/Makefile.am: Add wget_ftpls_fuzzer
* fuzz/wget_ftpls_fuzzer.c: New fuzzer
* fuzz/wget_ftpls_fuzzer.dict: Fuzzer dictionary
* fuzz/wget_ftpls_fuzzer.in/starter: Starting corpus
* src/ftp-ls.c: Parsing function take FILE * as argument,
  new function ftp_parse_ls_fp()
* src/ftp.c: Remove static from freefileinfo()
* src/ftp.h: Add ftp_parse_ls_fp() and freefileinfo()
2018-04-21 19:24:25 +02:00
Tim Rühsen
fbc5f3736e * fuzz/run-clang.sh: Remove -detect_leaks=0 from fuzzer command line 2018-04-21 18:22:37 +02:00
Tim Rühsen
7ecfe3ef70 * src/main.c (main): Fix memleak for fuzzing/testing 2018-04-21 18:21:52 +02:00
Tim Rühsen
7e635d173e * src/init.c: Fix fuzzing in case ~/.wgetrc doesn't exist 2018-04-21 16:33:45 +02:00
Tim Rühsen
e83dd5b0aa Fix fuzzer build for C++
* fuzz/wget_css_fuzzer.c: Include wget.h outside 'extern "C"',
  undef fopen_wgetrc directly after wget.h
* fuzz/wget_html_fuzzer.c: Likewise
2018-04-20 23:59:58 +02:00
Tim Rühsen
0b02993e39 * fuzz/Makefile.am: Add -I/lib to oss-fuzz builds 2018-04-20 23:28:12 +02:00
Tim Rühsen
23b0275feb Add new HTML parser fuzzer
* fuzz/Makefile.am: Add wget_html_fuzzer
* fuzz/wget_html_fuzzer.c: New fuzzer
* fuzz/wget_html_fuzzer.dict: HTML dictionary for fuzzing
* fuzz/wget_html_fuzzer.in: Initial corpora
* src/html-url.c: Add new function get_urls_html_fm()
* src/html-url.h: Add ne function get_urls_html_fm()
* src/wget.h: Fix define for fopen_wgetrc()
2018-04-20 22:33:58 +02:00
Tim Rühsen
77c31d301b * fuzz/wget_css_fuzzer.c: Fix build 2018-04-20 20:06:50 +02:00
Tim Rühsen
30bd99f3f5 * fuzz/wget_css_fuzzer.in/*: Update fuzzer corpora 2018-04-20 20:04:59 +02:00
Tim Rühsen
c9a091ae45 * src/css-url.c (get_uri_string): Fix buffer overflow (read) 2018-04-20 11:37:52 +02:00
Tim Rühsen
7a5db30b01 * src/iri.h: Fix C++ compile error 2018-04-20 10:17:55 +02:00
Tim Rühsen
9d899d7bb7 * src/http.c: Download and scan CSS files in spider mode 2018-04-19 23:05:06 +02:00
Tim Rühsen
d25d036fba * src/css-url.c (get_urls_css): Call yylex_destroy() to reset CSS scanner 2018-04-19 23:05:06 +02:00
Tim Rühsen
c1a368af89 Add new fuzzer wget_css_fuzzer.c
* fuzz/Makefile.am: Add wget_css_fuzzer.c
* fuzz/wget_css_fuzzer.c: New fuzzer
2018-04-19 23:05:06 +02:00
Tim Rühsen
ff3c7733b7 * src/html-url.h: Include needed header files 2018-04-18 20:41:08 +02:00
Tim Rühsen
cc7f15a529 * wget_options_fuzzer.in/*: Update fuzzer corpora 2018-04-18 16:46:44 +02:00
Tim Rühsen
ae6390e6a4 * fuzz/README.md: Add CXXFLAGS and more configure options 2018-04-18 16:43:51 +02:00
Tim Rühsen
3ae58dae13 Fix oss-fuzz issue with exit()
* src/wget.h: Define exit() as exit_wget()
* fuzz/wget_options_fuzzer.c: Implement exit_wget() and cleanup
2018-04-18 13:26:10 +02:00
Tim Rühsen
bb03572c2a * fuzz/wget_options_fuzzer.c: Declare fopen_* as C functions 2018-04-17 23:35:42 +02:00
Tim Rühsen
66b416b6cd Fix fopen/stdin issues with fuzzing
* fuzz/wget_options_fuzzer.c: Add fopen_wget() and fopen_wgetrc()
* src/utils.c: Use fopen_wgetrc() for config files,
  don't read from stdin when fuzzing
* src/wget.h: Define fopen as fopen_wget when fuzzing,
  define fopen_wgetrc as fopen when not fuzzing
2018-04-17 23:02:04 +02:00
Tim Rühsen
fdd86aada9 * configure.ac: AC_DEFINE FUZZING if --enable-fuzzing was given 2018-04-17 15:55:58 +02:00
Tim Rühsen
ce90ed78b6 * fuzz/wget_options_fuzzer.c: Write fuzzer crash reports 2018-04-17 12:41:34 +02:00
Tim Rühsen
3c4a6506a5 * src/log.c: Don't check_redirect_output() when fuzzing 2018-04-17 12:40:47 +02:00
Tim Rühsen
fbb4cd231e * src/main.c (promt_for_password): Avoid getpass() when fuzzing 2018-04-17 12:15:18 +02:00
Tim Rühsen
3ceb6e5630 Fix double fclose() with -d while fuzzing
* src/ftp.c (ftp_loop_internal): Set warc_tmp to NULL after ffclose()
* src/init.c (cleanup): Set output_stream to NULL after fclose()
* src/log.c (log_close): Set global stream vars to NULL after closing
* src/recur.c (retrieve_tree): Set rejectedlog to NULL after closing
* src/warc.c (warc_close): Set stream vars to NULL after closing
2018-04-17 11:59:54 +02:00
Tim Rühsen
eaf167aaaa * src/main.c (main): Don't background if TESTING 2018-04-17 11:50:36 +02:00
Tim Rühsen
7d5de64fc9 * src/init.c (initialize): Return error, don't exit() 2018-04-17 11:42:43 +02:00
Tim Rühsen
70042265be * src/init.c (cmd_use_askpass): Return false on error 2018-04-16 23:04:53 +02:00
Tim Rühsen
64758655c4 * src/utils.c (compile_posix_regex): Hard-code string to regcomp
regcomp() may be too cpu + memory intensive for fuzzing.
See https://sourceware.org/glibc/wiki/Security%20Exceptions
2018-04-16 22:04:54 +02:00
Tim Rühsen
e737c4b10e Fix 2 more memleaks
* src/init.c (initialize): Use global var for wgetrc filename
* src/iri.c (find_locale): Return strdup'ed locale string
* src/options.h (struct options): Add wgetrcfile
2018-04-16 22:02:11 +02:00
Tim Rühsen
05a8c064e9 * src/init.c (cleanup): Set output_stream to NULL after closing 2018-04-16 13:22:29 +02:00
Tim Rühsen
01002a168a Fix homedir memory leaks
* src/hsts.c: Use opt.homedir
* src/init.c: Likewise
* src/main.c: Likewise
* src/netrc.c: Likewise
* src/options.h (struct options): Add homedir
2018-04-16 13:19:03 +02:00
Tim Rühsen
73fd57585c * src/main.c (main): Free opt.encoding_remote properly 2018-04-16 12:21:52 +02:00
Tim Rühsen
7963260e76 * src/host.c (wait_ares): Free ptimer 2018-04-16 11:58:18 +02:00
Tim Rühsen
99a7039def * src/init.c (cleanup): Free regex objects properly 2018-04-16 11:57:39 +02:00
Tim Rühsen
d7e3acb2cc * src/init.c (cleanup): Never call cleanup() twice 2018-04-16 09:58:51 +02:00
Tim Rühsen
e0860dd1ff * src/init.c (cmd_bytes_sum): Fix integer over- and underflow 2018-04-16 09:58:51 +02:00
Tim Rühsen
15ef79f808 * src/main.c (save_hsts): Free hsts_store after closing 2018-04-16 09:58:51 +02:00
Tim Rühsen
79385a29fd Use strtol() instead of selfmade function
* src/init.c (cmd_number): Use strtol() instead of selfmade function
* bootstrap.conf: Add strtol gnulib module
2018-04-16 09:58:51 +02:00
Tim Rühsen
55da9f71f0 * src/hsts.c (hsts_hash_func): Allow integer overflow 2018-04-16 09:58:51 +02:00
Tim Rühsen
bec9816f40 * init.c (cmd_spec_mirror): Fix uninitialzed stack variable 2018-04-16 09:58:51 +02:00
Tim Rühsen
b86294e1c9 * src/init.c (cleanup): Free more variables 2018-04-16 09:58:51 +02:00
Tim Rühsen
7e1e5f3959 * wget_options_fuzzer.in: Add corpora directory 2018-04-16 09:58:51 +02:00
Tim Rühsen
4a6336d551 * fuzz/wget_options_fuzzer.c: Suppress error messages from wget 2018-04-16 09:58:51 +02:00
Tim Rühsen
328438e69b * src/utils.c (fopen_stat): Early return to allow fuzzing/fmemopen 2018-04-16 09:58:51 +02:00
Tim Rühsen
36f029d2f0 * src/init.c (initialize): Free mem before exit() 2018-04-16 09:58:51 +02:00
Tim Rühsen
a4402120ad Add OSS-Fuzz infrastruture
* Makefile.am: Add fuzz/ to SUBDIRS
* cfg.mk: Fix 'make syntax-check'
* configure.ac: Add --enable-fuzzing
* fuzz/Makefile.am: New file
* fuzz/README.md: New file
* fuzz/fuzzer.h: New file
* fuzz/get_all_corpora: New file
* fuzz/get_ossfuzz_corpora: New file
* fuzz/glob_crash.c: New file
* fuzz/main.c: New file
* fuzz/run-afl.sh: New file
* fuzz/run-clang.sh: New file
* fuzz/view-coverage.sh: New file
* fuzz/wget_options_fuzzer.c: New file
* fuzz/wget_options_fuzzer.dict: New file
* src/init.c (cleanup): Free more resources
* src/main.c (init_switches): Initialize only once,
  (print_usage): Don't print if TESTING is defined
* src/utils.h: Include wget.h
2018-04-16 09:58:51 +02:00
Tim Rühsen
de54c970b2 Move unit-test code to tests/
* src/Makefile.am: Remove test.c and test.h
* src/test.c: Rename to tests/unit-tests.c
* src/test.h: Rename to tests/unit-tests.h
* tests/Makefile.am: Add unit-tests.c and unit-tests.h
* src/hsts.c: Amend #include
* src/http.c: Likewise
* src/init.c: Likewise
* src/metalink.c: Likewise
* src/res.c: Likewise
* src/url.c: Likewise
* src/utils.c: Likewise
2018-04-05 15:06:47 +02:00
Tim Rühsen
3e84963e84 * src/main.c: Rename main() -> main_wget() for unit tests 2018-04-05 15:06:47 +02:00
Tim Rühsen
f56f970bc2 Fix some issues found by 'infer' 2018-03-14 14:43:35 +01:00
Tim Rühsen
0b54043d17 * src/openssl.c: Fix build for OpenSSL 1.1.0 without TLS1_3_VERSION 2018-03-08 16:17:14 +01:00
Tim Rühsen
040106b3e2 Add docs for --secure-protocol=TLSv1_3
* doc/wget.texi: Likewise
2018-03-08 15:30:28 +01:00
Loganaden Velvindron
fde8cefd13 Add TLS1.3 support for OpenSSL build
* src/init.c: Add 'tlsv1_3 for --secure-protocol
* src/openssl.c (ssl_init): Enable TLS1.3 if possible
* src/options.h: Add secure_protocol_tlsv1_3
* doc/wget.texi: Add description of TLSv1_3

Copyright-paperwork-exempt: Yes
2018-03-08 15:30:14 +01:00
Tim Rühsen
ba2b0654b4 * src/main.c: Add help text for --retry-on-http-error
Reported-by: Giovanni Tirloni
2018-03-07 10:32:08 +01:00
Tim Rühsen
375bfa98dc * src/url.c (convert_fname): Fix invalid free on iconv_open() failure
Reported-by: Volkmar Klatt
2018-03-01 16:03:29 +01:00
Tim Rühsen
bea54e0da4 * src/mswindows.c: Fix prototype of fork_to_background()
Reported-by: Gisle Vanem
2018-02-21 19:05:15 +01:00
Tim Rühsen
98ec26cc6d Fix warning to not print binary IP address
* tests/Test-https-badcerts.px: Likewise
* tests/Test-https-clientcert.px: Likewise
* tests/Test-https-crl.px: Likewise
* tests/Test-https-pfs.px: Likewise
* tests/Test-https-selfsigned.px: Likewise
* tests/Test-https-tlsv1.px: Likewise
* tests/Test-https-tlsv1x.px: Likewise
* tests/Test-https-weboftrust.px: Likewise
2018-02-09 10:33:07 +01:00
Tim Rühsen
9887b870d1 Use gnulib's utime()
* bootstrap.conf: Add modules utime and utime-h
* src/utils.c (touch): Remove own code for gnulib's utime()
2018-02-09 10:21:43 +01:00
Tim Rühsen
c722973212 Fix logging in background mode
* ../src/main.c: Re-init logfile if changed for background mode
* ../src/utils.c: fork_to_background() returns whether logfile changed
* ../src/utils.h: Set return type bool for fork_to_background()

Fixes: #53020
Reported-by: Noël Köthe
2018-02-09 10:21:43 +01:00
Tim Rühsen
bb7fa977a1 * src/http.c: Fix two typos in comments 2018-02-09 10:18:35 +01:00
Tim Rühsen
f2d6075a6e * testenv/conf/expected_files.py: Ignore pubring.gpg
Reported-by: Arkadiusz Miśkiewicz
2018-01-31 09:04:42 +01:00
Tim Rühsen
0caed28cda * bootstrap.conf: Replace gnulib module gettext by gettext-h 2018-01-31 09:01:54 +01:00
Tim Rühsen
d27032c446 Mention list and bugtracker for --help and in man page
* doc/wget.texi: Mention list and bugtracker in man page
* src/main.c: Mention list and bugtracker for --help
2018-01-22 10:39:49 +01:00
Darshit Shah
080fd0d93b * contrib/make-release: Add another release script 2018-01-21 13:50:45 +01:00
Darshit Shah
ab8a3c80df Update NEWS file for new release 2018-01-21 11:00:29 +01:00
Darshit Shah
11ecabcb3d Fix typo in documentation 2018-01-21 11:00:25 +01:00
Darshit Shah
d0a5d9f131 Switch off compression by default
Gzip compression has a number of bugs which need to be ironed out before
we can support it by default. Some of these stem from a misunderstanding
of the HTTP spec, but a lot of them are also due to many web servers not
being compliant with RFC 7231.

With this commit, I am marking GZip compression support as experimental
in GNU Wget pending further investigation and the addition of tests.

* src/init.c (defaults): Switch of compression support by default
* docs/wget.texi: State that compression is experimental
2018-01-21 10:51:11 +01:00
Darshit Shah
0ff71e5ff1 Pull gnulib module forward 2018-01-21 10:51:09 +01:00
Darshit Shah
0d0a95a01b Revert "* src/init.c (defaults): Set compression_none as the default compression"
This reverts commit 8283ac0846.
2018-01-21 10:50:44 +01:00
Darshit Shah
8283ac0846 * src/init.c (defaults): Set compression_none as the default compression 2018-01-21 10:17:39 +01:00
Reiji
a7cc4e2b37 * src/http.c (gethttp): Fix bug that prevented all files from being decompressed
Signed-off-by: Darshit Shah <darnir@gnu.org>
2018-01-20 14:04:28 +01:00
Tim Rühsen
55d25fc20c * src/host.c (sufmatch): Fix to domain matching 2018-01-19 19:32:01 +01:00
Gisle Vanem
513cc1c0c8 * src/netrc.c: Fix Standalone compilation of netrc file 2018-01-17 14:44:52 +01:00
Darshit Shah
3249045a91 * NEWS: Update NEWS file for new release 2018-01-14 12:17:31 +01:00
Darshit Shah
b1fc37f465 * bootstrap: Use the faster gnulib-tool.py script if possible 2018-01-14 11:33:52 +01:00
Darshit Shah
953bcdaba8 Sync the bootstrap script from gnulib 2018-01-14 11:28:51 +01:00
Darshit Shah
183fccdaad Update Copyright years 2018-01-14 11:24:43 +01:00
Darshit Shah
8240439e21 Replace HTTP urls with HTTPS where valid 2018-01-14 11:15:32 +01:00
Darshit Shah
a03716a0fe Update gnulib 2018-01-14 11:15:16 +01:00
Darshit Shah
d26c6c0028 * src/netrc.c: Search for the correct netrc file on Windows 2018-01-14 10:55:03 +01:00
Tim Rühsen
047746eb76 * src/http.c: Exclude *.gz and *.tgz from decompression 2018-01-10 15:46:13 +01:00
Tim Rühsen
d8df356d4b * src/utils.c (wg_pin_peer_pubkey): Fix format warning 2017-12-31 13:03:25 +01:00
Peter Wu
220c24ecb5 Avoid redirecting output to file when tcgetpgrp fails
* src/log.c (check_redirect_output): tcgetpgrp can return -1 (ENOTTY),
be sure to check whether a valid controlling terminal exists before
redirecting.

Fixes: #51181
2017-12-31 12:59:15 +01:00
Darshit Shah
3d2b2231cd Add new test for 416 responses
* testenv/server/http/http_server.py: If there are multiple requests in
which the requested range is unsatisfiable, then send a body in the in
the 2nd response onwards
* testenv/Test-416.py: New test to check how Wget handles 416 responses
2017-12-11 14:48:01 +01:00
Darshit Shah
693cee0109 Don't assume a 416 response has no body
* http.c(gethttp): In case of a 416 response, try to drain the socket of
any bytes before reusing the connection

Reported-By: Iru Cai <mytbk920423@gmail.com>
2017-12-08 18:44:17 +01:00
Tim Rühsen
6aa6b669ef Support building with OpenSSL 1.1 w/o deprecated features
* src/openssl.c (ssl_init): Fix code for the subject's issue

Reported-by: Matthew Thode
2017-11-26 18:59:47 +01:00
Tim Rühsen
8551ceccfe Avoid link conversion after 304 Not Modified
* src/http.c (gethttp): Handle 304 before setting document content type

Fixes: #52404
Reported-by: Ben Fuchs
2017-11-25 19:33:03 +01:00
YX Hao
19060db44f Fix printing mutibyte chars as unprintable chars on Windows
* src/log.c (get_warc_log_fp): Fix return value to stderr
* src/main.c (main): Init logging as soon as possible,
  fix locale/charset on Windows
2017-11-16 12:23:20 +01:00
Tim Rühsen
a30a5a9059 * tests/Test-https-weboftrust.px: Fix/add sslport 2017-11-16 12:12:16 +01:00
YX Hao
a9a953feee Convert remote path to local encoding
* src/url.c (url_file_name): Convert remote path to local encoding
2017-11-15 19:58:53 +01:00
Tim Rühsen
267cd51fff Do not use must-revalidate in Cache-Control header
As the bug report states, 'must-revalidate' is a request directive.

Fixes #52379
2017-11-10 10:57:46 +01:00
Darshit Shah
973c26ed7d Fix Segfault due to derefencing null ptr
* src/http.c(gethttp): When Encoding is gzip, ensure that the
Content-Type Header was actually seen. Without this, the "type" variable
is null causing a Segfault.

Reported-By: Noël Köthe <noel@debian.org>
2017-11-06 10:09:03 +01:00
Tim Rühsen
16d066f89c * src/http.c: Fix H_REDIRECTED 2017-11-03 22:23:04 +01:00
Tim Rühsen
a2477d487c * src/http.c: Add support for HTTP status code 308 2017-11-03 22:12:11 +01:00
Tim Rühsen
7ac72e063c Fix uninitialized value messages in tests
* tests/Test-https-*.px: Use correct variable in error message

Reported-by: Jeffrey Walton
2017-10-30 21:29:14 +01:00
Tim Rühsen
95891cbba3 Ignore dirmngr.conf, gpg.conf in test directories
* testenv/conf/expected_files.py: Ignore dirmngr.conf, gpg.conf

Libgpgme creates certain files if $HOME doesn't contain the .gnupg directory.
These files disturb some metalink tests if we don't ignore them.

Reported-by: Arkadiusz Miśkiewicz <arekm@maven.pl>
2017-10-30 13:04:09 +01:00
Tim Rühsen
2486320a2a * contrib/release: Cleanup, use lzip instead of xz 2017-10-27 11:26:37 +02:00
9542 changed files with 36042 additions and 2426 deletions

View File

@@ -1,9 +1,63 @@
# we utilize the images generated by the build-images project, to
# speed up CI runs. We also use ccache and store config.cache
# to speed up compilation. We include a version number in cache
# name to allow expiration of old caches.
cache:
key: "$CI_JOB_NAME-ver5"
paths:
- cache/
before_script:
# CCache Config
- mkdir -p cache
- export CCACHE_BASEDIR=${PWD}
- export CCACHE_DIR=${PWD}/cache
- echo $CCACHE_DIR
- export CC="ccache gcc"
after_script:
# somehow after_script loses environment
- export CCACHE_BASEDIR=${PWD}
- export CCACHE_DIR=${PWD}/cache
- ccache -s
variables:
BUILD_IMAGES_PROJECT: gnuwget/build-images
DEBIAN_BUILD: buildenv-debian-stretch
FEDORA_BUILD: buildenv-f25
FEDORA_BUILD: buildenv-f27
CENTOS7_BUILD: buildenv-centos7
MINGW_BUILD: buildenv-mingw
ALPINE_BUILD: buildenv-alpine
ARCH_BUILD: buildenv-arch
BASIC_BUILD: buildenv-basic
GET_SOURCES_ATTEMPTS: "3"
GIT_DEPTH: "5"
CONFIGURE_BASE_FLAGS: --enable-assert --cache-file cache/config.cache
CFLAGS_DEFAULT: -O0 -g -ggdb3 -Wall -Wextra
VPATH/Debian:
image: $CI_REGISTRY/$BUILD_IMAGES_PROJECT:$DEBIAN_BUILD
script:
- cp -a /builds/common/gnulib .
- export CFLAGS=$CFLAGS_DEFAULT
- ./bootstrap --skip-po
- mkdir vpath && cd vpath
- ../configure $CONFIGURE_BASE_FLAGS --cache-file ../cache/config.cache
- make -j$(nproc)
- make -j$(nproc) distcheck
tags:
- shared
- docker
artifacts:
expire_in: 2 weeks
when: on_failure
paths:
- ./*.log
- fuzz/*.log
- tests/*.log
- testenv/*.log
# In this build we combine
# * syntax-check
@@ -11,40 +65,51 @@ variables:
# * build/valgrind-check
# * build/asan-check
# * build w/Werror and ubsan-check
Debian GNU/Linux build:
OpenSSL/Debian:
image: $CI_REGISTRY/$BUILD_IMAGES_PROJECT:$DEBIAN_BUILD
script:
- echo "127.0.0.1 wgettestingserver" >>/etc/hosts
- cat /etc/hosts
- touch .manywarnings
- alias make="make -j$(nproc)"
- ./bootstrap
- ./configure -C
- ./bootstrap --skip-po
- ./configure $CONFIGURE_BASE_FLAGS --with-ssl=gnutls
- make syntax-check
- make check
# - make abi-check
- make distcheck
- make check-valgrind
- ./configure -C --with-ssl=openssl
- make clean
- make check
# - ./configure -C --enable-fsanitize-asan
# - make clean
# - make check
# - ./configure -C --enable-fsanitize-ubsan
# - make clean
# - make check
# - ./configure -C --enable-fsanitize-msan
# - make clean
# - make check
tags:
- shared
artifacts:
expire_in: 2 weeks
when: on_failure
paths:
- ./*.log
- fuzz/*.log
- tests/*.log
# - compat_reports/
- testenv/*.log
GnuTLS/Debian:
image: $CI_REGISTRY/$BUILD_IMAGES_PROJECT:$DEBIAN_BUILD
script:
- echo "127.0.0.1 wgettestingserver" >>/etc/hosts
- cat /etc/hosts
- alias make="make -j$(nproc)"
- ./bootstrap --skip-po
- ./configure $CONFIGURE_BASE_FLAGS --with-ssl=openssl
- make syntax-check
- make check
- make distcheck
- make check-valgrind
tags:
- shared
artifacts:
expire_in: 2 weeks
when: on_failure
paths:
- ./*.log
- fuzz/*.log
- tests/*.log
- testenv/*.log
#Centos7 build:
# image: $CI_REGISTRY/$BUILD_IMAGES_PROJECT:$CENTOS7_BUILD

2
.gitmodules vendored
View File

@@ -1,3 +1,3 @@
[submodule "gnulib"]
path = gnulib
url = git://git.sv.gnu.org/gnulib.git
url = https://git.savannah.gnu.org/git/gnulib.git

7
.lgtm.yml Normal file
View File

@@ -0,0 +1,7 @@
extraction:
cpp:
prepare:
packages:
- libgpgme11-dev
configure:
command: ./bootstrap && ./configure

View File

@@ -1,7 +1,7 @@
GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
@@ -645,7 +645,7 @@ the "copyright" line and a pointer to where the full notice is found.
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
along with this program. If not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
@@ -664,11 +664,11 @@ might be different; for a GUI interface, you would use an "about box".
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU GPL, see
<http://www.gnu.org/licenses/>.
<https://www.gnu.org/licenses/>.
The GNU General Public License does not permit incorporating your program
into proprietary programs. If your program is a subroutine library, you
may consider it more useful to permit linking proprietary applications with
the library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License. But first, please read
<http://www.gnu.org/philosophy/why-not-lgpl.html>.
<https://www.gnu.org/philosophy/why-not-lgpl.html>.

View File

@@ -7,21 +7,14 @@ Primary List
The primary mailinglist for discussion, bug-reports, or questions about
GNU Wget is at <bug-wget@gnu.org>. To subscribe, send an email to
<bug-wget-join@gnu.org>, or visit
`http://lists.gnu.org/mailman/listinfo/bug-wget'.
`https://lists.gnu.org/mailman/listinfo/bug-wget'.
You do not need to subscribe to send a message to the list; however,
please note that unsubscribed messages are moderated, and may take a
while before they hit the list--*usually around a day*. If you want
your message to show up immediately, please subscribe to the list
before posting. Archives for the list may be found at
`http://lists.gnu.org/pipermail/bug-wget/'.
An NNTP/Usenettish gateway is also available via Gmane
(http://gmane.org/about.php). You can see the Gmane archives at
`http://news.gmane.org/gmane.comp.web.wget.general'. Note that the
Gmane archives conveniently include messages from both the current
list, and the previous one. Messages also show up in the Gmane archives
sooner than they do at `lists.gnu.org'.
`https://lists.gnu.org/pipermail/bug-wget/'.
Obsolete Lists
--------------
@@ -31,10 +24,7 @@ discussion list, and another list, <wget-patches@sunsite.dk> was used
for submitting and discussing patches to GNU Wget.
Messages from <wget@sunsite.dk> are archived at
`http://www.mail-archive.com/wget%40sunsite.dk/' and at
`http://news.gmane.org/gmane.comp.web.wget.general' (which also
continues to archive the current list, <bug-wget@gnu.org>).
`https://www.mail-archive.com/wget%40sunsite.dk/' and at
Messages from <wget-patches@sunsite.dk> are archived at
`http://news.gmane.org/gmane.comp.web.wget.patches'.
`https://news.gmane.org/gmane.comp.web.wget.patches'.

View File

@@ -1,6 +1,6 @@
# Makefile for `Wget' utility
# Copyright (C) 1995, 1996, 1997, 2006, 2007, 2008, 2009, 2010, 2011
# Free Software Foundation, Inc.
# Copyright (C) 1995-1997, 2006-2018 Free Software Foundation,
# Inc.
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -13,7 +13,7 @@
# GNU General Public License for more details.
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# along with this program. If not, see <https://www.gnu.org/licenses/>.
# Additional permission under GNU GPL version 3 section 7
@@ -41,7 +41,7 @@ distuninstallcheck_listfiles = find . -type f | \
ACLOCAL_AMFLAGS = -I m4
# subdirectories in the distribution
SUBDIRS = lib src doc po tests util testenv
SUBDIRS = lib src doc po util fuzz tests testenv
EXTRA_DIST = MAILING-LIST \
msdos/config.h msdos/Makefile.DJ \
@@ -49,6 +49,7 @@ EXTRA_DIST = MAILING-LIST \
build-aux/build_info.pl build-aux/git-version-gen .version
CLEANFILES = *~ *.bak $(DISTNAME).tar.gz
DISTCLEANFILES = po/stamp-po
BUILT_SOURCES = .version

65
NEWS
View File

@@ -1,10 +1,73 @@
GNU Wget NEWS -- history of user-visible changes.
Copyright (C) 1997-2017 Free Software Foundation, Inc.
Copyright (C) 1997-2018 Free Software Foundation, Inc.
See the end for copying conditions.
Please send GNU Wget bug reports to <bug-wget@gnu.org>.
* Changes in Wget 1.20
** Add new option `--retry-on-host-error` to treat local errors as transient
and hence Wget will retry to download the file after a brief waiting period.
** Fixed multiple potential resource leaks as found by static analysis
** Wget will now not create an empty wget-log file when running with -q and -b
switches together
** When compiled using the GnuTLS >= 3.6.3, Wget now has support for TLSv1.3
** Now there is support for using libpcre2 for regex pattern matching
** When downloading over FTP recursively, one can now use the
--{accept,reject}-regex switches to fine-tune the downloaded files
** Building Wget from the git sources now requires autoconf 2.63 or above.
Building from the Tarballs works as it used to.
* Changes in Wget 1.19.5
* Fix cookie injection (CVE-2018-0494)
* Enable TLS1.3 with recent OpenSSL environment
* New option --ciphers to set GnuTLS / OpenSSL ciphers directly
* Updated CSS grammar to CSS 2.2
* Fixed several memleaks found by OSS-Fuzz
* Fixed several buffer overflows found by OSS-Fuzz
* Fixed several integer overflows found by OSS-Fuzz
* Several minor bug fixes
* Changes in Wget 1.19.4
* A major bug that caused GZip'ed pages to never be decompressed has been fixed
* Support for Content-Encoding and Transfer-Encoding have been marked as
experimental and disabled by default
* Changes in Wget 1.19.3
* Prevent erroneous decompression of .gz and .tgz files with broken servers
* Added support for HTTP 308 Permanent Redirect response
* Fix a segfault in some cases where the Content-Type header is not sent
* Support OpenSSL 1.1 builds without using deprecated features
* Fix netrc file detection on Windows
* Several minor bug fixes
* Changes in Wget 1.19.2
* Fix CVE-2017-13089 (Stack overflow in HTTP protocol handling)

6
README
View File

@@ -1,7 +1,7 @@
-*- text -*-
GNU Wget
========
Current Web home: http://www.gnu.org/software/wget/
Current Web home: https://www.gnu.org/software/wget/
GNU Wget is a free utility for non-interactive download of files from
the Web. It supports HTTP, HTTPS, and FTP protocols, as well as
@@ -50,10 +50,10 @@ resides at <ftp://ftp.gnu.org/pub/gnu/wget/>.
Please report bugs in Wget to <bug-wget@gnu.org>.
See the file `MAILING-LIST' for information about Wget mailing lists.
Wget's home page is at <http://www.gnu.org/software/wget/>.
Wget's home page is at <https://www.gnu.org/software/wget/>.
If you would like to contribute code for Wget, please read
http://wget.addictivecode.org/PatchGuidelines.
https://wget.addictivecode.org/PatchGuidelines.
Wget was originally written and mainained by Hrvoje Niksic. Please see
the file AUTHORS for a list of major contributors, and the ChangeLogs

View File

@@ -161,7 +161,8 @@ Testing and development
that bug, which might be tedious. Tests should only be run through GDB for that purpose.
Copyright (C) 2008,2010,2014,2015 Free Software Foundation, Inc.
Copyright (C) 2008, 2010, 2014-2015, 2018 Free Software Foundation,
Inc.
License GPLv3+: GNU GPL version 3 or later
<https://www.gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
@@ -202,7 +203,7 @@ References
32. https://www.gnu.org/software/libiconv/
33. https://git-scm.com/
34. https://github.com/rockdaboot/libpsl
35. http://www.pcre.org
35. https://www.pcre.org
36. http://wget.addictivecode.org/CompilingRepoSources?action=edit&editor=text
37. http://wget.addictivecode.org/CompilingRepoSources
38. http://wget.addictivecode.org/CompilingRepoSources?action=info

331
bootstrap
View File

@@ -1,10 +1,10 @@
#! /bin/sh
# Print a version string.
scriptversion=2016-01-24.06; # UTC
scriptversion=2018-10-13.05; # UTC
# Bootstrap this package from checked-out sources.
# Copyright (C) 2003-2016 Free Software Foundation, Inc.
# Copyright (C) 2003-2018 Free Software Foundation, Inc.
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -17,7 +17,7 @@ scriptversion=2016-01-24.06; # UTC
# GNU General Public License for more details.
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# along with this program. If not, see <https://www.gnu.org/licenses/>.
# Originally written by Paul Eggert. The canonical version of this
# script is maintained as build-aux/bootstrap in gnulib, however, to
@@ -47,6 +47,8 @@ PERL="${PERL-perl}"
me=$0
default_gnulib_url=git://git.sv.gnu.org/gnulib
usage() {
cat <<EOF
Usage: $me [OPTION]...
@@ -76,6 +78,37 @@ contents are read as shell variables to configure the bootstrap.
For build prerequisites, environment variables like \$AUTOCONF and \$AMTAR
are honored.
Gnulib sources can be fetched in various ways:
* If this package is in a git repository with a 'gnulib' submodule
configured, then that submodule is initialized and updated and sources
are fetched from there. If \$GNULIB_SRCDIR is set (directly or via
--gnulib-srcdir) and is a git repository, then it is used as a reference.
* Otherwise, if \$GNULIB_SRCDIR is set (directly or via --gnulib-srcdir),
then sources are fetched from that local directory. If it is a git
repository and \$GNULIB_REVISION is set, then that revision is checked
out.
* Otherwise, if this package is in a git repository with a 'gnulib'
submodule configured, then that submodule is initialized and updated and
sources are fetched from there.
* Otherwise, if the 'gnulib' directory does not exist, Gnulib sources are
cloned into that directory using git from \$GNULIB_URL, defaulting to
$default_gnulib_url.
If \$GNULIB_REVISION is set, then that revision is checked out.
* Otherwise, the existing Gnulib sources in the 'gnulib' directory are
used. If it is a git repository and \$GNULIB_REVISION is set, then that
revision is checked out.
If you maintain a package and want to pin a particular revision of the
Gnulib sources that has been tested with your package, then there are two
possible approaches: either configure a 'gnulib' submodule with the
appropriate revision, or set \$GNULIB_REVISION (and if necessary
\$GNULIB_URL) in $me.conf.
Running without arguments will suffice in most cases.
EOF
}
@@ -129,19 +162,12 @@ bootstrap_post_import_hook() { :; }
# Override it via your own definition in bootstrap.conf.
bootstrap_epilogue() { :; }
# The command to download all .po files for a specified domain into
# a specified directory. Fill in the first %s is the domain name, and
# the second with the destination directory. Use rsync's -L and -r
# options because the latest/%s directory and the .po files within are
# all symlinks.
# The command to download all .po files for a specified domain into a
# specified directory. Fill in the first %s with the destination
# directory and the second with the domain name.
po_download_command_format=\
"rsync --delete --exclude '*.s1' -Lrtvz \
'translationproject.org::tp/latest/%s/' '%s'"
# Fallback for downloading .po files (if rsync fails).
po_download_command_format2=\
"wget --mirror -nd -q -np -A.po -P '%s' \
http://translationproject.org/latest/%s/"
"wget --mirror --level=1 -nd -q -A.po -P '%s' \
https://translationproject.org/latest/%s/"
# Prefer a non-empty tarname (4th argument of AC_INIT if given), else
# fall back to the package name (1st argument with munging)
@@ -170,7 +196,15 @@ source_base=lib
m4_base=m4
doc_base=doc
tests_base=tests
gnulib_extra_files=''
gnulib_extra_files="
build-aux/install-sh
build-aux/mdate-sh
build-aux/texinfo.tex
build-aux/depcomp
build-aux/config.guess
build-aux/config.sub
doc/INSTALL
"
# Additional gnulib-tool options to use. Use "\newline" to break lines.
gnulib_tool_option_extras=
@@ -264,24 +298,18 @@ case "$0" in
*) test -r "$0.conf" && . ./"$0.conf" ;;
esac
# Extra files from gnulib, which override files from other sources.
test -z "${gnulib_extra_files}" && \
gnulib_extra_files="
build-aux/install-sh
build-aux/mdate-sh
build-aux/texinfo.tex
build-aux/depcomp
build-aux/config.guess
build-aux/config.sub
doc/INSTALL
"
if test "$vc_ignore" = auto; then
vc_ignore=
test -d .git && vc_ignore=.gitignore
test -d CVS && vc_ignore="$vc_ignore .cvsignore"
fi
if test x"$gnulib_modules$gnulib_files$gnulib_extra_files" = x; then
use_gnulib=false
else
use_gnulib=true
fi
# Translate configuration into internal form.
# Parse options.
@@ -418,28 +446,30 @@ sort_ver() { # sort -V is not generally available
done
}
get_version_sed='
# Move version to start of line.
s/.*[v ]\([0-9]\)/\1/
# Skip lines that do not start with version.
/^[0-9]/!d
# Remove characters after the version.
s/[^.a-z0-9-].*//
# The first component must be digits only.
s/^\([0-9]*\)[a-z-].*/\1/
#the following essentially does s/5.005/5.5/
s/\.0*\([1-9]\)/.\1/g
p
q'
get_version() {
app=$1
$app --version >/dev/null 2>&1 || { $app --version; return 1; }
$app --version 2>&1 |
sed -n '# Move version to start of line.
s/.*[v ]\([0-9]\)/\1/
# Skip lines that do not start with version.
/^[0-9]/!d
# Remove characters after the version.
s/[^.a-z0-9-].*//
# The first component must be digits only.
s/^\([0-9]*\)[a-z-].*/\1/
#the following essentially does s/5.005/5.5/
s/\.0*\([1-9]\)/.\1/g
p
q'
$app --version 2>&1 | sed -n "$get_version_sed"
}
check_versions() {
@@ -610,95 +640,101 @@ git_modules_config () {
test -f .gitmodules && git config --file .gitmodules "$@"
}
if $use_git; then
gnulib_path=$(git_modules_config submodule.gnulib.path)
test -z "$gnulib_path" && gnulib_path=gnulib
fi
# Get gnulib files. Populate $GNULIB_SRCDIR, possibly updating a
# submodule, for use in the rest of the script.
case ${GNULIB_SRCDIR--} in
-)
# Note that $use_git is necessarily true in this case.
if git_modules_config submodule.gnulib.url >/dev/null; then
echo "$0: getting gnulib files..."
git submodule init -- "$gnulib_path" || exit $?
git submodule update -- "$gnulib_path" || exit $?
elif [ ! -d "$gnulib_path" ]; then
echo "$0: getting gnulib files..."
trap cleanup_gnulib 1 2 13 15
shallow=
git clone -h 2>&1 | grep -- --depth > /dev/null && shallow='--depth 2'
git clone $shallow git://git.sv.gnu.org/gnulib "$gnulib_path" ||
cleanup_gnulib
trap - 1 2 13 15
if $use_gnulib; then
if $use_git; then
gnulib_path=$(git_modules_config submodule.gnulib.path)
test -z "$gnulib_path" && gnulib_path=gnulib
fi
GNULIB_SRCDIR=$gnulib_path
;;
*)
# Use GNULIB_SRCDIR directly or as a reference.
if $use_git && test -d "$GNULIB_SRCDIR"/.git && \
git_modules_config submodule.gnulib.url >/dev/null; then
echo "$0: getting gnulib files..."
if git submodule -h|grep -- --reference > /dev/null; then
# Prefer the one-liner available in git 1.6.4 or newer.
git submodule update --init --reference "$GNULIB_SRCDIR" \
"$gnulib_path" || exit $?
else
# This fallback allows at least git 1.5.5.
if test -f "$gnulib_path"/gnulib-tool; then
# Since file already exists, assume submodule init already complete.
git submodule update -- "$gnulib_path" || exit $?
else
# Older git can't clone into an empty directory.
rmdir "$gnulib_path" 2>/dev/null
git clone --reference "$GNULIB_SRCDIR" \
"$(git_modules_config submodule.gnulib.url)" "$gnulib_path" \
&& git submodule init -- "$gnulib_path" \
&& git submodule update -- "$gnulib_path" \
|| exit $?
# Get gnulib files. Populate $GNULIB_SRCDIR, possibly updating a
# submodule, for use in the rest of the script.
case ${GNULIB_SRCDIR--} in
-)
# Note that $use_git is necessarily true in this case.
if git_modules_config submodule.gnulib.url >/dev/null; then
echo "$0: getting gnulib files..."
git submodule init -- "$gnulib_path" || exit $?
git submodule update -- "$gnulib_path" || exit $?
elif [ ! -d "$gnulib_path" ]; then
echo "$0: getting gnulib files..."
trap cleanup_gnulib 1 2 13 15
shallow=
if test -z "$GNULIB_REVISION"; then
git clone -h 2>&1 | grep -- --depth > /dev/null && shallow='--depth 2'
fi
git clone $shallow ${GNULIB_URL:-$default_gnulib_url} "$gnulib_path" \
|| cleanup_gnulib
trap - 1 2 13 15
fi
GNULIB_SRCDIR=$gnulib_path
;;
*)
# Use GNULIB_SRCDIR directly or as a reference.
if $use_git && test -d "$GNULIB_SRCDIR"/.git && \
git_modules_config submodule.gnulib.url >/dev/null; then
echo "$0: getting gnulib files..."
if git submodule -h|grep -- --reference > /dev/null; then
# Prefer the one-liner available in git 1.6.4 or newer.
git submodule update --init --reference "$GNULIB_SRCDIR" \
"$gnulib_path" || exit $?
else
# This fallback allows at least git 1.5.5.
if test -f "$gnulib_path"/gnulib-tool; then
# Since file already exists, assume submodule init already complete.
git submodule update -- "$gnulib_path" || exit $?
else
# Older git can't clone into an empty directory.
rmdir "$gnulib_path" 2>/dev/null
git clone --reference "$GNULIB_SRCDIR" \
"$(git_modules_config submodule.gnulib.url)" "$gnulib_path" \
&& git submodule init -- "$gnulib_path" \
&& git submodule update -- "$gnulib_path" \
|| exit $?
fi
fi
GNULIB_SRCDIR=$gnulib_path
fi
;;
esac
if test -d "$GNULIB_SRCDIR"/.git && test -n "$GNULIB_REVISION" \
&& ! git_modules_config submodule.gnulib.url >/dev/null; then
(cd "$GNULIB_SRCDIR" && git checkout "$GNULIB_REVISION") || cleanup_gnulib
fi
;;
esac
# $GNULIB_SRCDIR now points to the version of gnulib to use, and
# we no longer need to use git or $gnulib_path below here.
# $GNULIB_SRCDIR now points to the version of gnulib to use, and
# we no longer need to use git or $gnulib_path below here.
if $bootstrap_sync; then
cmp -s "$0" "$GNULIB_SRCDIR/build-aux/bootstrap" || {
echo "$0: updating bootstrap and restarting..."
case $(sh -c 'echo "$1"' -- a) in
a) ignored=--;;
*) ignored=ignored;;
esac
exec sh -c \
'cp "$1" "$2" && shift && exec "${CONFIG_SHELL-/bin/sh}" "$@"' \
$ignored "$GNULIB_SRCDIR/build-aux/bootstrap" \
"$0" "$@" --no-bootstrap-sync
}
if $bootstrap_sync; then
cmp -s "$0" "$GNULIB_SRCDIR/build-aux/bootstrap" || {
echo "$0: updating bootstrap and restarting..."
case $(sh -c 'echo "$1"' -- a) in
a) ignored=--;;
*) ignored=ignored;;
esac
exec sh -c \
'cp "$1" "$2" && shift && exec "${CONFIG_SHELL-/bin/sh}" "$@"' \
$ignored "$GNULIB_SRCDIR/build-aux/bootstrap" \
"$0" "$@" --no-bootstrap-sync
}
fi
gnulib_tool=$GNULIB_SRCDIR/gnulib-tool
<$gnulib_tool || exit $?
fi
gnulib_tool=$GNULIB_SRCDIR/gnulib-tool
<$gnulib_tool || exit $?
# Get translations.
download_po_files() {
subdir=$1
domain=$2
echo "$me: getting translations into $subdir for $domain..."
cmd=$(printf "$po_download_command_format" "$domain" "$subdir")
eval "$cmd" && return
# Fallback to HTTP.
cmd=$(printf "$po_download_command_format2" "$subdir" "$domain")
cmd=$(printf "$po_download_command_format" "$subdir" "$domain")
eval "$cmd"
}
@@ -788,9 +824,9 @@ symlink_to_dir()
# Leave any existing symlink alone, if it already points to the source,
# so that broken build tools that care about symlink times
# aren't confused into doing unnecessary builds. Conversely, if the
# existing symlink's time stamp is older than the source, make it afresh,
# existing symlink's timestamp is older than the source, make it afresh,
# so that broken tools aren't confused into skipping needed builds. See
# <http://lists.gnu.org/archive/html/bug-gnulib/2011-05/msg00326.html>.
# <https://lists.gnu.org/r/bug-gnulib/2011-05/msg00326.html>.
test -h "$dst" &&
src_ls=$(ls -diL "$src" 2>/dev/null) && set $src_ls && src_i=$1 &&
dst_ls=$(ls -diL "$dst" 2>/dev/null) && set $dst_ls && dst_i=$1 &&
@@ -896,32 +932,33 @@ fi
# Import from gnulib.
gnulib_tool_options="\
--import\
--no-changelog\
--aux-dir $build_aux\
--doc-base $doc_base\
--lib $gnulib_name\
--m4-base $m4_base/\
--source-base $source_base/\
--tests-base $tests_base\
--local-dir $local_gl_dir\
$gnulib_tool_option_extras\
"
if test $use_libtool = 1; then
case "$gnulib_tool_options " in
*' --libtool '*) ;;
*) gnulib_tool_options="$gnulib_tool_options --libtool" ;;
esac
fi
echo "$0: $gnulib_tool $gnulib_tool_options --import ..."
$gnulib_tool $gnulib_tool_options --import $gnulib_modules \
|| die "gnulib-tool failed"
if $use_gnulib; then
gnulib_tool_options="\
--no-changelog\
--aux-dir=$build_aux\
--doc-base=$doc_base\
--lib=$gnulib_name\
--m4-base=$m4_base/\
--source-base=$source_base/\
--tests-base=$tests_base\
--local-dir=$local_gl_dir\
$gnulib_tool_option_extras\
"
if test $use_libtool = 1; then
case "$gnulib_tool_options " in
*' --libtool '*) ;;
*) gnulib_tool_options="$gnulib_tool_options --libtool" ;;
esac
fi
echo "$0: $gnulib_tool $gnulib_tool_options --import ..."
$gnulib_tool $gnulib_tool_options --import $gnulib_modules \
|| die "gnulib-tool failed"
for file in $gnulib_files; do
symlink_to_dir "$GNULIB_SRCDIR" $file \
|| die "failed to symlink $file"
done
for file in $gnulib_files; do
symlink_to_dir "$GNULIB_SRCDIR" $file \
|| die "failed to symlink $file"
done
fi
bootstrap_post_import_hook \
|| die "bootstrap_post_import_hook failed"
@@ -1018,7 +1055,7 @@ bootstrap_epilogue
echo "$0: done. Now you can run './configure'."
# Local variables:
# eval: (add-hook 'write-file-hooks 'time-stamp)
# eval: (add-hook 'before-save-hook 'time-stamp)
# time-stamp-start: "scriptversion="
# time-stamp-format: "%:y-%02m-%02d.%02H"
# time-stamp-time-zone: "UTC0"

View File

@@ -1,6 +1,5 @@
# bootstrap.conf - Bootstrap configuration.
# Copyright (C) 2007, 2008, 2009, 2010, 2011, 2012 Free Software Foundation,
# Inc.
# Copyright (C) 2007-2012, 2018 Free Software Foundation, Inc.
#
# This file is part of GNU Wget.
#
@@ -9,13 +8,13 @@
# the Free Software Foundation, either version 3 of the License, or (at
# your option) any later version.
#
# GNU Wgt is distributed in the hope that it will be useful, but
# GNU Wget is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see `http://www.gnu.org/licenses/'.
# along with this program. If not, see `https://www.gnu.org/licenses/'.
# gnulib library name
gnulib_name=libgnu
@@ -49,7 +48,7 @@ getopt-gnu
getpass-gnu
getpeername
getsockname
gettext
gettext-h
gitlog-to-changelog
git-version-gen
gnupload
@@ -103,6 +102,7 @@ strerror_r-posix
strpbrk
strptime
strtok_r
strtol
strtoll
symlink
sys_types
@@ -114,6 +114,8 @@ update-copyright
libunistring-optional
unistr/u8-strlen
unicase/u8-tolower
utime
utime-h
utimens
vasprintf
vsnprintf

View File

@@ -2,7 +2,7 @@
# Generate build_info.c.
# Copyright (C) 2009, 2010, 2011 Free Software Foundation, Inc.
# Copyright (C) 2009-2011, 2018 Free Software Foundation, Inc.
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by

6
cfg.mk
View File

@@ -1,5 +1,9 @@
# Explicit syntax-check exceptions.
exclude_file_name_regexp--sc_trailing_blank = ^doc/annou.*$$
exclude_file_name_regexp--sc_bindtextdomain = ^fuzz/.*.c$$
exclude_file_name_regexp--sc_require_config_h = fuzz/main.c$$
exclude_file_name_regexp--sc_require_config_h_first = fuzz/main.c$$
export VC_LIST_EXCEPT_DEFAULT=^(lib/.*|m4/.*|md5/.*|build-aux/.*|src/gettext\.h|.*ChangeLog|tests/certs/.*)$$
@@ -30,4 +34,4 @@ local-checks-to-skip = \
show-vc-list-except:
@$(VC_LIST_EXCEPT)
VC_LIST_ALWAYS_EXCLUDE_REGEX = ^ABOUT-NLS|((msdos|vms)/.*)|ChangeLog-2014-12-10|/.*.der|^tests/certs/.*$$
VC_LIST_ALWAYS_EXCLUDE_REGEX = ^ABOUT-NLS|((msdos|vms)/.*)|ChangeLog-2014-12-10|/.*\.der|^tests/certs/.*|^fuzz/.*\.(in|repro)/.*$$

View File

@@ -1,6 +1,6 @@
dnl Template file for GNU Autoconf
dnl Copyright (C) 1995, 1996, 1997, 2001, 2007, 2008, 2009, 2010, 2011, 2012,
dnl 2013, 2014 Free Software Foundation, Inc.
dnl Copyright (C) 1995-1997, 2001, 2007-2014, 2018 Free Software
dnl Foundation, Inc.
dnl This program is free software; you can redistribute it and/or modify
dnl it under the terms of the GNU General Public License as published by
@@ -33,7 +33,7 @@ dnl
AC_INIT([wget],
m4_esyscmd([build-aux/git-version-gen .tarball-version]),
[bug-wget@gnu.org])
AC_PREREQ(2.61)
AC_PREREQ(2.63)
dnl
dnl What version of Wget are we building?
@@ -87,6 +87,22 @@ dnl
dnl Process features
dnl
AC_ARG_ENABLE([fuzzing],
[AS_HELP_STRING([--enable-fuzzing], [Turn on fuzzing build (for developers)])],
[enable_fuzzing=yes;
AC_SUBST([LIB_FUZZING_ENGINE])
AC_DEFINE([FUZZING], 1, [Define to 1 if this is a fuzzing build])
], [enable_fuzzing=no; LIB_FUZZING_ENGINE=""])
#FUZZ_LIBS=$LIBS
#if test $enable_fuzzing = "yes"; then
# OLD_LIBS=$LIBS
# AC_SEARCH_LIBS([dlsym], [dl dld])
# FUZZ_LIBS=$LIBS
# LIBS=$OLD_LIBS
#fi
#AC_SUBST([FUZZ_LIBS])
AM_CONDITIONAL([FUZZING], [test "$enable_fuzzing" = "yes"])
dnl Opie: Support for opie s/key FTP logins
AC_ARG_ENABLE([opie],
[AS_HELP_STRING([--disable-opie], [disable support for opie or s/key FTP login])],
@@ -184,7 +200,7 @@ dnl
dnl Gettext
dnl
AM_GNU_GETTEXT([external],[need-ngettext])
AM_GNU_GETTEXT_VERSION([0.18.1])
AM_GNU_GETTEXT_VERSION([0.17])
AC_PROG_RANLIB
@@ -237,7 +253,7 @@ dnl
AC_HEADER_STDBOOL
AC_CHECK_HEADERS(unistd.h sys/time.h)
AC_CHECK_HEADERS(termios.h sys/ioctl.h sys/select.h utime.h sys/utime.h)
AC_CHECK_HEADERS(stdint.h inttypes.h pwd.h wchar.h)
AC_CHECK_HEADERS(stdint.h inttypes.h pwd.h wchar.h dlfcn.h)
AC_CHECK_DECLS(h_errno,,,[#include <netdb.h>])
@@ -275,7 +291,7 @@ AC_FUNC_MMAP
AC_FUNC_FSEEKO
AC_CHECK_FUNCS(strptime timegm vsnprintf vasprintf drand48 pathconf)
AC_CHECK_FUNCS(strtoll usleep ftello sigblock sigsetjmp memrchr wcwidth mbtowc)
AC_CHECK_FUNCS(sleep symlink utime strlcpy random)
AC_CHECK_FUNCS(sleep symlink utime strlcpy random fmemopen)
if test x"$ENABLE_OPIE" = xyes; then
AC_LIBOBJ([ftp-opie])
@@ -477,35 +493,6 @@ else
fi
fi
dnl
dnl Check for libmetalink
dnl
AS_IF([test x"$with_metalink" != xno], [
PKG_CHECK_MODULES([METALINK], libmetalink, [
LIBS="$METALINK_LIBS $LIBS"
CFLAGS="$METALINK_CFLAGS $CFLAGS"
AC_DEFINE([HAVE_METALINK], [1], [Define if using metalink.])
with_metalink=yes
], [
with_metalink=no
])
have_gpg=no
AS_IF([test x"$with_metalink" = xyes], [
dnl
dnl Check for GPGME
dnl
m4_ifdef([AM_PATH_GPGME], [
AM_PATH_GPGME([], [
LIBS="$GPGME_LIBS $LIBS"
CFLAGS="$GPGME_CFLAGS $CFLAGS"
AC_DEFINE([HAVE_GPGME], [1], [Define if GPGME is available.])
have_gpg=yes
])
])
])
])
dnl **********************************************************************
dnl Checks for IPv6
dnl **********************************************************************
@@ -699,18 +686,39 @@ AS_IF([test "x$uuid_mode" = x1], [
])
dnl
dnl Check for PCRE
dnl Check for PCRE2 / PCRE
dnl
AC_ARG_ENABLE(pcre2, AC_HELP_STRING([--disable-pcre2],
[Disable PCRE2 style regular expressions]))
AC_ARG_ENABLE(pcre, AC_HELP_STRING([--disable-pcre],
[Disable PCRE style regular expressions]))
AS_IF([test "X$enable_pcre" != "Xno"],[
AS_IF([test "X$enable_pcre2" != "Xno"],[
enable_pcre2=no
PKG_CHECK_MODULES([PCRE2], libpcre2-8, [
CFLAGS="$PCRE2_CFLAGS $CFLAGS"
LIBS="$PCRE2_LIBS $LIBS"
AC_DEFINE([HAVE_LIBPCRE2], [1], [Define if libpcre2 is available.])
enable_pcre2=yes
], [
AC_CHECK_HEADER(pcre2.h, [
AC_CHECK_LIB(pcre2-8, pcre2_compile_8, [
LIBS="${LIBS} -lpcre2-8"
AC_DEFINE([HAVE_LIBPCRE2], 1, [Define if libpcre2 is available.])
enable_pcre2=yes
])
])
])
])
AS_IF([test "X$enable_pcre" != "Xno" && test "X$enable_pcre2" != "Xyes"],[
PKG_CHECK_MODULES([PCRE], libpcre, [
CFLAGS="$PCRE_CFLAGS $CFLAGS"
AC_CHECK_HEADER(pcre.h, [
LIBS="$PCRE_LIBS $LIBS"
AC_DEFINE([HAVE_LIBPCRE], [1], [Define if libpcre is available.])
enable_pcre=yes
])
], [
AC_CHECK_HEADER(pcre.h, [
@@ -722,6 +730,8 @@ AS_IF([test "X$enable_pcre" != "Xno"],[
])
])
AS_IF([test "X$enable_pcre2" = Xyes], [PCRE_INFO="yes, via libpcre2"], [test "X$enable_pcre" = Xyes], [PCRE_INFO="yes, via libpcre"], [PCRE_INFO=no])
dnl
dnl Check for libcares (resolver library)
dnl
@@ -747,6 +757,38 @@ AS_IF([test "X$with_cares" = "Xyes"],[
RESOLVER_INFO="libc, --bind-dns-address and --dns-servers not available"
])
dnl
dnl Check for libmetalink
dnl
AS_IF([test x"$with_metalink" != xno], [
PKG_CHECK_MODULES([METALINK], libmetalink, [
LIBS="$METALINK_LIBS $LIBS"
CFLAGS="$METALINK_CFLAGS $CFLAGS"
AC_DEFINE([HAVE_METALINK], [1], [Define if using metalink.])
with_metalink=yes
], [
with_metalink=no
])
have_gpg=no
AS_IF([test x"$with_metalink" = xyes], [
dnl
dnl Check for GPGME
dnl
m4_ifdef([AM_PATH_GPGME], [
AM_PATH_GPGME([], [
# Put libgpgme to the end of the library list since it introduces a -L linker flags.
# That -L might break the build if there are two different version of
# a library (e.g. GnuTLS) in /usr/local and in the system directory.
LIBS="$LIBS $GPGME_LIBS"
CFLAGS="$GPGME_CFLAGS $CFLAGS"
AC_DEFINE([HAVE_GPGME], [1], [Define if GPGME is available.])
have_gpg=yes
])
])
])
])
dnl
dnl Extended Attribute support
dnl
@@ -783,7 +825,7 @@ dnl
dnl Create output
dnl
AC_CONFIG_FILES([Makefile src/Makefile doc/Makefile util/Makefile
po/Makefile.in tests/Makefile
po/Makefile.in tests/Makefile fuzz/Makefile
lib/Makefile testenv/Makefile
tests/certs/interca.conf tests/certs/rootca.conf])
AC_CONFIG_HEADERS([src/config.h])
@@ -801,6 +843,7 @@ AC_MSG_NOTICE([Summary of build options:
SSL: $with_ssl
Zlib: $with_zlib
PSL: $with_libpsl
PCRE: $PCRE_INFO
Digest: $ENABLE_DIGEST
NTLM: $ENABLE_NTLM
OPIE: $ENABLE_OPIE
@@ -812,4 +855,5 @@ AC_MSG_NOTICE([Summary of build options:
Resolver: $RESOLVER_INFO
GPGME: $have_gpg
IRI: $iri
Fuzzing build: $enable_fuzzing, $LIB_FUZZING_ENGINE
])

113
contrib/make-release Executable file
View File

@@ -0,0 +1,113 @@
#!/usr/bin/env sh
# GNU Wget: Make a new release
#
# This is a helper script to make a new release of GNU Wget
#
# Make sure that
set -e
set -o pipefail
set -u
export CFLAGS="-g -O2 -Wall"
EXTENSIONS="gzip lzip"
ask_continue() {
printf "Continue? [y/N]: "
read inp
[ "$inp" != "y" ] && [ "$inp" != "Y" ] && exit 1
# For some completely weird reason, this set +x is required. Else, the test
# *ALWAYS* fails
set +x
}
if [ ! -d ".git" ]; then
echo "Please run this script from the root of the git repository"
fi
echo "This script will make a new release and upload it to the GNU FTP Servers"
echo "Will run git clean -dxf to get a clean dir"
ask_continue
# Make sure we have a clean working directory
git clean -dxfq
echo "=> Bootstrap"
./bootstrap > /dev/null 2>&1
echo "=> Configure"
./configure -q
APP_NAME=$(grep "^PACKAGE_NAME='" configure | cut -d "'" -f2)
APP_VER=$(grep "^PACKAGE_VERSION='" configure | cut -d "'" -f2)
echo "Making release for $APP_NAME $APP_VER"
ask_continue
NEWS_VERSION=$(grep -i "Changes in $APP_NAME" NEWS | head -1 | awk '{print $5}')
if [ "$NEWS_VERSION" != "$APP_VER" ]; then
echo "Latest version in NEWS file does not match latest tag"
exit 1
fi
CUR_VER=$(git describe | sed 's/^v//')
PREV_VER=$(git describe --abbrev=0 --tags $(git rev-list --tags --skip=1 --max-count=1) | sed 's/^v//')
LNO_CVER=$(grep -in "Changes in $APP_NAME" NEWS | head -1 | cut -f1 -d:)
LNO_PVER=$(grep -in "Changes in $APP_NAME" NEWS | head -2 | tail -1 | cut -f1 -d:)
sed -n "${LNO_CVER},${LNO_PVER}p" NEWS
echo "This is the current contents of your NEWS"
ask_continue
echo "=> Make distcheck"
make -s distcheck > /dev/null 2>&1
for ext in $EXTENSIONS; do
[ "$ext" = 'gz' ] && ext="gzip"
[ "$ext" = 'lz' ] && ext="lzip"
echo "=> Making $APP_NAME-$CUR_VER.tar.$ext"
make -s "dist-$ext" > /dev/null
done
echo ""
REL_TYPE=
REL_LOC=
while [ -z $REL_TYPE ]; do
printf "Enter release type (alpha, beta, stable): "
read x
case $x in
a|A|alpha|Alpha) REL_TYPE=alpha && REL_LOC="alpha.gnu.org";;
b|B|beta|Beta) REL_TYPE=beta && REL_LOC="alpha.gnu.org";;
s|S|stable|Stable) REL_TYPE=stable && REL_LOC="ftp.gnu.org";;
esac
done
GPG_KEY=
while [ -z $GPG_KEY ]; do
printf "Enter GPG Key ID for Signing: "
read key
gpg --list-keys "$key" || continue
printf "Will use this key to sign releases. Continue? [y/N]: "
read x
[ "$x" = "y" ] || [ "$x" = "Y" ] && GPG_KEY="$key"
done
echo ""
APP_LOWER="$(echo "$APP_NAME" | tr '[:upper:]' '[:lower:]')"
./build-aux/gnupload --to "$REL_LOC:$APP_LOWER" --user "$GPG_KEY" --symlink-regex "$APP_LOWER-$CUR_VER".tar.*
echo ""
build-aux/announce-gen --release-type "$REL_TYPE" --package-name "$APP_LOWER" --previous-version="$PREV_VER" --current-version "$CUR_VER" --gpg-key-id="$GPG_KEY" --url-directory "https://${REL_LOC}/gnu/${APP_LOWER}" --news=NEWS > "$APP_NAME-$APP_VER.announce"
echo ""
echo "Announce email template placed at $APP_NAME-$APP_VER.announce"
prev_tag=$(git tag | tail -2 | head -1)
echo ""
echo "Don't forget to mention the following contributors in the announcement:"
(
git log "${prev_tag}.." --format="%aN" --reverse
git log "${prev_tag}.." --reverse | grep -i reported | cut -d':' -f2 | cut -d'<' -f1 | cut -d' ' -f2- | tr -d '"'
) | sort -u

View File

@@ -1,14 +1,28 @@
#!/bin/bash -e
#
# Written 2016 by Tim Ruehsen
#
# To get the version number right: add a signed tag and
# make sure the working dir is clean (git diff).
# If that fails, put the version into '.tarball-version'.
extensions="gz xz"
export CFLAGS="-g -O2 -Wall"
echo
echo "* Bootstrap"
#./bootstrap >/dev/null
echo
echo "* Configure"
#./configure -q
extensions="gz lz"
app_name=$(grep "^PACKAGE_NAME='" configure|cut -d "'" -f2)
app_ver=$(grep "^PACKAGE_VERSION='" configure|cut -d "'" -f2)
echo "Current version of $app_name is $app_ver"
echo
echo -n "Enter Y if that is OK, else enter N and edit configure.ac: "
echo -n "Is that OK ? [y/N] "
read x
[ "$x" != "y" -a "$x" != "Y" ] && exit 0
@@ -19,28 +33,24 @@ read x
[ "$x" != "y" -a "$x" != "Y" ] && exit 0
echo
export CFLAGS="-g -O2 -Wall"
echo "* Bootstrap"
./bootstrap >/dev/null
echo "* Configure"
./configure -q
echo "* Make"
make -s clean
make -s
#make -s clean
#make -s
echo
echo "* Make distcheck"
make -s distcheck >/dev/null
#make -s distcheck >/dev/null
echo
echo "* Make dist"
make -s dist >/dev/null
make -s dist-xz >/dev/null
for ext in $extensions; do
[[ $ext = 'gz' ]] && ext="gzip"
[[ $ext = 'lz' ]] && ext="lzip"
make -s dist-$ext >/dev/null
done
while true; do
echo "Alpha or Regular release ? [r/A] "
echo "Alpha or Regular release ? [R/A] "
read x
if test "$x" = "r" || test "$x" = "R"; then
directory="/incoming/ftp"
@@ -82,10 +92,11 @@ else
done
fi
unset IFS
echo $signing_key
# fetch app_version again, just in case
app_ver=$(grep "^PACKAGE_VERSION='" configure|cut -d "'" -f2)
echo
echo -n "Really upload to ftp-upload.gnu.org/$directory ? "
read x
[ "$x" != "y" -a "$x" != "Y" ] && exit 0
for ext in $extensions; do
app_tarball="${app_name}-${app_ver}.tar.$ext"

View File

@@ -1,6 +1,6 @@
# Makefile for `wget' utility
# Copyright (C) 1995, 1996, 1997, 2007, 2008, 2009, 2010, 2011, 2015
# Free Software Foundation, Inc.
# Copyright (C) 1995-1997, 2007-2011, 2015, 2018 Free Software
# Foundation, Inc.
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by

View File

@@ -5,7 +5,7 @@
@c hence no sectioning command or @node.
@display
Copyright @copyright{} 2000, 2001, 2002, 2007, 2008, 2015 Free Software
Copyright @copyright{} 2000-2002, 2007-2008, 2015, 2018 Free Software
Foundation, Inc.
@uref{http://fsf.org/}

View File

@@ -1,7 +1,7 @@
#! /usr/bin/env perl
# Copyright (C) 1999, 2000, 2001, 2003, 2007, 2009, 2010, 2011, 2015
# Free Software Foundation, Inc.
# Copyright (C) 1999-2001, 2003, 2007, 2009-2011, 2015, 2018 Free
# Software Foundation, Inc.
# This file is part of GCC.

View File

@@ -31,9 +31,8 @@ This file documents the GNU Wget utility for downloading network
data.
@c man begin COPYRIGHT
Copyright @copyright{} 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003,
2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2015 Free Software
Foundation, Inc.
Copyright @copyright{} 1996-2011, 2015, 2018 Free Software Foundation,
Inc.
@iftex
Permission is granted to make and distribute verbatim copies of
@@ -1544,10 +1543,13 @@ If @samp{auto} or @samp{gzip} are specified, Wget asks the server to
compress the file using the gzip compression format. If the server
compresses the file and responds with the @code{Content-Encoding}
header field set appropriately, the file will be decompressed
automatically. This is the default.
automatically.
If @samp{none} is specified, wget will not ask the server to compress
the file and will not decompress any server responses.
the file and will not decompress any server responses. This is the default.
Compression support is currently experimental. In case it is turned on,
please report any bugs to @code{bug-wget@@gnu.org}.
@cindex redirect
@item --max-redirect=@var{number}
@@ -1748,6 +1750,10 @@ some few obscure servers, which never send HTTP authentication
challenges, but accept unsolicited auth info, say, in addition to
form-based authentication.
@item --retry-on-host-error
Consider host errors, such as ``Temporary failure in name resolution'',
as non-fatal, transient errors.
@item --retry-on-http-error=@var{code[,code,...]}
Consider given HTTP response codes as non-fatal, transient errors.
Supply a comma-separated list of 3-digit HTTP response codes as
@@ -1778,25 +1784,34 @@ If Wget is compiled without SSL support, none of these options are available.
@cindex SSL protocol, choose
@item --secure-protocol=@var{protocol}
Choose the secure protocol to be used. Legal values are @samp{auto},
@samp{SSLv2}, @samp{SSLv3}, @samp{TLSv1}, @samp{TLSv1_1}, @samp{TLSv1_2}
and @samp{PFS}. If @samp{auto} is used, the SSL library is given the
liberty of choosing the appropriate protocol automatically, which is
@samp{SSLv2}, @samp{SSLv3}, @samp{TLSv1}, @samp{TLSv1_1}, @samp{TLSv1_2},
@samp{TLSv1_3} and @samp{PFS}. If @samp{auto} is used, the SSL library is
given the liberty of choosing the appropriate protocol automatically, which is
achieved by sending a TLSv1 greeting. This is the default.
Specifying @samp{SSLv2}, @samp{SSLv3}, @samp{TLSv1}, @samp{TLSv1_1} or
@samp{TLSv1_2} forces the use of the corresponding protocol. This is
useful when talking to old and buggy SSL server implementations that
make it hard for the underlying SSL library to choose the correct
protocol version. Fortunately, such servers are quite rare.
Specifying @samp{SSLv2}, @samp{SSLv3}, @samp{TLSv1}, @samp{TLSv1_1},
@samp{TLSv1_2} or @samp{TLSv1_3} forces the use of the corresponding
protocol. This is useful when talking to old and buggy SSL server
implementations that make it hard for the underlying SSL library to choose
the correct protocol version. Fortunately, such servers are quite rare.
Specifying @samp{PFS} enforces the use of the so-called Perfect Forward
Security cipher suites. In short, PFS adds security by creating a one-time
key for each SSL connection. It has a bit more CPU impact on client and server.
We use known to be secure ciphers (e.g. no MD4) and the TLS protocol.
We use known to be secure ciphers (e.g. no MD4) and the TLS protocol. This mode
also explicitly excludes non-PFS key exchange methods, such as RSA.
@item --https-only
When in recursive mode, only HTTPS links are followed.
@item --ciphers
Set the cipher list string. Typically this string sets the
cipher suites and other SSL/TLS options that the user wish should be used, in a
set order of preference (GnuTLS calls it 'priority string'). This string
will be fed verbatim to the SSL/TLS engine (OpenSSL or GnuTLS) and hence
its format and syntax is dependant on that. Wget will not process or manipulate it
in any way. Refer to the OpenSSL or GnuTLS documentation for more information.
@cindex SSL certificate, check
@item --no-check-certificate
Don't check the server certificate against the available certificate
@@ -4131,7 +4146,11 @@ via IRC at @code{irc.freenode.org}, @code{#wget}. Come check it out!
@c man begin BUGS
You are welcome to submit bug reports via the GNU Wget bug tracker (see
@url{https://savannah.gnu.org/bugs/?func=additem&group=wget}).
@url{https://savannah.gnu.org/bugs/?func=additem&group=wget}) or to our
mailing list @email{bug-wget@@gnu.org}.
Visit @url{https://lists.gnu.org/mailman/listinfo/bug-wget} to
get more info (how to subscribe, list archives, ...).
Before actually submitting a bug report, please try to follow a few
simple guidelines.

87
fuzz/Makefile.am Normal file
View File

@@ -0,0 +1,87 @@
AM_CFLAGS = $(WERROR_CFLAGS) $(WARN_CFLAGS) -Wno-unused-parameter -Wno-pedantic
AM_CPPFLAGS = -I$(top_srcdir)/src -I$(srcdir) -I$(top_builddir)/lib -I$(top_srcdir)/lib \
-DSRCDIR=\"$(abs_srcdir)\"
LDADD = ../lib/libgnu.a \
$(GETADDRINFO_LIB) $(HOSTENT_LIB) $(INET_NTOP_LIB) $(INET_PTON_LIB) \
$(LIBSOCKET) $(LIB_CLOCK_GETTIME) $(LIB_CRYPTO) $(LIB_GETLOGIN) $(LIB_NANOSLEEP) $(LIB_POLL) \
$(LIB_POSIX_SPAWN) $(LIB_PTHREAD_SIGMASK) $(LIB_SELECT) $(LIBICONV) $(LIBINTL) \
$(LIBMULTITHREAD) $(LIBTHREAD) $(SERVENT_LIB) @INTL_MACOSX_LIBS@ \
-ldl
WGET_TESTS = \
wget_css_fuzzer$(EXEEXT) \
wget_cookie_fuzzer$(EXEEXT) \
wget_ftpls_fuzzer$(EXEEXT) \
wget_html_fuzzer$(EXEEXT) \
wget_netrc_fuzzer$(EXEEXT) \
wget_options_fuzzer$(EXEEXT) \
wget_robots_fuzzer$(EXEEXT) \
wget_url_fuzzer$(EXEEXT)
if FUZZING
bin_PROGRAMS = $(WGET_TESTS)
LDADD += $(LIB_FUZZING_ENGINE)
MAIN = fuzzer.h
# AM_LDFLAGS = -no-install -all-static
else
AM_CPPFLAGS += -DTEST_RUN
AM_TESTS_ENVIRONMENT = export VALGRIND_TESTS"=@VALGRIND_TESTS@";
TESTS = $(WGET_TESTS)
check_PROGRAMS = $(WGET_TESTS)
MAIN = main.c fuzzer.h
endif
# Make libunittest "PHONY" so we're always sure we're up-to-date.
.PHONY: ../src/libunittest.a
../src/libunittest.a:
$(MAKE) $(AM_MAKEFLAGS) -C ../src libunittest.a
wget_css_fuzzer_SOURCES = wget_css_fuzzer.c $(MAIN)
wget_css_fuzzer_LDADD = ../src/libunittest.a $(LDADD)
wget_cookie_fuzzer_SOURCES = wget_cookie_fuzzer.c $(MAIN)
wget_cookie_fuzzer_LDADD = ../src/libunittest.a $(LDADD)
wget_ftpls_fuzzer_SOURCES = wget_ftpls_fuzzer.c $(MAIN)
wget_ftpls_fuzzer_LDADD = ../src/libunittest.a $(LDADD)
wget_html_fuzzer_SOURCES = wget_html_fuzzer.c $(MAIN)
wget_html_fuzzer_LDADD = ../src/libunittest.a $(LDADD)
wget_netrc_fuzzer_SOURCES = wget_netrc_fuzzer.c $(MAIN)
wget_netrc_fuzzer_LDADD = ../src/libunittest.a $(LDADD)
wget_options_fuzzer_SOURCES = wget_options_fuzzer.c $(MAIN)
wget_options_fuzzer_LDADD = ../src/libunittest.a $(LDADD)
wget_robots_fuzzer_SOURCES = wget_robots_fuzzer.c $(MAIN)
wget_robots_fuzzer_LDADD = ../src/libunittest.a $(LDADD)
wget_url_fuzzer_SOURCES = wget_url_fuzzer.c $(MAIN)
wget_url_fuzzer_LDADD = ../src/libunittest.a $(LDADD)
#EXTRA_DIST = $(wildcard *.options) $(wildcard *.dict) \
# $(wildcard *.in) $(wildcard *.repro)
dist-hook:
find $(srcdir) -name '*.options' -exec cp -v '{}' $(distdir) ';'
find $(srcdir) -name '*.dict' -exec cp -v '{}' $(distdir) ';'
# find $(srcdir) -name '*.in' -exec cp -vr '{}' $(distdir) ';'
find $(srcdir) -name '*.repro' -exec cp -vr '{}' $(distdir) ';'
clean-local:
rm -rf *.gc?? *.log lcov
oss-fuzz:
if test "$$OUT" != ""; then \
XLIBS="-lpsl -lgnutls -lnettle -lhogweed -lidn2 -lunistring"; \
for ccfile in wget*_fuzzer.c; do \
fuzzer=$$(basename $$ccfile .c); \
$$CXX $$CXXFLAGS -I$(top_srcdir)/src -I$(top_srcdir) -I$(top_srcdir)/lib \
"$${fuzzer}.c" -o "$${fuzzer}" \
../src/libunittest.a ../lib/libgnu.a $${LIB_FUZZING_ENGINE} \
-Wl,-Bstatic $${XLIBS} -Wl,-Bdynamic; \
done; \
fi
.PHONY: oss-fuzz

84
fuzz/README.md Normal file
View File

@@ -0,0 +1,84 @@
# Fuzzers
These are fuzzers designed for use with `libFuzzer` or `afl`. They can
be used to run on Google's OSS-Fuzz (https://github.com/google/oss-fuzz/).
The convention used here is that the initial values for each parser fuzzer
are taken from the $NAME.in directory.
Crash reproducers from OSS-Fuzz are put into $NAME.repro directory for
regression testing with top dir 'make check' or 'make check-valgrind'.
# Running a fuzzer using clang
Use the following commands on top dir:
```
export CC=clang-6.0
export CXX=clang++-6.0
# address sanitizer:
export CFLAGS="-O1 -g -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=undefined,integer,nullability -fsanitize=address -fsanitize-address-use-after-scope -fsanitize-coverage=trace-pc-guard,trace-cmp"
# undefined sanitizer;
export CFLAGS="-O1 -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=bool,array-bounds,float-divide-by-zero,function,integer-divide-by-zero,return,shift,signed-integer-overflow,vla-bound,vptr -fno-sanitize-recover=bool,array-bounds,float-divide-by-zero,function,integer-divide-by-zero,return,shift,signed-integer-overflow,vla-bound,vptr -fsanitize=fuzzer-no-link"
export CXXFLAGS="$CFLAGS -stdlib=libc++"
export LIB_FUZZING_ENGINE="-lFuzzer -lstdc++"
./configure --enable-fuzzing --without-metalink --without-zlib --disable-pcre --without-libuuid
make clean
make -j$(nproc)
cd fuzz
# run wget_options_fuzzer
UBSAN_OPTIONS=print_stacktrace=1 ASAN_SYMBOLIZER_PATH=/usr/lib/llvm-6.0/bin/llvm-symbolizer \
./run-clang.sh wget_options_fuzzer
```
If you see a crash, then a crash corpora is written that can be used for further
investigation. E.g.
```
==2410==ERROR: AddressSanitizer: heap-use-after-free on address 0x602000004e90 at pc 0x00000049cf9c bp 0x7fffb5543f70 sp 0x7fffb5543720
...
Test unit written to ./crash-adc83b19e793491b1c6ea0fd8b46cd9f32e592fc
```
To reproduce the crash:
```
./wget_options_fuzzer < ./crash-adc83b19e793491b1c6ea0fd8b46cd9f32e592fc
```
You can also copy/move that file into wget_options_fuzzer.repro/
and re-build the project without fuzzing for a valgrind run, if you like that better.
Just a `./configure` and a `make check-valgrind` should reproduce it.
# Running a fuzzer using AFL
Use the following commands on top dir:
```
$ export LIB_FUZZING_ENGINE=""
$ CC=afl-clang-fast ./configure --enable-fuzzing
$ make -j$(nproc) clean all
$ cd fuzz
$ ./run-afl.sh wget_options_fuzzer
```
# Fuzz code coverage using the corpus directories *.in/
Code coverage reports currently work best with gcc+lcov+genhtml.
In the top directory:
```
CC=gcc CFLAGS="-O0 -g" ./configure
make fuzz-coverage
xdg-open lcov/index.html
```
To work on corpora for better coverage, `cd fuzz` and use e.g.
`./view-coverage.sh wget_options_fuzzer`.
# Creating wget_options_fuzzer.dict
```
for i in `../src/wget --help|tr ' ' '\n'|grep ^--|cut -c 3-|sort`;do echo \"$i\"; done >wget_options_fuzzer.dict
```

26
fuzz/fuzzer.h Normal file
View File

@@ -0,0 +1,26 @@
/*
* Copyright(c) 2017-2018 Free Software Foundation, Inc.
*
* This file is part of GNU Wget.
*
* GNU Wget is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 3 of the License, or
* (at your option) any later version.
*
* GNU Wget is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Wget. If not, see <https://www.gnu.org/licenses/>.
*/
#include <stddef.h> // size_t
#include <stdint.h> // uint8_t
#ifdef __cplusplus
extern "C"
#endif
int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size);

11
fuzz/get_all_corpora Executable file
View File

@@ -0,0 +1,11 @@
#!/bin/sh -eu
if ! grep -q FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION Makefile; then
echo "The fuzzers haven't been built for fuzzing (maybe for regression testing !?)"
echo "Please built regarding README.md and try again."
exit 1
fi
for fuzzer in *_fuzzer.c; do
./get_ossfuzz_corpora $(basename $fuzzer .c)
done

46
fuzz/get_ossfuzz_corpora Executable file
View File

@@ -0,0 +1,46 @@
#!/bin/sh -eu
# As a first step see README.md and follow the steps under "Running a fuzzer using clang".
# You might need 'gsutil' to download new corpora from the Google cloud:
# Read the docs at https://github.com/google/oss-fuzz/blob/master/docs/corpora.md
# then install 'google-cloud-sdk' and execute 'gcloud init'.
# Now 'gsutil' should be ready to use.
if test -z "$1"; then
echo "Usage: $0 <fuzzer target>"
echo "Example: $0 wget_options_fuzzer"
exit 1
fi
if ! grep -q FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION Makefile; then
echo "The fuzzers haven't been built for fuzzing (maybe for regression testing !?)"
echo "Please built regarding README.md and try again."
exit 1
fi
fuzzer=$1
project=wget
# sync/copy the OSS-Fuzz corpora into the .new directory
mkdir -p ${fuzzer}.new
cp -fp ${fuzzer}.in/* ${fuzzer}.new
gsutil cp $(gsutil ls gs://${project}-backup.clusterfuzz-external.appspot.com/corpus/libFuzzer/${fuzzer}|tail -n 1) ${fuzzer}.new
(cd ${fuzzer}.new && unzip -q -f -o *.zip && rm *.zip)
# create fuzzer target
BUILD_ONLY=1 ./run-clang.sh ${fuzzer}
# merge the corpora into the .in directory
./${fuzzer} -merge=1 ${fuzzer}.in ${fuzzer}.new
# now clear .new dir and put all corpora there
rm -rf ${fuzzer}.new
mv ${fuzzer}.in ${fuzzer}.new
mkdir ${fuzzer}.in
# now merge again (optimizes number of corpora)
./${fuzzer} -merge=1 ${fuzzer}.in ${fuzzer}.new
echo
echo "If new files have been added, 'git add' and 'git commit' them."

32
fuzz/glob_crash.c Normal file
View File

@@ -0,0 +1,32 @@
/*
* Created 19.10.2017 by Tim Rühsen
*
* Call glob() using data from fuzzer crash file
*
* Build and execute with instrumented gnulib (amend -I paths as needed):
*
* clang build (spills out WRITE heap buffer overflow)
* export CC=clang-6.0
* export CFLAGS="-O1 -g -fno-omit-frame-pointer -fsanitize=address -fsanitize-address-use-after-scope"
* $CC $CFLAGS -I.. -I../lib glob_crash.c -o glob_crash ../lib/.libs/libgnu.a
* ./glob_crash
*
* gcc build (spills out READ heap buffer overflow):
* export CC=gcc
* export CFLAGS="-O1 -g -fno-omit-frame-pointer -fsanitize=address -fsanitize-address-use-after-scope"
* $CC $CFLAGS -I.. -I../lib glob_crash.c -o glob_crash ../lib/.libs/libgnu.a
* ./glob_crash
*/
#include <glob.h>
int main(int argc, char **argv)
{
static unsigned char data[] = "1e";
glob_t pglob;
if (glob(data, GLOB_TILDE|GLOB_ONLYDIR|GLOB_NOCHECK, NULL, &pglob) == 0)
globfree(&pglob);
return 0;
}

154
fuzz/main.c Normal file
View File

@@ -0,0 +1,154 @@
/*
* Copyright(c) 2017-2018 Free Software Foundation, Inc.
*
* This file is part of GNU Wget.
*
* GNU Wget is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 3 of the License, or
* (at your option) any later version.
*
* GNU Wget is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Wget. If not, see <https://www.gnu.org/licenses/>.
*/
#include <config.h>
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <string.h>
#include <fcntl.h>
#include "fuzzer.h"
#include "utils.h"
#ifdef TEST_RUN
#include <dirent.h>
#ifdef _WIN32
# define SLASH '\\'
#else
# define SLASH '/'
#endif
static int test_all_from(const char *dirname)
{
DIR *dirp;
if ((dirp = opendir(dirname))) {
struct dirent *dp;
while ((dp = readdir(dirp))) {
if (*dp->d_name == '.') continue;
char fname[strlen(dirname) + strlen(dp->d_name) + 2];
snprintf(fname, sizeof(fname), "%s/%s", dirname, dp->d_name);
struct file_memory *fmem;
if ((fmem = wget_read_file(fname))) {
printf("testing %ld bytes from '%s'\n", fmem->length, fname);
LLVMFuzzerTestOneInput((uint8_t *)fmem->content, fmem->length);
wget_read_file_free(fmem);
}
}
closedir(dirp);
return 0;
}
return 1;
}
int main(int argc, char **argv)
{
// if VALGRIND testing is enabled, we have to call ourselves with valgrind checking
const char *valgrind = getenv("VALGRIND_TESTS");
const char *target;
size_t target_len;
if (!valgrind || !*valgrind || !strcmp(valgrind, "0")) {
// fallthrough
}
else if (!strcmp(valgrind, "1")) {
char cmd[strlen(argv[0]) + 256];
snprintf(cmd, sizeof(cmd), "VALGRIND_TESTS=\"\" valgrind --error-exitcode=301 --leak-check=yes --show-reachable=yes --track-origins=yes %s", argv[0]);
return system(cmd) != 0;
} else {
char cmd[strlen(valgrind) + strlen(argv[0]) + 32];
snprintf(cmd, sizeof(cmd), "VALGRIND_TESTS="" %s %s", valgrind, argv[0]);
return system(cmd) != 0;
}
if ((target = strrchr(argv[0], SLASH)))
target = strrchr(target, '/');
else
target = strrchr(argv[0], '/');
target = target ? target + 1 : argv[0];
if (strncmp(target, "lt-", 3) == 0)
target += 3;
target_len = strlen(target);
#ifdef _WIN32
target_len -= 4; // ignore .exe
#endif
{
int rc;
char corporadir[sizeof(SRCDIR) + 1 + target_len + 8];
snprintf(corporadir, sizeof(corporadir), SRCDIR "/%.*s.in", (int) target_len, target);
rc = test_all_from(corporadir);
if (rc)
fprintf(stderr, "Failed to find %s\n", corporadir);
snprintf(corporadir, sizeof(corporadir), SRCDIR "/%.*s.repro", (int) target_len, target);
if (test_all_from(corporadir) && rc)
return 77; // SKIP
}
return 0;
}
#else
#ifndef __AFL_LOOP
static int __AFL_LOOP(int n)
{
static int first = 1;
if (first) {
first = 0;
return n && --n > 0;
}
return 0;
}
#endif
int main(int argc, char **argv)
{
int ret;
unsigned char buf[64 * 1024];
while (__AFL_LOOP(10000)) { // only works with clang - we have to use 1 because static/global vars in wget
ret = fread(buf, 1, sizeof(buf), stdin);
if (ret < 0)
return 0;
LLVMFuzzerTestOneInput(buf, ret);
}
return 0;
}
#endif /* #ifdef TEST_RUN */

59
fuzz/run-afl.sh Executable file
View File

@@ -0,0 +1,59 @@
#!/bin/sh -eu
# Copyright(c) 2017 Free Software Foundation, Inc.
#
# This file is part of GNU Wget.
#
# GNU Wget is free software: you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# GNU Wget is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public License
# along with GNU Wget. If not, see <https://www.gnu.org/licenses/>.
if ! grep -q '^CC=.afl-clang-fast' ../config.log; then
echo "compile first library as:"
echo "CC=afl-clang-fast ./configure"
exit 1
fi
if test -z "$1"; then
echo "Usage: $0 test-case"
echo "Example: $0 wget_options_fuzzer"
exit 1
fi
fuzzer=$1
#rm -f $fuzzer
#afl-clang-fast -O2 -g -I.. main.c "${fuzzer}.c" -o "${fuzzer}"
### minimize test corpora
if test -d ${fuzzer}.in; then
mkdir -p ${fuzzer}.min
for i in `ls ${fuzzer}.in`; do
fin="${fuzzer}.in/$i"
fmin="${fuzzer}.min/$i"
if ! test -e $fmin || test $fin -nt $fmin; then
afl-tmin -m 500 -i $fin -o $fmin -- ./${fuzzer}
fi
done
fi
TMPOUT=${fuzzer}.out
mkdir -p ${TMPOUT}
if test -f ${fuzzer}.dict; then
afl-fuzz -m 500 -i ${fuzzer}.min -o ${TMPOUT} -x ${fuzzer}.dict -- ./${fuzzer}
else
afl-fuzz -m 500 -i ${fuzzer}.min -o ${TMPOUT} -- ./${fuzzer}
fi
echo "output was stored in $TMPOUT"
exit 0

57
fuzz/run-clang.sh Executable file
View File

@@ -0,0 +1,57 @@
#!/bin/bash -e
#
# Copyright(c) 2017 Free Software Foundation, Inc.
#
# This file is part of GNU Wget.
#
# GNU Wget is free software: you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# GNU Wget is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public License
# along with GNU Wget. If not, see <https://www.gnu.org/licenses/>.
trap ctrl_c INT
ctrl_c() {
./${fuzzer} -merge=1 ${fuzzer}.in ${fuzzer}.new
rm -rf ${fuzzer}.new
}
if test -z "$1"; then
echo "Usage: $0 <fuzzer target>"
echo "Example: $0 wget_options_fuzzer"
exit 1
fi
if ! grep -q FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION Makefile; then
echo "The fuzzers haven't been built for fuzzing (maybe for regression testing !?)"
echo "Please built regarding README.md and try again."
exit 1
fi
# you'll need ~2GB free memory per worker !
fuzzer=$1
workers=$(($(nproc) - 0))
jobs=$workers
if test -n "$BUILD_ONLY"; then
exit 0
fi
# create directory for NEW test corpora (covering new areas of code)
mkdir -p ${fuzzer}.new
if test -f ${fuzzer}.dict; then
./${fuzzer} -dict=${fuzzer}.dict ${fuzzer}.new ${fuzzer}.in -jobs=$jobs -workers=$workers
else
./${fuzzer} ${fuzzer}.new ${fuzzer}.in -jobs=$jobs -workers=$workers
fi
exit 0

37
fuzz/view-coverage.sh Executable file
View File

@@ -0,0 +1,37 @@
#!/bin/bash -eu
#
# (C)2017 Tim Ruehsen tim.ruehsen@gmx.de
#
# View the coverage report for one or more fuzzers.
# 1. execute 'make fuzz-coverage' in the top directory
# 2. execute './view-coverage.sh <fuzz target(s)>
# Example with single fuzzer:
# ./view-coverage.sh wget_options_fuzzer
# Example with two fuzzers:
# ./view-coverage.sh wget_options_fuzzer wget_html_parse_fuzzer
if test -z "$1"; then
echo "Usage: $0 <fuzz target(s)>"
echo "Example: $0 wget_options_fuzzer"
exit 1
fi
#fuzzer="./"$1
LCOV_INFO=coverage.info
#./coverage.sh $fuzzer
#lcov --capture --initial --directory ../src/.libs --directory . --output-file $LCOV_INFO
#lcov --capture --directory ../src/.libs --output-file $LCOV_INFO
#lcov --remove $LCOV_INFO '*/test_linking.c' '*/css_tokenizer.lex' '*/<stdout>' '*/*.h' -o $LCOV_INFO
#genhtml --prefix . --ignore-errors source $LCOV_INFO --legend --title "$1" --output-directory=lcov
lcov --zerocounters --directory ../src/
lcov --capture --initial --directory ../src/.libs --directory . --output-file $LCOV_INFO
make check TESTS="$*" CFLAGS="$(CFLAGS) --coverage" LDFLAGS="$(LDFLAGS) --coverage"
lcov --capture --directory ../src/.libs --output-file $LCOV_INFO
lcov --remove $LCOV_INFO '*/css_tokenizer.lex' '*/*.h' -o $LCOV_INFO
genhtml --prefix . --ignore-errors source $LCOV_INFO --legend --title "$*" --output-directory=lcov
xdg-open lcov/index.html

97
fuzz/wget_cookie_fuzzer.c Normal file
View File

@@ -0,0 +1,97 @@
/*
* Copyright(c) 2017-2018 Free Software Foundation, Inc.
*
* This file is part of GNU Wget.
*
* GNU Wget is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 3 of the License, or
* (at your option) any later version.
*
* GNU Wget is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Wget. If not, see <https://www.gnu.org/licenses/>.
*/
#include <config.h>
#include <sys/types.h>
#include <dirent.h> // opendir, readdir
#include <stdint.h> // uint8_t
#include <stdio.h> // fmemopen
#include <string.h> // strncmp
#include <stdlib.h> // free
#include "wget.h"
#undef fopen_wgetrc
#ifdef __cplusplus
extern "C" {
#endif
#include "cookies.h"
// declarations for wget internal functions
int main_wget(int argc, const char **argv);
void cleanup(void);
FILE *fopen_wget(const char *pathname, const char *mode);
FILE *fopen_wgetrc(const char *pathname, const char *mode);
void exit_wget(int status);
#ifdef __cplusplus
}
#endif
#include "fuzzer.h"
FILE *fopen_wget(const char *pathname, const char *mode)
{
return fopen("/dev/null", mode);
}
FILE *fopen_wgetrc(const char *pathname, const char *mode)
{
return NULL;
}
#ifdef FUZZING
void exit_wget(int status)
{
}
#else
void exit(int status)
{
}
#endif
int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
{
FILE *bak;
struct cookie_jar *cookie_jar;
char *set_cookie;
if (size > 1024) // same as max_len = ... in .options file
return 0;
set_cookie = (char *) malloc(size + 1);
memcpy(set_cookie, data, size);
set_cookie[size] = 0;
bak = stderr;
stderr = fopen("/dev/null", "w");
cookie_jar = cookie_jar_new();
cookie_handle_set_cookie(cookie_jar, "x", 81, "p", set_cookie);
cookie_handle_set_cookie(cookie_jar, "x", 81, "p", set_cookie);
cookie_handle_set_cookie(cookie_jar, "x", 80, "p/d/", set_cookie);
cookie_jar_delete(cookie_jar);
fclose(stderr);
stderr = bak;
free(set_cookie);
return 0;
}

View File

@@ -0,0 +1,6 @@
tag1="expires="
tag2="path="
tag3="domain="
tag4="httponly"
tag5="secure"
tag6="max-age"

View File

@@ -0,0 +1 @@
__Host-ID=x; expires=MoHos__Host-ID=x; expires=Mon, 29 Feb 2016 07:48:97 GMT; mh=/; domain=x.y; httponly; sec__Host-ID=x; expires=Mon, 29 Feb 2016 07:48:97 GMT; mh=/; domain=x.y; htut-ID=x; expires=Mon, 29 Feb 2016 07:48:97 GMT; mh=/; domain=x.y; hture

View File

@@ -0,0 +1 @@
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDNDDDDD廛max-ageDDDDDD'DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD僖DDDDDhttponlyDDDDDDDDDDDDDDDDDDD;

View File

@@ -0,0 +1 @@
__Hos_o=;;yõ=m;tmID=m;=;y=m;y=m;<;yõ=m;tmID=m;

View File

@@ -0,0 +1 @@
__expires=x; expires=Mon, 14 4032 07:48Mon, 14 4032 07:48:5$ GMT; stmID=x; expires=Mon, 14 4000 07:48:54 GMT; matp=x; expires=Mon, 43 4032 07:48:5$ G032 07:48:27 GMT; stm 14 4000 07ÅË8:54 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%GMT; m:48:54 GMT; stmI»D=x; e|pires=Mon, 14 4000 07ID=x; expires=Mon, 07 4032 07:48:5$ GMT; stmID=x; expin, 14 4032 07:48:5$ GMT; stmID=x; expires=Mon, 14 4000 07:48:54 GMT; matp=x; expires=Mon, 14 4032 07:48:5$ G!32 07:48:54 GMT; stm 14 4000 07ÅË8:54 GMT; m:48:54 GMT; stmI»n4 1, 4032 07:48:5$ GMT; stmID=x; expcure

View File

@@ -0,0 +1,2 @@
__H@=x qqqqqqqqqqqqqqqqq*qqqqqqFpath=; secure
x-age; secure

View File

@@ -0,0 +1 @@
_dict=libwget_cookie_fuzzer.dmutate_depth _e=;Ä;Ø;;;;Ø;;;;Ä;=;;Ø;;;;Ä;=;__Host-I@=x; expires=Mon, 29 Feb@=x; expir st-016 07:48es=Mon, 28 Feb 2016 07::54 GMT;"max-48:

View File

@@ -0,0 +1 @@
_use¡<>šÂÊà pa<70>h=_Host-ID=xGMT; max-age=5; pa<70>h=ÑÄß<C39F>ž•n=x.y; httpomax-agecure

View File

@@ -0,0 +1 @@
*=';:n*='d

View File

@@ -0,0 +1 @@
__Host-ID=x; expires=29 Feb 2016 07:48:54 GMT;t-ID=x; expires=29 Feb 2016 10:48:54 GMT; max-age=5; p; expires=29 Feb 2016 07:48:54 GMT;t-ID=x; expires=58 Feb 2016 10:48:54 GMT; max-age=5; pathomt-ID=x; expires=29 Feb 2016 07:48:54 GMT;t-IDBx; exp=x.y; athomt-ID=x; expires=29 Feb 2016 07:48:54 GMT;t-IDBx; expires=29 Feb 2016 07:48:54 GMT; max-age=5; pathomai;ost-ID=x; expires=29 Feb 2016 07:48:54 GMT;t-ID=x; expires=2; Feb 2016 10:48:5: GMT; max-age=5; pathomt-ID=x; expires=29 Feb 2016 07:48:54 GMT;t-ID=x; expires=29 Feb 2096 07:48:54 GMT; max-age=5; pathomai;=x.y; httponly

View File

@@ -0,0 +1 @@
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

View File

@@ -0,0 +1 @@
xexpires= domain==/; domain==/; domain=th=/; domain=th=/; domain=th=/; domain==/; domain=th=/; domain==/; domain=th=/; domain==/; domain=ht=/; domain==/; domain=th=/; domain==/; domain=th=/expires=; domain==/; domain=th=/;omain==/;*=/; >domain=h=/; domain==/; doamin==/; doiain=th=/; domain=th=/:; domain=th=/; domain==/; domain=th=/; dom'ain==/; domain=th=/; domain==/; domain=th=/; ƒƒƒƒƒƒƒdomain==/; domain=th=/; domai=th=/; domain ==/; domain=th=/expires=; domain==/; domain=th=/;omain==/; domain=th=/; domain==/; domain=th=/expires=; domainy=

View File

@@ -0,0 +1 @@
_httpominimize_crash16 07:48??¿??????????????????????????????????,,,,,,,,,,6:48??¿????????????????????????expires=??????????,,,,,,,,,,,,,,ŠŠŠŠŠŠŠŠŠŠŠ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,???????54 GMT; max-age=5; paagominimize_crash16 07:48??¿????????????;?????????????????????,,,,,,,,,,7:48??¿????????????????????????expires=??????????,,,,,,,,,,,,,,ŠŠŠŠŠŠŠŠŠŠŠ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,???????5path=2 GMT; max-age=5; paagŠŠŠŠŠŠŠŠŠŠ,,,,,,,,,,,,,,,,,,,,,,,,,l,,,???????54 GMT; ŠŠŠ,,,,,,,,,,,,,,,,,,,,,,,,,l,,,???????54 GMT;ge=5=x.ÿÿÿ ly; sñcure

View File

@@ -0,0 +1,2 @@
_ÿÿÿÿÿÿ
ä=x; expi!res=-86148:54 =x; expires=-43855 Feb 6119 07:48:0Tx; expires=-437708 Feb 6120 07:48:54 8:54 =x; expires=-86148:54 =x; expires=-43855 Feb 6119 07:48:5Tx; expires=-437708 Feb 6120 07:48:54 8:54 =x; expires=-43855 Feb 6119 07:48:5TxC; expires=-437708 Feb 120 07:48:54 =b 6119 07:48:5Tx; expires=-218854 Feb 6120 07:48:54 =_axost-ID=x; 6120 07:48:54 =st-ID=x; expires=-87708 Feb 6119 07:48:54 =x; expires=3-4855 Feb 6119 07:48:5Tx; emax-age43855 Feb 6119 07:48:5Tx; expires=-437708 Feb 6120 07:48:54 =b 6119 07:48:5Tx; expires=-437708 Feb 6120 07:48:54 48:5Tx; expires=-437708 Feb 6120 07:48:54 =_axost-ID=x; ex88888888888888888888888888888888888888888pires=-c87708 Feb 6119 07:48:54 =x; expires=-43855 Feb 6119 07:48:5Tx; expires=-43855 Feb 611907:48:5T

View File

@@ -0,0 +1 @@
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""

View File

@@ -0,0 +1 @@
ě=;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;D;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;mai;Dmaa;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;Dmai;=

View File

@@ -0,0 +1 @@
_ÿÿÿÿÿÿ ¤seedANShttponly.;domain=.;_;D=_;domain=.;domain=.;_;;domain=.;domain=.;;;doma=o;id;n.main=.;;d(o'main=.;domainn=.;domain=.;_;Domain=.;(;domain=.=_;domain=.;domain=.;_;;domain=.;domaiQQQQQQQQQQQQQQQQQQQQQQQQQQQQomain=.;;domain=.;domain=.;domain=.;(;domain=.;;;domain=.;domain=.;;;domain=.;d_D=_;domain=.;domain=.;_;n=.;domain=.;;;domain=.;;domain=.;;domain=.;domainain=.;domain=.;_;;domain=.;domain=.;;;domain=.;;domain=.;;domain=.;domain=.;domain=.;(;domain=.;;;domain=.;domain=.;om

View File

@@ -0,0 +1,3 @@
__Host-ID=x; 4032 age=2;HHHHHHdomain=HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHh=l; domain=x.y; httpohttponly; sñcurnly;ýly; seceur
e
securd

View File

@@ -0,0 +1 @@
:=;domain=';path=/p/

View File

@@ -0,0 +1 @@
_o=;=;=;y=;;=;=;;<3B>y=_e=';˙˙ qa<71>h=_Host-ID=xGMT; <20>h=_Host-ID=xGMT; domain=˙˙˙˙˙˙˙httponly; securSSSSSSSSSSSSSSSSnly; securSSSSSÓSSSSSSSSSSSSSSSSSSSSSSSSSSSS;=;=;=SSSSSSSSSS;;=;=;=;y=;=;=;=;;y=;=;=;=;=;=;=;==;=SSSS;=;==SSSSSSSSSSSSSSSSSSSS:SSSe

View File

@@ -0,0 +1 @@
_D=_;domain=.;domain=.;domain=.;;;do“ž“=.;;domain=.;;domain=.;domTTT(TTTTTTTTTTTTTTT*************************************************************************************TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTainn=.;domainin=.;domain=.;;;doman;=o;i.dmain=.;;domain=.;domain=.;domain=.;(;domain=.;;;domain=.;domain=.;om

View File

@@ -0,0 +1 @@
WWhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhghhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh

View File

@@ -0,0 +1 @@
__Host-KDFeec)ure

View File

@@ -0,0 +1,2 @@
__Hos__Host-ID__Host-ID=x; expires=Mon, 5=x.y; httpontsecure-IDmain=ontsecureontsecure-ID=x; expires=Mon, 29 Feb 2016UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU«ªªªªªªªUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUax-age=2; pat$=/; x.y; httponly; ly; secure
secure

View File

@@ -0,0 +1,2 @@
__H@=x @=x; expires=Mon, 29 Femmmmmmmmmmmqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq*qqqqqqFpath=; secure
x-age; secure

View File

@@ -0,0 +1 @@
__Host-ID=x; expires=29 Feb 2016 07:48:54 GMT;t-ID=x; expiMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM³¨MMMMMMMMMMMMMMMMMMMMMMMMMres=29 Feb 2expires=016 10:48:54 GMT; max-ageGMT; max-age=5; pathomt-ID=x; expires=29 Feb 2016 07:48:54 GMT;t-IDBx; exp=x.y; athomt-ID=x; expires=58 Feb 2016 07:48:54 GMT;t-IDBx; expires=29 Feb 2016 07:48:54 GM;t-IDBx; exp=x.y; athomt-ID=x; expires=29 Fÿÿÿÿÿÿÿÿÿÿeb 2016 07:48:54 GMT;t-IDBx; expires=29 Feb 2016 07:48:54 GMT; max-age=5; pathomai;ost-ID=x; expires=29 FT; max-age=5; pathomai;ost-ID=x; expires=29 Feb 2016 07:48:54 GMT;t-ID=x; expires=2; Feb 2016 10:48:5: GMT; max-age=5; pathomt-ID=x; max-age=5; pathomai;=x.y; httponly

View File

@@ -0,0 +1 @@
__Host-ID=x; e httponjjjjjjjjjjjjjjjjjjjjzjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjzjjjjjjjjjjjjjjjjjjjjjjjjGMT;Žexpires=n=x.; sec}re

View File

@@ -0,0 +1 @@
_D=_;domain=.;domain=.;_;D=_;domain=.;domain=.;_;;domain=.;domain=.;;;domain=.;;domain=.;;domain=.;domainn=.;domain=.;_;Domain=.;(;domain=.=_;domain=.;domain=.;_;;domain=.;domain=.;;;domain=.;;domain=.;;domain=.;domain=.;domain=.;(;domain=.;;;domain=.;domain=.;;;domain=.;d_D=_;domain=.;domain=.;_;D=_;dom_;D=_;domain=.;domain=.;_;;domain=.;domain==_;dom_;D=_;domain=.;domain=.;_;;domain=.;domain=.;;;domain=.;_NNN)NNNNires=M/n,çË4 F; dNNNNNNNNNNNNNNNNy; ;domain=.;;domain=.;domainn=.;domain=.;_;D=_;domain=.;domain=.;domain=.;;;domain=.;domain=.;;;domain=.;;domain=..;;;domain=.;;domain=.;;domain=.;domainn=.;domain=.;_;D=_;domain=.;domain=.;domain=.;;;domain=.;domain=.;;;domain=.;;domain=.;;domain=.;domainhttp=.;domain=.;;domain=.;;;domain=.;domain=.;domain=.;(;domain=.;m.=ndin=.;;;domain=.;d_D=_;domain=.;domain=.;_;D=_;domain=.;domain=.;_;;doin=.;;;domain=.;domain=.;;;domain=.;;domainin=.;domain=.;domain=.;(;domain=.;;;domain=.;domain=onlyre.;om

View File

@@ -0,0 +1,4 @@
(_pathnul)__Hos__Host-Id=x; expires,M 2on=domain=064 07:4:854 st-Id=x; expires,M 2on=domain=064 07:4:854(nuŽ)_ GMT; max-)__Hos__Host-Id=x; expires,M 2on=domain=064 07:4:854 st-Id=x; expires,M 2o)n=domain=064 07:4:854(nuŽ)_ GMT; max-À;e=4 páth:=/; s__Host-Id=x; expires,M 2on=domain=064 07:4:854 st-Id=x; expires,M 2on=domain=064 07:4:854 GMT; a; ÿÿÿ)GMmaire
secÀ;e=4 páth:=/; s__Host-Id=x; expires,M 2on=domain=064 07:4:854 st-Id=x; expires,M 2on=domain=064 07:4:854 GMT; a__Hosnul)__Hos__Host-Id=x; expires,M 2on=domain=064 07:4:854 st-Id=x; expires,M 2on=domain=064 07:4:854(nuŽ)_ GMT; max-)__Hos__Host-Id=x; expires,M 2on=domain=064 07:4:854 st-Id=x; expires,M 2o)n=domain=064 07:4:854(nuŽ)_ GMT; max-À;e=4 páth:=/; s__Host-Id=x; expires,M 2on=domain=064 07:4:854 st-I__Host-Id=x; expires,M 2on=domain=064 07:4:854 st-Id=x; expires,M 2on=domain=064 07:4:854(nuŽ)_ GMT; maire
secure

View File

@@ -0,0 +1 @@
9=;exp=;expires=¹;expires=8=;expires=¹;expires=¹99=;expires=¹;expires==;expires=¹;expires=¹;e;expires=¹;expires=¹99kk=;expires=¹;expires==;expires=¹;expires=¹;expires==ires=¹;expires=8=;expires=¹;expires=¹99=;expires=¹;expires==;expires=¹;expires=¹;e;exp_res=¹;expires=¹;expires==ires=¹;expires=8=;expires=¹;expires=¹99=;expires=¹;expires==;expires=¹;expires=¹;e;exps=¹;expires=8=;expires=¹;expires=¹99=;expires=¹;expires==;expires=¹;expires=¹;e;expires=¹;expires=¹;expires==ires=¹;expires=8=;expires=¹;expires=¹;expires=¹99kk=;expires=¹;ex99kk=;expires=¹;expires==;expires=s=¹;expires=¹99=;expires=¹;expires==;expires=¹;expires=¹;e;expires=¹;expires=¹99kk=;expiresA¹;expires==;expires=¹;expires=¹;expires==ires=¹;expires=8=;expires=¹;expires=¹99=;expires=¹;expires==;expires=¹;expires=¹;e;expires=¹;expires=¹;expires==ires=¹;expires=8=;expires=¹;expires=¹99=;expires=¹;expires==;expires=¹==;expires=¹;expires=¹;expires==ires=¹;expires=8=;99kk=;expires=¹;expires==;expires=¹;expires=¹;exp¹;expires=¹;expires==;;e

View File

@@ -0,0 +1 @@
_16 07:48'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''':53 Gmain=xcrse ;

View File

@@ -0,0 +1 @@
xmth=/;.;secure@=x..;secure@=x..;secure@=x..;secpath=ure@=x;xmth=/;.;secure@=x..;sectre@=x..;secure@=x;xmth=/;.;secure@=x..;secure@=x..;secure@=x;xmth=/;.;secure@=httponlyre@=x..;secure@=x;xmth=/;.;secure@=x..;secure@=x..;secure@=x;xmth=/;.;secure@=x..;secure@=x..;secure@=x;xmth=/;.;ctre@=x..;.;secure@=x..;secure@=x;

View File

@@ -0,0 +1 @@
_ANSI_D=_;domain=.;domain=.;domain=.;;;do“ž“=.;;domain=.;;domain=.;domTTTTTTTTTTTTTTTTTTT*************************************************************************************TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTainn=.;domain=.;_;Domain=.;(;domain=.=_;domain=.;domain=.;_;;domain=.;domain=.;;};domain=.;;domain=.;;domain=.;do=_;domain=.;domain=.;_;D=_;domain=*;domain=.;_;;domain=.;domain=.;;;doman;=o;i.dmain=.;;domain=.;domain=.;domain=.;(;domain=.;;;domain=.;domain=.;om

View File

@@ -0,0 +1 @@
x/ict="" <20><>==

View File

@@ -0,0 +1 @@
__Hos__Hostk__Hos__Host-__Hos__Host-ID=x; expires=Mon, 29 Feb 4000 07:48:5-4 GMT; íID=x; expires=Mon, 29 Feb 4000 07:48:5-4 GMT;Host-ID=x; expires=Mon, 29 Feb 4000 07:48:5-4 GMT__Host-__Hos__Host-ID=x; expires=Mon, 29 Feb 4000 07: íax-ÀpD=x; expires-a-ID=x; expires=Mon, 29 Feb 4000 07:48:5-4 GMT__Host-__Hos__Host-ID=x; expires=Mon, 29 Feb 4000 07:48:5-4 GMT; íID=x; expires=Mon, 29 Feb 4000 07:48:5-4 , 29 Feb 4000 07: íax-ÀpD=x; expires-a-ID=x; expires=Mon, 29 Feb 4000 07:48:5-4 GMT__Host-__Hos__Host-ID=x; expires=Mon, 29 Feb 4000 07:48:5-4 GMT; íID=x; expires=Mon, 29 Feb 4000 07:48:5-4 GMT;Host-ID=x; expires=Mon, 29 Feb 4000 07:48:5-4 GMT__Host-__Hos__Host-ID=x; expires=Mon, 29 Feb 4000 07: íax-ÀpD=GMT;Host-ID=x; expires=Mon, 29 Feb 4000 07:48:5-4 GMT__Host-__Hos__Host-ID=x; expires=Mon, 29 Feb 4000 07: íax-ÀpD=x; expires-age=2; p™ontsecurgÏe-ID=xÏ

View File

@@ -0,0 +1 @@
D=seï;DT=;D=þ;D=;DD=;;D=;DD=;D=;D=;D=;D=þ;D=w;DD=;D=;D=;DD=:;D=;DT=;D=;D=w;domain=D=;D=

View File

@@ -0,0 +1 @@
__Host-ID=x; expires=Mon, 44 F ;httponly domain=x.y; httponlyre

View File

@@ -0,0 +1 @@
xmth=/; 4 ..<2E>;<3B>e<EFBFBD>.;<3B>n<EFBFBD>e<EFBFBD>.;<3B>nly=

View File

@@ -0,0 +1 @@
_dict=wget_cook{{{{{{{{{{{{{{{{{{{{{{{{˙˙˙˙˙˙)˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙;;path=/; domain=x.y; h˙˙˙˙˙˙˙˙˙˙˙˙ttđPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPő˙˙˙˙˙ŕŕsńcure

View File

@@ -0,0 +1,2 @@
__Hires=˙{{{{{{{{{{{{{{{{{{{{{{{{{{{{˙˙˙˙˙˙˙˙˙˙˙˙˙˙ ˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙use_cmp˙˙˙˙˙˙VVVVVVVVVVVVVVVVV˙˙˙˙˙˙˙VVVVVVVVVVVVVVVVVVVVVVonly; sy; ecure
sńcure

View File

@@ -0,0 +1,5 @@
ahttponlyi!=x&y;qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqQqqqqqqqqqqqi!=x&y;qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq path=&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&; secure
x-age; sec29 Feb 2015 07:48:54 GMT; max-domai!=x&y;qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq path=&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&; secure
x-age; secuqqqqq path=&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&; secure
x-age; sec29 Feb 2015 07:48:54 GM!=x&y;qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq path=&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&(&&&&&&&&&&&; secure
x-age; secure

View File

@@ -0,0 +1 @@
__Host-ax-domain=x.y; patI@=\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\ax-doma®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®®in=x.y; patMon, ;T; #mŸ‡Ò<C392>žŒn=x.y; path=h path=x.y; pt=ahh path=httponl@=x; expires=Mon, ;T; max-domain=x.y;__Host-I@=x; expires=-domain=x.y; patI@=x; eres=Mon, ;T; #max-domain=x.y; patMon, ;T; #max-domain=x.y;p atx; expires=Mon, ;T; max-doure

View File

@@ -0,0 +1,11 @@
__Hos__Host-ID=x; expires=Mottpont-ID=x; exp7:48:54 GMT; mires=Moe
secure
aø-Àge=5; path=/; domain=x.y exp7:48:57:48:54 GMT; mires=Moe
secure
aø-À&&&&&&&ge=5; path=/; domain=x.y exp7:48:54 GMT; mires=Moe
secuANT; max-\ge=2ÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇ; xain=x.y; hexp4 GMT; mires=Moe
seexpires=Mottpont-ID=x; exp7:48:54 GMT; mires=Moe
secure
aø-Àge=5; path=/; domain=x.y exp7:48:54 GMT; mires=Moe
secuANT; max-\ge=2ÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇ; xaÇÇÇÇÇÇ; xain=x.y; hcuANT; max-\ge=2ÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇ9>; xain=x.y; hexpiÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇsecureÇÇÇÇÇÇÇÇÇÇ; xain=x.y; hexpires= ly; secure
secure

View File

@@ -0,0 +1 @@
Pxdict="get_cookmagth=/doiimain=~"~~~~~~~~~~~~~~~~~max-age~~~~~~~~~~~~~~~~~~~~~~~~~~~~=˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙˙==/;(domain=th=/=/; domaininy=

View File

@@ -0,0 +1,4 @@
__Hos__Host-ID=x; expires=Mon, 29 Feb 4000 07:48:54 GMT; max-Àge=5; path=/; domain=x.y; main=x.y; httpont-ID=x; expires=Mon, 29 Feb 2016 07:48:54 GMT; max-age=2; path=/; domain=x.y; httponly; ly; sCCCCCCCCCCCCCCCCCCCCCCCCCCCe}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}cure__Ho=x; expires=Mon, 29 Feb 4000 0:7:48:54 GMs__Host-ID=x; expires=Mon, 29 Feb 4000 07:48:54 GT; max-Àge=5; path=/; domain=x.y; httpont-Îecureexpires=Mon, 29 Feb 2016 0::48:54MT; max-Àge=5; path=/; domain=x.y; httpont-ID=x; expires=Mon, 29 Feb 2016 07:48:54 GMT; max-age=2; pa GMT; max-age=2; path=/; domai
seth=/; domai
secure
cure

View File

@@ -0,0 +1,2 @@
__Hos__Host-ID__Host-IDcure-ID=x; expires=Mon, 32 Feb 2016UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU«ªªªªªªªUUUUUUUost-IDcure-ID=x; expires=Mon, 32 Feb 2016UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU«ªªªªªªªUUUUUUUUUUUUUUUUUUU¥ªUUUUUUUUUUUUUUax-a httponly; ly; sUUUUUUUUUUUU¥ªUÕUUUUUUUUUUUUax-a httponly; ly; secure
secure

View File

@@ -0,0 +1,2 @@
__Hos_ßH6 07:48:54 ÿÿÿÿ; expires=Modict=wget_cookie_:48:54 ÿÿÿy; èttpont-ID=x; expires=MANSI_on, 29 Fgb 2016 07:4 httponly;GANSI_*MT; max-age=httponlySt0ctypeIcE/; domain=x.y; httponly; ly; sec!re
secure

View File

@@ -0,0 +1 @@
=;*=;*=;*=;*=;*=;*=;*==;*=;*=;*=;*=;*=;*=þ;*=;*=;**=;*=;*=;*=;*=;*=;*==;*=;*=;*=;*=;*=;*=;*=;*=;*=;*=;*=;*=;*=;**=;*=;*=;*=;*=;*=;*==;*=;*=;*=;=;*=;*=;*=;*=;*==;*=;*=;*=;*=;*=;*=;*==;*=;*=;*=;*=;*=;*=;*=;*=;*=;*=;*=;*=;*=;*=;*=;**=;*=;*=;*=;*=;*=;*==;*=;*=;*=;*=;*=;*=;*=;*=;*=;*=;*=;*=;*=;**=;*=;*=;*=;^^*=;*=;*==;*=;*;*=;*=;*=secure;**=;*=;*=;*=;*=;*=;*==;*=;*=;*=;*=;*=;*=;*=;*=;*=;*=;*=;*=;*=;**=;*=;*=;*=;*=;*=;*==;*=;*=;*=;=;*=;*=;*=;*=;*=;*=;*==;*=;*=;*=;*=jjj;*=;*=;*=;*=;**=;*=;*=;*=;*=;*=;*==;*=;*=;*=;*=;*=;*=;*=;*=;*=;*=;*=;*=;*=;**=;*=;*=;

View File

@@ -0,0 +1 @@
xdicp="get_cookmagth=/doiimain=~"~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~=ÿÿÿÿÿÿÿxpi…šŒÂÄß<C39F>mainy=

View File

@@ -0,0 +1 @@
xmagth=/; domain=~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~=/; domain==/; domain=th=/; domain=th=/; domain=th=/; domain==/; domain=th=/; domain==/; domain=th=/; domain==/; domain=th=/; domain==/; domain=th=/; domain==/; domain=th=/expires=; domain==/; domain=th=/;omain==/; domain=th=/; domain==/; d=/; domain==/; domain=th=/; domain*=/; domain=h=/; domain==/; domain==/; domain=th=/; domain=th=/; domain=th=/;á domain==/; domain=th=/; ain=th=/; domain==/; domain=th=/expires=; domainy=

View File

@@ -0,0 +1 @@
__Host- max-age=5; paph=/; domai29! 2016 07:48omai__Host-IDn, 29! 2016 07:48:54 GMT; max-age=5K path=/; max-age=RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR5 dge=5; path´/; d4 GMT; m

View File

@@ -0,0 +1,3 @@
__Hos__Host-ID=x; expires=Mottpont-ID=x; exp7:48:54 GMT; mires=Moe
secure
aø-Àge=5;ax-\ge=2ÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇ; xain=x.y; hexpiÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇ; xain=x.y; hcuANT; max-\ge=2ÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇÇires= ly; secure(secure

View File

@@ -0,0 +1 @@
_==x;max-age=4; max-age=8; max-age=8ag; max-age=9; max-age=8; max-age=8ag; ax-age=8ag; max-age=9; max-age=8;àmax-age=8ag; max-age=9; max-age=83; max-agm=9a-x;eage=8_;max-age=4; max-age=8; max-age=8=ag; max-age=9; max-age=83; max-age=9;max-age=8_; mmax-age=9; max-age=83; max-age=9;max-age=8_;max-age=4; max-age=8; max-__Host-ID=:54 GMT; max-age=5K path=/; max-age=5; path=T; max-age=5K path=/; max-age=5; path=/7:48:54 GMT; max-age=5K path=/age=8=

View File

@@ -0,0 +1 @@
vvvvvvvvvvvvvvvmvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvmvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv

View File

@@ -0,0 +1 @@
xcan*='; do*='; omaiainmain*='ma=/; domainman*='; do*='; omaih=/; ådÕomainmain*='; dinexpires=y=

View File

@@ -0,0 +1 @@
_D=_;;;;domain=.;;expires=;;do^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^main=.;mdaion=.;domain=.;(;ddomain=.;domain=.;^;D=_;domain=.;domain=.;_;;domain=.;domain=.;;;domain=.;;domain=.;;domain(.;domainn=.;domain=.;_;Domain=.;(;domain=.=_;domain=.;d<>ž_;.=d;n;omamax-agemain=.;;;domain=.;;expires=;;do^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^main=.;mdaion=.;domain=.;(;domain=.;m.=ndin=.;;;domain=.ain=.;domain=.;om

Some files were not shown because too many files have changed in this diff Show More