mirror of
https://github.com/moparisthebest/wget
synced 2024-07-03 16:38:41 -04:00
b89cda0a77
Squashed Commit from parallel-wget of: b31e6e4 Add support for HTTPS Servers b828a6e Sleep for n seconds before calling Wget Executable 7effa90 Support programatically setting Handler class variables 7e1f4c1 Correct the call to stop_HTTP_Server f616192 Improve error handling when wget executable isn't available 31868fe Split large function to improve readability and extensibility
292 lines
12 KiB
Plaintext
292 lines
12 KiB
Plaintext
2014-01-02 Darshit Shah <darnir@gmail.com>
|
|
* Makefile.am: Add new Test--https.py to list of tests and EXTRA_DIST.
|
|
Also replace all tabs with spaces in file for conformity.
|
|
* Test--https.py: New test to check if Wget works correctly with HTTPS
|
|
servers
|
|
* HTTPServer.py: Import new modules for use in HTTPS Servers
|
|
(HTTPSServer): New class that generates a SSL-wrapped socket for use in a
|
|
HTTPS Server.
|
|
(HTTPSd): HTTPS daemon class. Analogous to the HTTPd class
|
|
* WgetTest.py: Define global variables HTTP and HTTPS to reflect Server
|
|
types
|
|
(CommonMethods.exec_wget): Add the protocol information to the URL before
|
|
passing it to wget
|
|
(HTTPTest.__init__): Edit syntax. The servers variable now accepts a list of
|
|
servers defined by their type. E.g. HTTP, HTTPS.
|
|
(HTTPTest.Server_setup): Reflect change in type of variable servers.
|
|
However, we maintin the value of self.servers to allow most of the code to
|
|
remain unchanged.
|
|
(HTTPTest.init_HTTPS_Server): Initialize a HTTPS Server
|
|
* Test-Parallel-Proto.py: Edit to reflect slight change in Test Fiel Syntax.
|
|
* Test-Proto.py: Same
|
|
|
|
2014-01-02 Darshit Shah <darnir@gmail.com>
|
|
|
|
* WgetTest.py (CommonMentods.exec_wget): Wait for n seconds before calling
|
|
the Wget executable.
|
|
|
|
2013-12-27 Darshit Shah <darnir@gmail.com>
|
|
|
|
* WgetTest.py: Add modeline
|
|
(CommonMethods.ServerConf): New pre-test hook that sets
|
|
BaseHTTPRequestHandler class variables in all available servers
|
|
* HTTPServer.py (HTTPd.ServerConf): Call the respective method in the Server
|
|
to set the class variables
|
|
(StoppableHTTPServer.server_sett): Set the handler class variables
|
|
|
|
2013-12-26 Darshit Shah <darnir@gmail.com>
|
|
|
|
* WgetTest.py (HTTPTest.call_test): Correct the call to stop_HTTP_Server.
|
|
|
|
2013-12-25 Darshit Shah <darnir@gmail.com>
|
|
|
|
* WgetTest.py (CommonMehtods.exec_wget): Catch and handle exception if the
|
|
Wget executable is not found at src/wget
|
|
(HTTPTest.call_test): In case of error during execution, remove all existing
|
|
servers before quitting
|
|
|
|
2013-12-15 Darshit Shah <darnir@gmail.com>
|
|
|
|
* WgetTest.py (HTTPTest.HTTP_setup): Rename to Server_setup so it can be
|
|
easily reused for other non-HTTP servers.
|
|
(HTTPTest.__init__): Call Server_setup instead of HTTP_setup
|
|
(HTTPTest.Server_setup): Split into three more functions, that handle
|
|
pre-hooks, test execution and post-hooks respectively.
|
|
(HTTPTest.pre_hook_call): Set up and execute the pre-test hooks. Code split
|
|
from HTTPTest.Server_setup
|
|
(HTTPTest.call_test): Execute wget and log exit code. Code split from
|
|
HTTPTest.Server_setup
|
|
(HTTPTest.post_hook_call): Set up and execute post-test hooks. Code split
|
|
from HTTPTest.Server_setup
|
|
|
|
2013-10-14 Giuseppe Scrivano <gscrivan@redhat.com>
|
|
|
|
* Makefile.am (XFAIL_TESTS): Remove Test--spider-r.py.
|
|
|
|
2013-10-06 Giuseppe Scrivano <gscrivan@redhat.com>
|
|
|
|
* Makefile.am (EXTRA_DIST): Distribute test files.
|
|
|
|
2013-09-16 Darshit Shah <darnir@gmail.com>
|
|
|
|
* README: Update documentation
|
|
|
|
2013-09-14 Darshit Shah <darnir@gmail.com>
|
|
|
|
* HTTPServer.py (StoppableHTTPServer): Define object variable
|
|
request_headers which stores a list of requests received by the server
|
|
(StoppableHTTPServer.get_req_headers): Return the list of Request
|
|
Headers stored by the server
|
|
(_Handler.do_HEAD): Send the Request MEthod string for identification
|
|
(_Handler.do_GET): Same
|
|
(_Handler.__log_request): Log the request in Request_Headers list
|
|
(_Handler.send_head): Make a call to __log_request
|
|
* Test--spider-r.py: Add new list, Request_List, which contains all
|
|
the requests that Wget is expected to send. This will allow for
|
|
fine-grained tests on recursive downloading.
|
|
* WgetTest.py (CommonMethods.FilesCrawled): New Post-Test Hook, that
|
|
ensures that all the expected Files on the server were accessed as
|
|
expected.
|
|
(HTTPTest.stop_HTTP_server): On stopping server, asks it to respond
|
|
with list of all requests it received.
|
|
|
|
2013-09-13 Darshit Shah <darnir@gmail.com>
|
|
|
|
* Test--spider-r.py: Test retrieval in recursive spider mode.
|
|
* Makefile.am: add new file
|
|
|
|
2013-09-13 Darshit Shah <darnir@gmail.com>
|
|
|
|
* HTTPServer.py (_Handler.do_HEAD): If requested path is /, respond
|
|
with /index.html
|
|
(_Handler.do_HEAD): Smartly guess value of Content-Type Header from
|
|
file extension
|
|
(_Handler.guess_type): Use a preset list of extensions and
|
|
Content-Type strings. If the extension matches one in the list, use
|
|
that string, else default to "text/plain"
|
|
|
|
2013-09-13 Darshit Shah <darnir@gmail.com>
|
|
|
|
* WgetTest.py (CommonMethods._replace_substring): New method that will
|
|
replace a substring delimited by {{ }} characters by the value of
|
|
self.<substring> variable
|
|
(CommonMethods.WgetCommands): Use the _replace_substring () call to
|
|
replace the substrings in the the command line.
|
|
(CommonMethods.ServerFiles): Run the _replace_substring () method on
|
|
the File contents too.
|
|
|
|
2013-09-11 Darshit Shah <darnir@gmail.com>
|
|
|
|
* WgetTest.py (CommonMethods.exec_wget): Expect domain_list instead of
|
|
domain.
|
|
(CommonMethods.get_cmd_line): Same. Generate command line by
|
|
prepending to each file it's respective domain string
|
|
(CommonMethods.ServerFiles): Generate file_list and server_rules for
|
|
each Server and set the config details
|
|
(HTTPTest): New named parameter, servers which signifies number of
|
|
servers to spawn
|
|
(HTTPTest.HTTP_setup): This method now takes servers as a new
|
|
parameter. Instead of storing server and domain, we now store
|
|
server_list and domain_list. Each server must be initialized through a
|
|
loop.
|
|
(HTTPTest.stop_HTTP_server): Stop all servers in a loop.
|
|
* Test-Parallel-Proto.py: Prototype test file for multiple servers.
|
|
|
|
2013-09-10 Darshit Shah <darnir@gmail.com>
|
|
|
|
* WgetTest.py (HTTPTest.stop_HTTP_server): With the threaded servers,
|
|
we can simply use the socketserver.shutdown() method to close the
|
|
server instead of sending a QUIT command
|
|
* HTTPServer.py (StoppabelHTTPServer.serve_forever): Delete method. No
|
|
need to override this method anymore.
|
|
(WgetHTTPRequestHandler.do_QUIT): No longer required
|
|
(HTTPd): Rename self.server to self.server_inst to reduce ambiguity
|
|
when referenced from WgetTest
|
|
|
|
2013-09-08 Darshit Shah <darnir@gmail.com>
|
|
|
|
* README (File Structure): Add explanation about various variables
|
|
used consistently across all tests.
|
|
|
|
2013-09-07 Darshit Shah <darnir@gmail.com>
|
|
|
|
* HTTPServer.py: Remove bunch of old code artefacts
|
|
* WgetTest.py: Same
|
|
|
|
2013-09-07 Darshit Shah <darnir@gmail.com>
|
|
|
|
* HTTPServer.py (StoppableHTTPServer.server_conf): Change global
|
|
variable fileSys to an object variable. This is good programming
|
|
practice and required for parallel-wget support.
|
|
(StoppableHTTPServer.server_forever): Edit overridden method to remove
|
|
the global queue variable. No longer required under the new working
|
|
(WgetHTTPRequestHandler.do_QUIT): Don't push fileSys through the queue
|
|
(_Handler): Rename class __Handler to _Handler to match Python's
|
|
encapsulation rules
|
|
(_Handler.do_POST): fileSys is now an object variable of the server
|
|
(_Handler.do_PUT): Same
|
|
(_Handler.send_put): Same
|
|
(_Handler.send_head): Same
|
|
(HTTPd): New class that wraps around the server for Threading
|
|
(create_server): Make new object of HTTPd.
|
|
(spawn_server): Start the thread created through create_server
|
|
(ret_fileSys): Removed method. No longer required.
|
|
* WgetTest.py (HTTPTest.__init__): Don't explicitly set
|
|
self.act_retcode. Instead toggle tests_passed boolean to set the
|
|
correct return code.
|
|
(HTTPTest.HTTP_setup): We no longer call HTTPServer.spawn_server to
|
|
start a new instance of the server.
|
|
(HTTPTest.init_HTTP_server): We no longer call the old
|
|
create_server(), spawn_server() methods. Instead use the new HTTPd
|
|
class interface to create new instances of the server
|
|
(HTTPTest.stop_HTTP_server): Don't ask server to return fileSys.
|
|
|
|
2013-09-07 Darshit Shah <darnir@gmail.com>
|
|
|
|
* Test-Post.py: Test basic functionality for sending HTTP POST
|
|
requests using the --method command
|
|
* Makefile.am: Add new test
|
|
|
|
2013-09-06 Darshit Shah <darnir@gmail.com>
|
|
|
|
* WgetTest.py (CommonMethods.__check_downloaded_files): Print a
|
|
unified diff in case there is a mismatch in the file contents
|
|
|
|
2013-09-06 Darshit Shah <darnir@gmail.com>
|
|
|
|
* HTTPServer.py (WgetHTTPRequestHandler.test_cookies): Comment out the
|
|
old test_cookies code. This is no longer used and was causing problems
|
|
with expected cookies. The code will soon be removed anyways
|
|
* Test-cookie.py: Add new test for basic cookie functionality
|
|
* Test-cookie-401.py: Ensure cookies are saved during a 401 response
|
|
* Test-cookie-expires.py: Ensure that the Expires field is correctly
|
|
handled
|
|
* Test-cookies-domain-mismatch.py: Ensure that mismatched domains are
|
|
handled by Wget
|
|
* Makefile.am: Add the new tests
|
|
|
|
2013-09-06 Darshit Shah <darnir@gmail.com>
|
|
|
|
* README: New section on pending work. Will keep updating this to keep
|
|
track of work that remains to be done on this implementation
|
|
|
|
2013-09-05 Darshit Shah <darnir@gmail.com>
|
|
|
|
* Test-auth-with-content-disposition.py: Add test that ensures Content
|
|
Disposition works alongwith authentication
|
|
* Makefile.am: Add new test
|
|
|
|
2013-09-04 Darshit Shah <darnir@gmail.com>
|
|
|
|
* Test-c-full.py: Test Continue options
|
|
* Makefile.am: Add Test-c-full.py and Test-O
|
|
|
|
2013-09-02 Darshit Shah <darnir@gmail.com>
|
|
|
|
* Makefile.am: Add new Test
|
|
* Test-Head.py: New Test to ensure HEAD requests are handled correctly
|
|
|
|
2013-08-31 Darshit Shah <darnir@gmail.com>
|
|
|
|
* README: Explain that TEST_NAME needs to be unique
|
|
* Test-auth-no-challenge.py: Edit non-unique TEST_NAME
|
|
|
|
2013-08-31 Darshit Shah <darnir@gmail.com>
|
|
|
|
* HTTPTest.py (ServerError): Define new Exception for handling
|
|
internal control flow.
|
|
(StoppableHTTPServer.SendHeader): Simply pass. Do nothing. Adding
|
|
functionality here seems to crash for no apparent reason.
|
|
(stoppableHTTPServer.send_cust_headers): Minor optimization. No need
|
|
for extra variable.
|
|
(__Handler.Response): Handle explicit Response Code Rules
|
|
(__Handler.Authentication): Handle Authentication rules
|
|
(__Handler.handle_auth): Actual worker method for authentication
|
|
(__Handler.ExpectHeader): Ensure Expected Headers are received
|
|
(__Handler.RejectHeader): Ensure Blacklisted Headers are NOT received
|
|
(__Handler.send_HEAD): Dynamically call server rule functions based on
|
|
the self.rules list. This feature will later be added to POST/PUT, etc
|
|
|
|
2013-08-31 Darshit Shah <darnir@gmail.com>
|
|
|
|
* WgetTest.py: Remove import module defaultdict.
|
|
(CommonMethods.get_server_rules): server_rules should be a dict, not a
|
|
defaultdict (list).
|
|
* HTTPServer.py (WgetHTTPRequestHandler.get_rule_list): If rule does
|
|
not exist, return None. Not an emppty list.
|
|
(WgetHTTPRequestHandler.test_cookies): Rule variable is not a list
|
|
(__Handler.send_cust_headers): Same
|
|
(__Handler.custom_response): Same
|
|
(__Handler.is_authorized): Same
|
|
(__Handler.expect_headers): Same
|
|
(__Handler.reject_headers): Same
|
|
|
|
2013-08-31 Darshit Shah <darnir@gmail.com>
|
|
|
|
* README: (newfile) Simple help / instructions about using the Test
|
|
Environment.
|
|
* Makefile.am: (newfile) Makefile for the Test Environment. Uses the
|
|
Automake Parallel Test Harness
|
|
* WgetTest.py: (newfile) Base module that executes the Test.
|
|
* HTTPServer.py: (newfile) Contains the custom HTTP Server for the
|
|
Test Environment. Creates an instance of http.server in Python3.
|
|
* FTPServer.py: (newfile) Overrides methods from pyftpdlib for use in
|
|
the Test Environment. ** Work under progress **.
|
|
* ColourTerm.py: (newfile) A custom module to output coloured text to
|
|
the terminal. Known to work on POSIX shells.
|
|
* Test-Proto.py: (newfile) A prototype Test File. This should be
|
|
copied when writing a new Test Case.
|
|
* Test-Content-disposition-2.py: Test Content Disposition clobbering
|
|
* Test-Content-disposition.py: Test Content Disposition Headers
|
|
* Test-O.py: Test Output filename command
|
|
* Test-auth-basic-fail.py: Test returncode on auth failure
|
|
* Test-auth-basic.py: Test Basic Auth negotiation
|
|
* Test-auth-both.py: Test handling of Multiple auth providers. This
|
|
test currently fails.
|
|
* Test-auth-digest.py: Test Digest Auth Negotiation
|
|
* Test-auth-no-challenge-url.py: Ensure --auth-no-challenge is handled
|
|
when auth details are in-URL.
|
|
* Test-auth-no-challenge.py: Ensure --auth-no-challenge is honoured
|
|
* Test-auth-retcode.py: Ensure correct return code after 403 Forbidden
|
|
response.
|