wget/testenv/ChangeLog

2014-03-13  Zihang Chen  <chsc4698@gmail.com>

	* base_test.py:
	(CommonMethods): Rename to BaseTest.
	(BaseTest): Implement __init__ method where the class-wide variables are
	initialized. Also variable names like `xxx_list` is renamed to its plural
	form, e.g. `server_list` => `servers`.
	(BaseTest.init_test_env): Remove name argument due to its unnecessarity.
	(BaseTest.get_test_dir): Because the path of the test directory is needed
	in multiple methods, this method is implemented.
	(BaseTest.get_domain_addr): Rewrite the return statement utilizing str
	formatting (which is more Pythonic).
	(BaseTest.get_cmd_line): Rename to gen_cmd_line. Change the variables with
	capitcal characters to lower ones. Also, the nested for loop is rewritten
	to a plain loop using the zip function.
	(BaseTest.__gen_local_filesys): Rename to gen_local_fs_snapshot. Move to
	ExpectedFiles in conf/expected_files.py and is marked as a static
	method. Refactor to a less verbose implementation.
	(BaseTest._check_downloaded_files): Rename to __call__ to agree with the
	invocation in test case classes. Move to ExpectedFiles in
	conf/expected_files.py.
	(BaseTest.get_server_rules): Refactor to a more Pythonic form utilizing
	dict.items() and is marked static.
	(BaseTest.stop_server): (new method) an abstract method which should stop
	the currently using servers.
	(BaseTest.instantiate_server_by): (new method) an abstract method which
	should instantiate a server instance according to the given argument.
	(BaseTest.__enter__): (new method) method which initialize the context
	manager
	(BaseTest.__exit__): (new method) method that finilize the context manager
	and deal with the exceptions during the execution of the with statement,
	subclasses can override this method for extensibility
	* http_test.py:
	(HTTPTest.__init__): Add call to super.__init__. Default values of
	pre_hook, test_params, post_hook are set to None to avoid a subtle bug of
	Python. Argument servers is renamed to protocols.
	(HTTPTest.Server_setup): Move to BaseTest and rename to server_setup.
	Calls to pre_hook_call, call_test, post_hook_call are removed.
	(HTTPTest.hook_call, pre_hook_call, call_test, post_hook_call): Move to
	BaseTest for that both HTTP test cases and FTP test cases may use these
	methods.
	(HTTPTest.init_HTTP_Server, init_HTTPS_Server): Merge and rename to
	instantiate_server_by to implement the abstract method in BaseTest.
	(HTTPTest.stop_HTTP_Server): Rename to stop_server to implement the
	abstract method in BaseTest. Also, pull out the part where remaining
	requests are gathered into a new method request_remaining.
	(BaseTest.act_retcode): Rename to ret_code because ExpectedRetCode is
	moved out from BaseTest, so the name act_retcode is actually a bit
	verbose.
	* conf/expected_ret_code.py:
	(ExpectedRetCode.__call__): Rewrite the str into a more readable form.
	* conf/files_crawled.py:
	(FilesCrawled.__call__): Refactor this method into a more Pythonic form
	utilizing the zip function.
	* conf/local_files.py:
	(LocalFiles__call__): Rewrite this method with the recommended with
	statement.
	* conf/server_conf.py:
	(ServerConf.__call__): Rewrite this method due to BaseTest.server_list is
	renamed to BaseTest.servers.
	* conf/server_files.py:
	(ServerFiles.__call__): Refactor the nested for loop into a plain one
	utilizing the zip function.
	* conf/urls.py:
	(URLs): Rename url_list to urls.
	* conf/wget_commands.py:
	(WgetCommands): Rename command_list to commands, rename test_obj.options
	to test_obj.wget_options.
	* Test--https.py, Test-Proto.py, Test-Parallel-Proto.py: Argument servers
	is changed to protocols due to change in the signature of
	HTTPTest.__init__.

2014-03-13  Zihang Chen  <chsc4698@gmail.com>

	* test: (new package) package for test case classes
	* WgetTest.py: Split into test/base_test.py and test/http_test.py.
	* Test-*.py: Optimize the imports according to changes of WgetTest.py

2014-03-13  Zihang Chen  <chsc4698@gmail.com>

	* server: (new package) package for the server classes
	* server.http: (new package) package for HTTP server
	* server.ftp: (new package) package for FTP server
	* HTTPServer.py: Move to server/http/http_server.py. Also change the
	CERTFILE to '../certs/wget-cert.pem'.
	* FTPServer.py: Move to server/ftp/ftp_server.py.
	* WgetTest.py: Optimize import respect to the server classes.

2014-03-13  Zihang Chen  <chsc4698@gmail.com>

	* conf: (new package) package for rule classes and hook methods
	* WgetTest.py:
	(CommonMethods.Authentication): Move to conf/authentication.py.
	(CommonMethods.ExpectHeader): Move to conf/expect_header.py.
	(CommonMethods.RejectHeader): Move to conf/reject_header.py.
	(CommonMethods.Response): Move to conf/response.py.
	(CommonMethods.SendHeader): Move to conf/send_header.py.
	(CommonMethods.ServerFiles): Move to conf/server_files.py.
	(CommonMethods.LocalFiles): Move to conf/local_files.py.
	(CommonMethods.ServerConf): Move to conf/server_conf.py.
	(CommonMethods.WgetCommands): Move to conf/wget_commands.py.
	(CommonMethods.Urls): Move to conf/urls.py.
	(CommonMethods.ExpectedRetcode): Move to conf/expected_retcode.py.
	(CommonMethods.ExpectedFiles): Move to conf/expected_files.py.
	(CommonMethods.FilesCrawled): Move to conf/files_crawled.py.
	(CommonMethods.__check_downloaded_files): Rename to
	_check_downloaded_files, so that the method is callable from outside the
	class.
	(CommomMethods.get_server_rules): Modify so that it utilizes the conf
	package.
	(HTTPTest): Add a method hook_call(configs, name) to reduce duplications
	in pre_hook_call, call_test and post_hook_call utilizing the conf package.
	* conf/hook_sample.py: (new file) sample for hooks
	* conf/rule_sample.py: (new file) sample for rules
	* REAMDE: Update sections about customizing rules and hooks.

2014-03-13  Zihang Chen  <chsc4698@gmail.com>

	* exc: (new package) package for miscellaneous exceptions
	* WgetTest.py: Move TestFailed to exc/test_failed.py.

2014-03-13  Zihang Chen  <chsc4698@gmail.com>

	* Test-Proto.py: Fix a typo (line 71: server to servers).

2014-03-13  Zihang Chen  <chsc4698@gmail.com>

	* WgetTest.py: Move WgetFile to package misc.
	* README: Modify documentation respect to WgetFile.
	* Test-*.py: Optimize imports about WgetFile.

2014-03-13  Zihang Chen  <chsc4698@gmail.com>

	* misc: (new package) package for miscellaneous modules
	* ColourTerm.py: Move to package misc and rename to colour_terminal.py,
	add print_color functions to reduce the use of string literals like
	"BLUE", "RED" etc.
	* WgetTest.py:
	(CommonMethods.Server_setup): Change invocation to printer to print_blue.
	(CommonMethods.FilesCrawled): Change invocation to printer to print_red.
	(HTTPTest.__init__): Change invocations to printer to print_red and
	print_green respectively.

2014-01-02  Darshit Shah  <darnir@gmail.com>
	* Makefile.am: Add new Test--https.py to list of tests and EXTRA_DIST.
	Also replace all tabs with spaces in file for conformity.
	* Test--https.py: New test to check if Wget works correctly with HTTPS
	servers
	* HTTPServer.py: Import new modules for use in HTTPS Servers
	(HTTPSServer): New class that generates a SSL-wrapped socket for use in a
	HTTPS Server.
	(HTTPSd): HTTPS daemon class. Analogous to the HTTPd class
	* WgetTest.py: Define global variables HTTP and HTTPS to reflect Server
	types
	(CommonMethods.exec_wget): Add the protocol information to the URL before
	passing it to wget
	(HTTPTest.__init__): Edit syntax. The servers variable now accepts a list of
	servers defined by their type. E.g. HTTP, HTTPS.
	(HTTPTest.Server_setup): Reflect change in type of variable servers.
	However, we maintin the value of self.servers to allow most of the code to
	remain unchanged.
	(HTTPTest.init_HTTPS_Server): Initialize a HTTPS Server
	* Test-Parallel-Proto.py: Edit to reflect slight change in Test Fiel Syntax.
	* Test-Proto.py: Same

2014-01-02  Darshit Shah  <darnir@gmail.com>

	* WgetTest.py (CommonMentods.exec_wget): Wait for n seconds before calling
	the Wget executable.

2013-12-27  Darshit Shah  <darnir@gmail.com>

	* WgetTest.py: Add modeline
	(CommonMethods.ServerConf): New pre-test hook that sets
	BaseHTTPRequestHandler class variables in all available servers
	* HTTPServer.py (HTTPd.ServerConf): Call the respective method in the Server
	to set the class variables
	(StoppableHTTPServer.server_sett): Set the handler class variables

2013-12-26  Darshit Shah  <darnir@gmail.com>

	* WgetTest.py (HTTPTest.call_test): Correct the call to stop_HTTP_Server.

2013-12-25  Darshit Shah  <darnir@gmail.com>

	* WgetTest.py (CommonMehtods.exec_wget): Catch and handle exception if the
	Wget executable is not found at src/wget
	(HTTPTest.call_test): In case of error during execution, remove all existing
	servers before quitting

2013-12-15  Darshit Shah  <darnir@gmail.com>

	* WgetTest.py (HTTPTest.HTTP_setup): Rename to Server_setup so it can be
	easily reused for other non-HTTP servers.
	(HTTPTest.__init__): Call Server_setup instead of HTTP_setup
	(HTTPTest.Server_setup): Split into three more functions, that handle
	pre-hooks, test execution and post-hooks respectively.
	(HTTPTest.pre_hook_call): Set up and execute the pre-test hooks. Code split
	from HTTPTest.Server_setup
	(HTTPTest.call_test): Execute wget and log exit code. Code split from
	HTTPTest.Server_setup
	(HTTPTest.post_hook_call): Set up and execute post-test hooks. Code split
	from HTTPTest.Server_setup

2013-10-14  Giuseppe Scrivano  <gscrivan@redhat.com>

	* Makefile.am (XFAIL_TESTS): Remove Test--spider-r.py.

2013-10-06  Giuseppe Scrivano  <gscrivan@redhat.com>

	* Makefile.am (EXTRA_DIST): Distribute test files.

2013-09-16  Darshit Shah  <darnir@gmail.com>

	* README: Update documentation

2013-09-14  Darshit Shah  <darnir@gmail.com>

	* HTTPServer.py (StoppableHTTPServer): Define object variable
	request_headers which stores a list of requests received by the server
	(StoppableHTTPServer.get_req_headers): Return the list of Request
	Headers stored by the server
	(_Handler.do_HEAD): Send the Request MEthod string for identification
	(_Handler.do_GET): Same
	(_Handler.__log_request): Log the request in Request_Headers list
	(_Handler.send_head): Make a call to __log_request
	* Test--spider-r.py: Add new list, Request_List, which contains all
	the requests that Wget is expected to send. This will allow for
	fine-grained tests on recursive downloading.
	* WgetTest.py (CommonMethods.FilesCrawled): New Post-Test Hook, that
	ensures that all the expected Files on the server were accessed as
	expected.
	(HTTPTest.stop_HTTP_server): On stopping server, asks it to respond
	with list of all requests it received.

2013-09-13  Darshit Shah  <darnir@gmail.com>

	* Test--spider-r.py: Test retrieval in recursive spider mode.
	* Makefile.am: add new file

2013-09-13  Darshit Shah  <darnir@gmail.com>

	* HTTPServer.py (_Handler.do_HEAD): If requested path is /, respond
	with /index.html
	(_Handler.do_HEAD): Smartly guess value of Content-Type Header from
	file extension
	(_Handler.guess_type): Use a preset list of extensions and
	Content-Type strings. If the extension matches one in the list, use
	that string, else default to "text/plain"

2013-09-13  Darshit Shah  <darnir@gmail.com>

	* WgetTest.py (CommonMethods._replace_substring): New method that will
	replace a substring delimited by {{ }} characters by the value of
	self.<substring> variable
	(CommonMethods.WgetCommands): Use the _replace_substring () call to
	replace the substrings in the the command line.
	(CommonMethods.ServerFiles): Run the _replace_substring () method on
	the File contents too.

2013-09-11  Darshit Shah  <darnir@gmail.com>

	* WgetTest.py (CommonMethods.exec_wget): Expect domain_list instead of
	domain.
	(CommonMethods.get_cmd_line): Same. Generate command line by
	prepending to each file it's respective domain string
	(CommonMethods.ServerFiles): Generate file_list and server_rules for
	each Server and set the config details
	(HTTPTest): New named parameter, servers which signifies number of
	servers to spawn
	(HTTPTest.HTTP_setup): This method now takes servers as a new
	parameter. Instead of storing server and domain, we now store
	server_list and domain_list. Each server must be initialized through a
	loop.
	(HTTPTest.stop_HTTP_server): Stop all servers in a loop.
	* Test-Parallel-Proto.py: Prototype test file for multiple servers.

2013-09-10  Darshit Shah  <darnir@gmail.com>

	* WgetTest.py (HTTPTest.stop_HTTP_server): With the threaded servers,
	we can simply use the socketserver.shutdown() method to close the
	server instead of sending a QUIT command
	* HTTPServer.py (StoppabelHTTPServer.serve_forever): Delete method. No
	need to override this method anymore.
	(WgetHTTPRequestHandler.do_QUIT): No longer required
	(HTTPd): Rename self.server to self.server_inst to reduce ambiguity
	when referenced from WgetTest

2013-09-08  Darshit Shah  <darnir@gmail.com>

	* README (File Structure): Add explanation about various variables
	used consistently across all tests.

2013-09-07  Darshit Shah  <darnir@gmail.com>

	* HTTPServer.py: Remove bunch of old code artefacts
	* WgetTest.py: Same

2013-09-07  Darshit Shah  <darnir@gmail.com>

	* HTTPServer.py (StoppableHTTPServer.server_conf): Change global
	variable fileSys to an object variable. This is good programming
	practice and required for parallel-wget support.
	(StoppableHTTPServer.server_forever): Edit overridden method to remove
	the global queue variable. No longer required under the new working
	(WgetHTTPRequestHandler.do_QUIT): Don't push fileSys through the queue
	(_Handler): Rename class __Handler to _Handler to match Python's
	encapsulation rules
	(_Handler.do_POST): fileSys is now an object variable of the server
	(_Handler.do_PUT): Same
	(_Handler.send_put): Same
	(_Handler.send_head): Same
	(HTTPd): New class that wraps around the server for Threading
	(create_server): Make new object of HTTPd.
	(spawn_server): Start the thread created through create_server
	(ret_fileSys): Removed method. No longer required.
	* WgetTest.py (HTTPTest.__init__): Don't explicitly set
	self.act_retcode. Instead toggle tests_passed boolean to set the
	correct return code.
	(HTTPTest.HTTP_setup): We no longer call HTTPServer.spawn_server to
	start a new instance of the server.
	(HTTPTest.init_HTTP_server): We no longer call the old
	create_server(), spawn_server() methods. Instead use the new HTTPd
	class interface to create new instances of the server
	(HTTPTest.stop_HTTP_server): Don't ask server to return fileSys.

2013-09-07  Darshit Shah  <darnir@gmail.com>

	* Test-Post.py: Test basic functionality for sending HTTP POST
	requests using the --method command
	* Makefile.am: Add new test

2013-09-06  Darshit Shah  <darnir@gmail.com>

	* WgetTest.py (CommonMethods.__check_downloaded_files): Print a
	unified diff in case there is a mismatch in the file contents

2013-09-06  Darshit Shah  <darnir@gmail.com>

	* HTTPServer.py (WgetHTTPRequestHandler.test_cookies): Comment out the
	old test_cookies code. This is no longer used and was causing problems
	with expected cookies. The code will soon be removed anyways
	* Test-cookie.py: Add new test for basic cookie functionality
	* Test-cookie-401.py: Ensure cookies are saved during a 401 response
	* Test-cookie-expires.py: Ensure that the Expires field is correctly
	handled
	* Test-cookies-domain-mismatch.py: Ensure that mismatched domains are
	handled by Wget
	* Makefile.am: Add the new tests

2013-09-06  Darshit Shah  <darnir@gmail.com>

	* README: New section on pending work. Will keep updating this to keep
	track of work that remains to be done on this implementation

2013-09-05  Darshit Shah  <darnir@gmail.com>

	* Test-auth-with-content-disposition.py: Add test that ensures Content
	Disposition works alongwith authentication
	* Makefile.am: Add new test

2013-09-04  Darshit Shah  <darnir@gmail.com>

	* Test-c-full.py: Test Continue options
	* Makefile.am: Add Test-c-full.py and Test-O

2013-09-02  Darshit Shah  <darnir@gmail.com>

	* Makefile.am:	Add new Test
	* Test-Head.py: New Test to ensure HEAD requests are handled correctly

2013-08-31  Darshit Shah  <darnir@gmail.com>

	* README: Explain that TEST_NAME needs to be unique
	* Test-auth-no-challenge.py: Edit non-unique TEST_NAME

2013-08-31  Darshit Shah  <darnir@gmail.com>

	* HTTPTest.py (ServerError): Define new Exception for handling
	internal control flow.
	(StoppableHTTPServer.SendHeader): Simply pass. Do nothing. Adding
	functionality here seems to crash for no apparent reason.
	(stoppableHTTPServer.send_cust_headers): Minor optimization. No need
	for extra variable.
	(__Handler.Response): Handle explicit Response Code Rules
	(__Handler.Authentication): Handle Authentication rules
	(__Handler.handle_auth): Actual worker method for authentication
	(__Handler.ExpectHeader): Ensure Expected Headers are received
	(__Handler.RejectHeader): Ensure Blacklisted Headers are NOT received
	(__Handler.send_HEAD): Dynamically call server rule functions based on
	the self.rules list. This feature will later be added to POST/PUT, etc

2013-08-31  Darshit Shah  <darnir@gmail.com>

	* WgetTest.py: Remove import module defaultdict.
	(CommonMethods.get_server_rules): server_rules should be a dict, not a
	defaultdict (list).
	* HTTPServer.py (WgetHTTPRequestHandler.get_rule_list): If rule does
	not exist, return None.	Not an emppty list.
	(WgetHTTPRequestHandler.test_cookies): Rule variable is not a list
	(__Handler.send_cust_headers): Same
	(__Handler.custom_response): Same
	(__Handler.is_authorized): Same
	(__Handler.expect_headers): Same
	(__Handler.reject_headers): Same

2013-08-31  Darshit Shah  <darnir@gmail.com>

	* README: (newfile) Simple help / instructions about using the Test
	Environment.
	* Makefile.am: (newfile) Makefile for the Test Environment. Uses the
	Automake Parallel Test Harness
	* WgetTest.py: (newfile) Base module that executes the Test.
	* HTTPServer.py: (newfile) Contains the custom HTTP Server for the
	Test Environment. Creates an instance of http.server in Python3.
	* FTPServer.py: (newfile) Overrides methods from pyftpdlib for use in
	the Test Environment. ** Work under progress **.
	* ColourTerm.py: (newfile) A custom module to output coloured text to
	the terminal. Known to work on POSIX shells.
	* Test-Proto.py: (newfile) A prototype Test File. This should be
	copied when writing a new Test Case.
	* Test-Content-disposition-2.py: Test Content Disposition clobbering
	* Test-Content-disposition.py: Test Content Disposition Headers
	* Test-O.py: Test Output filename command
	* Test-auth-basic-fail.py: Test returncode on auth failure
	* Test-auth-basic.py: Test Basic Auth negotiation
	* Test-auth-both.py: Test handling of Multiple auth providers. This
	test currently fails.
	* Test-auth-digest.py: Test Digest Auth Negotiation
	* Test-auth-no-challenge-url.py: Ensure --auth-no-challenge is handled
	when auth details are in-URL.
	* Test-auth-no-challenge.py: Ensure --auth-no-challenge is honoured
	* Test-auth-retcode.py: Ensure correct return code after 403 Forbidden
	response.