moparisthebest/curl
1
0
Fork 0
mirror of https://github.com/moparisthebest/curl synced 2024-08-13 17:03:50 -04:00
Code Issues Releases Wiki Activity

FAQ: Update FTP directory listing section for MLSD command

Browse Source 
Explain how some FTP servers support the machine readable listing
format MLSD from RFC 3659 and compare it to LIST.

Ref: https://github.com/curl/curl/issues/906
This commit is contained in:
Jay Satiro 2016-07-09 03:05:55 -04:00
parent 7c9cfd6c51
commit f9eed596a3
1 changed files with 20 additions and 10 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

30
docs/FAQ
View File

@ -1402,17 +1402,27 @@ FAQ
CURLOPT_CUSTOMREQUEST to alter what exact listing command libcurl would use
to list the files.
The follow-up question that tend to follow the previous one, is how a
program is supposed to parse the directory listing. How does it know what's
a file and what's a dir and what's a symlink etc. The harsh reality is that
FTP provides no such fine and easy-to-parse output. The output format FTP
servers respond to LIST commands are entirely at the server's own liking and
the NLST output doesn't reveal any types and in many cases don't even
include all the directory entries. Also, both LIST and NLST tend to hide
unix-style hidden files (those that start with a dot) by default so you need
to do "LIST -a" or similar to see them.
The follow-up question tends to be how is a program supposed to parse the
directory listing. How does it know what's a file and what's a dir and what's
a symlink etc. If the FTP server supports the MLSD command then it will
return data in a machine-readable format that can be parsed for type. The
types are specified by RFC3659 section 7.5.1. If MLSD is not supported then
you have to work with what you're given. The LIST output format is entirely
at the server's own liking and the NLST output doesn't reveal any types and
in many cases doesn't even include all the directory entries. Also, both LIST
and NLST tend to hide unix-style hidden files (those that start with a dot)
by default so you need to do "LIST -a" or similar to see them.
The application thus needs to parse the LIST output. One such existing
Example - List only directories.
ftp.funet.fi supports MLSD and ftp.kernel.org does not:
curl -s ftp.funet.fi/pub/ -X MLSD | \
perl -lne 'print if s/(?:^|;)type=dir;[^ ]+ (.+)$/$1/'
curl -s ftp.kernel.org/pub/linux/kernel/ | \
perl -lne 'print if s/^d[-rwx]{9}(?: +[^ ]+){7} (.+)$/$1/'
If you need to parse LIST output in libcurl one such existing
list parser is available at https://cr.yp.to/ftpparse.html Versions of
libcurl since 7.21.0 also provide the ability to specify a wildcard to
download multiple files from one FTP directory.