Add http_upload implementation compatible with prosody's mod_http_upload_external

This commit is contained in:
Travis Burtrum 2017-01-19 23:16:16 -05:00
parent 59b672a08d
commit 0def21ee84
4 changed files with 224 additions and 3 deletions

28
http_upload.sh Executable file
View File

@ -0,0 +1,28 @@
#!/bin/sh
set -e
# export these from another file, or set them here if you are lazy
# http_upload_url='https://example.com/up/'
# http_upload_hmac_key='this is your secret string'
[ -z "$http_upload_url" ] && echo "variable http_upload_url must be set, exiting..." 1>&2 && exit 1
[ -z "$http_upload_hmac_key" ] && echo "variable http_upload_hmac_key must be set, exiting..." 1>&2 && exit 1
[ -z "$http_upload_file_size_limit" ] && http_upload_file_size_limit=$((100 * 1024 * 1024)) # bytes, default to 100 * 1024 * 1024 = 100 MB
file_to_upload="$1"
base_name="$(basename "$file_to_upload")"
file_size="$(stat -c %s "$file_to_upload")"
[ $file_size -gt $http_upload_file_size_limit ] && echo "file size $file_size greater than limit of $http_upload_file_size_limit, exiting..." 1>&2 && exit 1
uuid="$(uuidgen 2>/dev/null || cat /proc/sys/kernel/random/uuid 2>/dev/null || cat /compat/linux/proc/sys/kernel/random/uuid)"
hmac_secret="$(echo -n "${uuid}/${base_name} $file_size" | openssl dgst -sha256 -hmac "$http_upload_hmac_key" -r | awk '{ print $1 }')"
get_url="${http_upload_url}${uuid}/${base_name}"
curl -f -T "$file_to_upload" "${get_url}?v=${hmac_secret}"
echo "$get_url"

172
nginx_http_upload.php Normal file
View File

@ -0,0 +1,172 @@
<?php
/*
PHP script to handle file uploads and downloads for Prosody's mod_http_upload_external
Tested with Nginx 1.11.8 and PHP 7.0.14
** Why this script?
This script only allows uploads that have been authorized by mod_http_upload_external. It
does allow the file name to be specified by the user which can be dangerous but this makes
every attempt to ensure file names are safe. It does this by only allowing one folder deep
(only once instance of /) with the folder being a valid uuid and the name only containing
these characters [A-Za-z0-9\-\._] which *should* be safe on any filesystem.
With that said, I do not consider myself a PHP developer, and at the time of writing, this
code has had no external review. Use it at your own risk. I make no claims that this code
is secure.
** How to use?
Put this file someplace the nginx user can access it, it doesn't need to be in any web root.
It is YOUR responsibility to ensure the upload directory is not executable by the web server
in any way, for example mod_php on apache happily executes *.php and *.php3 and a ton of other
things unless you stop it. You must ensure it's a pure data directory that cannot be executed.
In nginx, set your configuration similar to:
location ^~ /up {
limit_except GET HEAD PUT {
deny all;
}
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME /path/to/nginx_http_upload.php;
fastcgi_param DOCUMENT_ROOT /var/www;
fastcgi_param HTTP_UPLOAD_CONFIG_STORE_DIR /var/www/up/;
fastcgi_param HTTP_UPLOAD_CONFIG_WEB_ROOT /up/;
fastcgi_param HTTP_UPLOAD_CONFIG_SECRET "this is your secret string";
if (\$request_method = PUT) {
fastcgi_pass unix:/var/run/php-fpm.sock;
}
}
In Prosody set:
http_upload_external_base_url = "https://example.com/up/"
http_upload_external_secret = "this is your secret string"
** License
(C) 2016 Matthew Wild <mwild1@gmail.com>
(C) 2016 Travis Burtrum <admin@moparisthebest.com>
Permission is hereby granted, free of charge, to any person obtaining a copy of this software
and associated documentation files (the "Software"), to deal in the Software without restriction,
including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial
portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
/*\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\*/
/* CONFIGURATION OPTIONS */
/*\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\*/
/* Change this to an absolute path to a directory that is writable by your web server and available at $CONFIG_WEB_ROOT below (like '/var/www/up/') */
$CONFIG_STORE_DIR = $_SERVER['HTTP_UPLOAD_CONFIG_STORE_DIR'];
/* Change this to a web root pointing to the $CONFIG_STORE_DIR above (like '/up/') */
$CONFIG_WEB_ROOT = $_SERVER['HTTP_UPLOAD_CONFIG_WEB_ROOT'];
/* This must be the same as 'http_upload_external_secret' that you set in Prosody's config file */
$CONFIG_SECRET = $_SERVER['HTTP_UPLOAD_CONFIG_SECRET'];
/* For people who need options to tweak that they don't understand... here you are */
$CONFIG_CHUNK_SIZE = 4096;
/*\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\*/
/* END OF CONFIGURATION */
/*\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\*/
/* Do not edit below this line unless you know what you are doing (spoiler: nobody does) */
$upload_file_name = substr($_SERVER['PHP_SELF'], strlen($CONFIG_WEB_ROOT));
$store_file_name = $CONFIG_STORE_DIR . $upload_file_name;
$request_method = $_SERVER['REQUEST_METHOD'];
if(array_key_exists('v', $_GET) === TRUE && $request_method === 'PUT') {
$upload_file_size = $_SERVER['CONTENT_LENGTH'];
$upload_token = $_GET['v'];
$calculated_token = hash_hmac('sha256', "$upload_file_name $upload_file_size", $CONFIG_SECRET);
// hash_equals compares in constant time, if your version doesn't have it, look for replacement here:
// https://secure.php.net/manual/en/function.hash-equals.php
if(!hash_equals($calculated_token, $upload_token)) {
header('HTTP/1.0 403 Forbidden');
exit;
}
// validate file name
// should only have one /
if(substr_count($upload_file_name, '/') > 1) {
header('HTTP/1.0 403 Forbidden');
exit;
}
$dir_file = explode('/', $upload_file_name);
$uuid = $dir_file[0];
$file = $dir_file[1];
// validate uuid
if (!preg_match('/^[0-9A-F]{8}-[0-9A-F]{4}-4[0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$/i', $uuid)) {
header('HTTP/1.0 403 Forbidden');
exit;
}
// validate filename, can be a little less strict
if (!preg_match('/^[A-Za-z0-9\-\._]+$/', $file)) {
header('HTTP/1.0 403 Forbidden');
exit;
}
// safe to create uuid folder now
// ensure the single directory is created, with only web server having access
@mkdir($CONFIG_STORE_DIR . $uuid, 0700);
/* Open a file for writing */
$store_file = @fopen($store_file_name, 'x');
// file already exists
if($store_file === FALSE) {
header('HTTP/1.0 409 Conflict');
exit;
}
// now that the file has been created, ensure whole file is under our $CONFIG_STORE_DIR
// this really should be impossible due to our regex's above, but no harm in being extra
// paranoid before we write to a file
$real_store_file_name = realpath($store_file_name);
if ($real_store_file_name === false || strpos($real_store_file_name, $CONFIG_STORE_DIR) !== 0) {
header('HTTP/1.0 403 Forbidden');
exit;
}
/* PUT data comes in on the stdin stream */
$incoming_data = fopen('php://input', 'r');
/* Read the data a chunk at a time and write to the file */
while ($data = fread($incoming_data, $CONFIG_CHUNK_SIZE)) {
fwrite($store_file, $data);
}
/* Close the streams */
fclose($incoming_data);
fclose($store_file);
} else {
header('HTTP/1.0 400 Bad Request');
}
exit;

View File

@ -13,9 +13,10 @@ set -e # exit on error
[ -z "$puush_api_key" ] && export puush_api_key='' # find API key here: http://puush.me/account/settings
[ -z "$imgur_api_key" ] && export imgur_api_key='486690f872c678126a2c09a9e196ce1b' # nabbed from here: https://github.com/dave1010/scripts/blob/master/shoot
[ -z "$imgup_path" ] && export imgup_path='' # example: 'ssh user@host ~/imgup.sh ~/htdocs/s http://host/s png'
[ -z "$http_upload_path" ] && export http_upload_path='' # example: '~/bin/http_upload.sh' without quotes where your required variables are already exported
# if these are empty, go with defaults we know to exist and work without configuration
[ -z "$upload" ] && export upload='imgur' # must be one of 'puush', 'imgur', or 'imgup'
[ -z "$upload" ] && export upload='imgur' # must be one of 'puush', 'imgur', 'imgup', or 'http_upload'
[ -z "$shorturl" ] && export shorturl='' # must be one of 'tinyurl', 'b1tit', or '' (no shorturl)
filename="$1" # if there is no filename to upload, we take a screenshot and upload that
@ -39,6 +40,11 @@ function upload_imgup {
$imgup_path < "$1"
}
function upload_http_upload {
[ -z "$http_upload_path" ] && echo '$imgup_path is empty, cannot upload!' && return
"$http_upload_path" "$1"
}
####################################################################################################################################
# The following are implemented shorturl methods, they take one argument, the long url, and echo the URL the long was shortened to #
####################################################################################################################################

View File

@ -4,7 +4,7 @@ open-screeny.sh
------------
This script takes screenshots, uploads them to a service, optionally shortens the URL, puts it in your clipboard, and opens it in a browser.
Currently supported services are [imgur][1], [puush][2], and imgup.sh (which is a script included with this one that can be called locally or over ssh)
Currently supported services are [imgur][1], [puush][2], http_upload.sh, and imgup.sh (the latter 2 being included in this repo)
There is also an open-source implementation of the [puush server API][5] this should work with.
Currently supported URL shortening services are [tinyurl][3] and [b1t.it][4]
@ -15,10 +15,22 @@ See open-screeny.sh for the enviromental variables that need set for certain ser
You probably want to bind this to 'Print-Screen' or some other button combination for the best ease-of-use.
http_upload.sh
------------
This script uploads the file given in the first argument to a special http_upload script on the server, compatible with prosody's [mod_http_upload_external][6].
One example PHP script for the server included here as nginx_http_upload.php in this repository, another is included with the prosody module.
Mainly meant to be used for images from scripts like open-screeny.sh, it can really be used for any file uploads, nothing is format specific.
Required dependencies are openssl, curl, and standard unix utilities stat, awk, and basename
imgup.sh
------------
This script reads a file from stdin, and moves it to a certain directory with the shortest name possible that doesn't conflict based on the sha1sum, then echos the URL the file will be available at.
This can be called locally or over ssh for ease of use.
Mainly meant to be used for images from scripts like open-screeny.sh, it can really be used for any file uploads, nothing is format specific.
Required dependencies are sha1sum, and standard unix utilities tee and cut
@ -27,8 +39,11 @@ Licensing
------------
Seriously? It consists of trivial shell scripts using standard unix utilities. If you must have a license, take your pick of any GNU, Apache, BSD, or MIT license, any version. If you need to modify this code though, you should contribute back to it, if just to be nice.
nginx_http_upload.php is licensed seperately as mentioned at the top of the file since it's a derived work.
[1]: http://imgur.com/
[2]: http://puush.me/
[3]: https://tinyurl.com/
[4]: http://b1t.it/
[5]: https://github.com/Hidendra/puush-api
[6]: https://modules.prosody.im/mod_http_upload_external.html