getting very close to automatic deploy

This commit is contained in:
Timothy Prepscius 2013-09-01 18:17:08 -04:00
parent 28925e122e
commit 993aaf9119
23 changed files with 163 additions and 57 deletions

96
DEV_BRAIN_DUMP.txt Normal file
View File

@ -0,0 +1,96 @@
This is the document to read if you are going to further the development of Mailiverse.
This is *NOT* the document to read if you just want to install it somewhere.
For the time being I'm going to just write down anything that pops into my head.
* Why is the BouncyCastle package name changed to org.bc?
I don't trust google. Google puts the BouncyCastle package in the system loader of android phones.
This makes it impossible to override. Google may very well have modified the BouncyCastle
package to do the bidding of the NSA.
Hence, I use BouncyCastle from the original source. However I used Eclipse to modify the package name from org.bouncycastle to org.bc.
* What is the fucking deal with the Callbacks, it makes Mailiverse's java code super sucky.
Unfortunately this is because Javascript is super sucky. Javascript, at the time of Mailiverse's coding does not have threads. The Java is cross compiled to Javascript for the web code.
So.
I came up with strings of callbacks.
Let's say you are going to do a set of operations A -> B -> C -> D -> E -> F -> G.
C and F happen to be computationally expensive.
Basically, a chain of callbacks is created, and the C callback is offloaded to a worker frame.
C's result is sent back to the original browser frame and the callbacks continue. Eventually F is offloaded, etc etc.
You may say, well, why didn't you off load *everything* to the worker frame?
Well, because it makes it a pain in the ass to reference objects. You get all of these copies of everything, things still suck. Maybe I should have, I don't know.
So, things still suck. Don't blame me. Blame Sun, Firefox and Google.
The web could have been a beautiful place in which true games/environments run at native c++ compiled speed within a browser. I could write in the language of my choice -- and not compile to fucking javascript. Instead we have stupid people selling stupid shit to other stupid people.
* Ok, so tell me what these projects are, why did you do all this soft linking (ln -s) ?
So, I'm writing for Java & Web & Android. Each one has different screw ups and each one has low level intricacies.
So for instance logging is different on Web and Java.
PBE operations are different from Android to Java to Web.
Accessing web resources is different on Web and Java and Android.
Instead of creating umpteen billion factories, I create a core set of files, Mailiverse.Core, and soft link what I need for each platform.
Mailiverse-GWT references almost all of the Mailiverse.Core. It leaves out things for the server and it redoes some of the encryption routines and logging, etc.
Mailiverse.Mail-WebServer is the apache tomcat server. It references only the things needed to run the server from Mailiverse.Core.
Mailiverse.Web houses the static web files.
* Why did you use Dropbox and S3 to store received mail? Why not from the mail server?
Well, *you* can add the key store if you'd like. It's not that hard to do.
For my particular situation, I wanted to keep the mail server running in my physical proximity.
I wanted the ability to walk over and root the box through shear presence.
But, bandwidth out of my apartment is incredibly low. Wat do?
I worked out a solution where bandwidth to the box is spent authenticating (very very small amount) and handling incoming/outgoing mail (where things can be queued and slower than light speed).
Then files which you need to go light speed are stored on providers (S3/Dropbox) which can handle light speed.
* What sort of costs in S3 am I going to have.
Probably about $0.02 a month per user.
* What happens if mysql crashes and I lose all the backups of logins?
You are screwed. Totally screwed. Make backups to a safe place of the mysql server.

1
HOW_TO_INSTALL.txt Normal file
View File

@ -0,0 +1 @@
coming soon to a theater near you..

View File

@ -1,2 +1,2 @@
M_HOST_PROD=`cat config/hosts/prod` M_HOST_PROD=`cat config/hosts/web_prod`
M_HOST_DEV=`cat config/hosts/dev` M_HOST_DEV=`cat config/hosts/web_dev`

View File

@ -3,8 +3,8 @@ if [ -z "$1" ]; then echo "Must supply server name as argument"; exit 0; fi
dev=$1 dev=$1
echo $dev > primary_dev echo $dev > primary_dev
echo $dev:8080 > tomcat_dev echo mail.$dev > tomcat_dev
echo $dev:8000 > web_dev echo mail.$dev > web_dev
echo $dev > auth_dev echo $dev > auth_dev
echo $dev > dev echo $dev > dev

View File

@ -4,7 +4,7 @@ prod=$1
echo $prod > prod echo $prod > prod
echo $prod > primary_prod echo $prod > primary_prod
echo $prod > web_prod echo mail.$prod > web_prod
echo mail.$prod > tomcat_prod echo mail.$prod > tomcat_prod
echo mail.$prod > auth_prod echo mail.$prod > auth_prod
echo mail.$prod > local_mail_prod echo mail.$prod > local_mail_prod

View File

@ -1 +1 @@
../../../keys/james/final/mail_mailiverse_com.jks ../../../keys/james/final/mail_servers.jks

View File

@ -1,46 +1,10 @@
#user nobody;
worker_processes 1;
error_log ~/run.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;
#pid logs/nginx.pid;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
#log_format main '$remote_addr - $remote_user [$time_local] "$request" '
# '$status $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';
access_log ~/run.log main;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 65;
gzip on;
# HTTPS server
#
server { server {
listen 443; listen 443;
# server_name mail.mailiverse.com; # server_name mail.mailiverse.com;
ssl on; ssl on;
ssl_certificate ~/resources/nginx-ssl.crt; ssl_certificate nginx-ssl.crt;
ssl_certificate_key ~/resources/nginx-ssl.key; ssl_certificate_key nginx-ssl.key;
ssl_session_timeout 5m; ssl_session_timeout 5m;
@ -50,7 +14,7 @@ http {
location / { location / {
autoindex off; autoindex off;
root ~/www/; root /var/local/www/;
}
} }
} }

View File

@ -4,5 +4,5 @@ echo "----------------------------------------------" >> ../run.log
date >> ../run.log date >> ../run.log
echo "----------------------------------------------" >> ../run.log echo "----------------------------------------------" >> ../run.log
/usr/bin/nginx -t -c ~/resources/nginx.conf sudo /etc/init.d/nginx start

View File

@ -1,3 +1,6 @@
cd ~ cd ~
ln -fs deploy/www
ln -fs deploy/resources ln -fs deploy/resources
ln -fs /var/log/nginx/error.log run.log
rsync -avz --delete www/ /var/local/www/

View File

@ -1 +1 @@
/usr/bin/nginx -c ~/resources/nginx.conf -s stop sudo /etc/init.d/nginx stop

8
install/enable-rsync Executable file
View File

@ -0,0 +1,8 @@
if [ -z "$1" ]; then echo "Must supply name"; exit 0; fi
set -x
M_ROOT=`cat requirements/server_root_account`
M_HOST=$1
scp enable-rsync.remote $M_ROOT@$M_HOST:
ssh $M_ROOT@$M_HOST ./enable-rsync.remote

4
install/enable-rsync.remote Executable file
View File

@ -0,0 +1,4 @@
apt-get update
apt-get install rsync

View File

@ -10,4 +10,4 @@ M_HOST=$1
./setup-mysql $M_HOST ./setup-mysql $M_HOST
./setup-tomcat $M_HOST ./setup-tomcat $M_HOST
./setup-web $M_HOST ./setup-web $M_HOST
./enable-rsync $M_HOST

View File

@ -3,5 +3,6 @@ set -x
M_ROOT=`cat requirements/server_root_account` M_ROOT=`cat requirements/server_root_account`
M_HOST=$1 M_HOST=$1
scp sudoers.d-web $M_ROOT@$M_HOST:
scp setup-web.remote $M_ROOT@$M_HOST: scp setup-web.remote $M_ROOT@$M_HOST:
ssh $M_ROOT@$M_HOST ./setup-web.remote ssh $M_ROOT@$M_HOST ./setup-web.remote

View File

@ -1,3 +1,18 @@
set -x set -x
sudo apt-get install nginx --yes sudo apt-get install nginx --yes
cp sudoers.d-web /etc/sudoers.d/web
chmod 0440 /etc/sudoers.d/web
cd /etc/nginx
ln -s /home/web/resources/nginx-ssl.crt
ln -s /home/web/resources/nginx-ssl.key
cd conf.d
ln -s /home/web/resources/nginx.conf
cd /var/local
mkdir www
chown web www

1
install/sudoers.d-web Normal file
View File

@ -0,0 +1 @@
web ALL = (root) NOPASSWD :/etc/init.d/nginx start,/etc/init.d/nginx stop

2
keys/james/clean Executable file
View File

@ -0,0 +1,2 @@
rm mail_servers.*
rm final/mail_servers.*

View File

@ -1,4 +1,4 @@
cp ../mail_mailiverse_com.jks . cp ../mail_servers.jks .
#keytool -import -trustcacerts -alias root -file ../mail_mailiverse_com/AddTrustExternalCARoot.crt -keystore mail_mailiverse_com.jks #keytool -import -trustcacerts -alias root -file ../mail_servers/AddTrustExternalCARoot.crt -keystore mail_servers.jks
keytool -import -trustcacerts -alias POSITIVESSL -file ../mail_mailiverse_com/PositiveSSLCA2.crt -keystore mail_mailiverse_com.jks keytool -import -trustcacerts -alias POSITIVESSL -file ../mail_servers/PositiveSSLCA2.crt -keystore mail_servers.jks
keytool -import -trustcacerts -alias james -file ../mail_mailiverse_com/mail_mailiverse_com.crt -keystore mail_mailiverse_com.jks keytool -import -trustcacerts -alias james -file ../mail_servers/mail_servers.crt -keystore mail_servers.jks

View File

@ -1,2 +1,7 @@
keytool -genkeypair -alias james -keyalg RSA -keysize 2048 -keystore mail_mailiverse_com.jks echo -----------------
keytool -certreq -alias james -keyalg RSA -file mail_mailiverse_com.csr -keystore mail_mailiverse_com.jks echo the default installation uses 'password' for password
echo sorry about that, feel free to modify sources
echo -----------------
keytool -genkeypair -alias james -keyalg RSA -keysize 2048 -keystore mail_servers.jks
keytool -certreq -alias james -keyalg RSA -file mail_servers.csr -keystore mail_servers.jks

View File

@ -0,0 +1,6 @@
if [ -z "$1" ]; then echo "Must supply directory name of result"; exit 0; fi
rm -rf mail_servers
mkdir mail_servers
cp $1/* mail_servers/
mv mail_servers/mail_* mail_servers/mail_servers.crt

View File

@ -1,12 +1,12 @@
set -x set -x
JKSSTORE=mail_mailiverse_com.jks JKSSTORE=mail_servers.jks
cp $JKSSTORE store.jks cp $JKSSTORE store.jks
keytool -importkeystore -srckeystore store.jks -srcstoretype JKS -deststoretype PKCS12 -destkeystore store.p12 keytool -importkeystore -srckeystore store.jks -srcstoretype JKS -deststoretype PKCS12 -destkeystore store.p12
openssl pkcs12 -in store.p12 -nocerts -out store.key openssl pkcs12 -in store.p12 -nocerts -out store.key
openssl rsa -in store.key -out final.key openssl rsa -in store.key -out final.key
cat mail_mailiverse_com.crt > final.crt cat mail_servers.crt > final.crt
cat PositiveSSLCA2.crt >> final.crt cat PositiveSSLCA2.crt >> final.crt
cat AddTrustExternalCARoot.crt >> final.crt cat AddTrustExternalCARoot.crt >> final.crt