getting very close to automatic deploy

DEV_BRAIN_DUMP.txt

@@ -0,0 +1,96 @@
+This is the document to read if you are going to further the development of Mailiverse.
+This is *NOT* the document to read if you just want to install it somewhere.
+
+For the time being I'm going to just write down anything that pops into my head.
+
+
+
+* Why is the BouncyCastle package name changed to org.bc?
+
+I don't trust google.  Google puts the BouncyCastle package in the system loader of android phones.
+This makes it impossible to override.  Google may very well have modified the BouncyCastle
+package to do the bidding of the NSA.
+
+Hence, I use BouncyCastle from the original source.  However I used Eclipse to modify the package name from org.bouncycastle to org.bc.
+
+
+
+
+* What is the fucking deal with the Callbacks, it makes Mailiverse's java code super sucky.
+
+Unfortunately this is because Javascript is super sucky.  Javascript, at the time of Mailiverse's coding does not have threads.  The Java is cross compiled to Javascript for the web code.
+
+So.
+
+I came up with strings of callbacks.
+Let's say you are going to do a set of operations A -> B -> C -> D -> E -> F -> G.
+C and F happen to be computationally expensive.
+
+Basically, a chain of callbacks is created, and the C callback is offloaded to a worker frame.
+C's result is sent back to the original browser frame and the callbacks continue.  Eventually F is offloaded, etc etc.
+
+You may say, well, why didn't you off load *everything* to the worker frame?
+Well, because it makes it a pain in the ass to reference objects.  You get all of these copies of everything, things still suck.  Maybe I should have, I don't know.
+
+
+
+So, things still suck.  Don't blame me.  Blame Sun, Firefox and Google.
+
+The web could have been a beautiful place in which true games/environments run at native c++ compiled speed within a browser.  I could write in the language of my choice -- and not compile to fucking javascript.  Instead we have stupid people selling stupid shit to other stupid people.
+
+
+
+
+* Ok, so tell me what these projects are, why did you do all this soft linking (ln -s) ?
+
+So, I'm writing for Java & Web & Android.  Each one has different screw ups and each one has low level intricacies.
+
+So for instance logging is different on Web and Java.
+PBE operations are different from Android to Java to Web.
+Accessing web resources is different on Web and Java and Android.
+
+Instead of creating umpteen billion factories, I create a core set of files, Mailiverse.Core, and soft link what I need for each platform.
+
+Mailiverse-GWT references almost all of the Mailiverse.Core.  It leaves out things for the server and it redoes some of the encryption routines and logging, etc.
+
+Mailiverse.Mail-WebServer is the apache tomcat server.  It references only the things needed to run the server from Mailiverse.Core.
+
+Mailiverse.Web houses the static web files.
+
+
+
+
+
+* Why did you use Dropbox and S3 to store received mail?  Why not from the mail server?
+
+Well, *you* can add the key store if you'd like.  It's not that hard to do.
+For my particular situation, I wanted to keep the mail server running in my physical proximity.
+I wanted the ability to walk over and root the box through shear presence.
+
+But, bandwidth out of my apartment is incredibly low.  Wat do?
+
+I worked out a solution where bandwidth to the box is spent authenticating (very very small amount) and handling incoming/outgoing mail (where things can be queued and slower than light speed).
+
+Then files which you need to go light speed are stored on providers (S3/Dropbox) which can handle light speed.
+
+
+
+
+* What sort of costs in S3 am I going to have.
+
+Probably about $0.02 a month per user. 
+
+
+
+
+
+* What happens if mysql crashes and I lose all the backups of logins?
+
+You are screwed.  Totally screwed.  Make backups to a safe place of the mysql server.
+
+
+
+
+
+
+

HOW_TO_INSTALL.txt

@@ -0,0 +1 @@
+coming soon to a theater near you..

deploy/config.hosts

@@ -1,2 +1,2 @@
-M_HOST_PROD=`cat config/hosts/prod`
+M_HOST_PROD=`cat config/hosts/web_prod`
-M_HOST_DEV=`cat config/hosts/dev`
+M_HOST_DEV=`cat config/hosts/web_dev`

deploy/config/hosts/make-dev

@@ -3,8 +3,8 @@ if [ -z "$1" ]; then echo "Must supply server name as argument"; exit 0; fi
 dev=$1
 
 echo $dev > primary_dev
-echo $dev:8080 > tomcat_dev
+echo mail.$dev > tomcat_dev
-echo $dev:8000 > web_dev
+echo mail.$dev > web_dev
 echo $dev > auth_dev
 echo $dev > dev
 

deploy/config/hosts/make-prod

@@ -4,7 +4,7 @@ prod=$1
 
 echo $prod > prod
 echo $prod > primary_prod
-echo $prod > web_prod
+echo mail.$prod > web_prod
 echo mail.$prod > tomcat_prod
 echo mail.$prod > auth_prod
 echo mail.$prod > local_mail_prod

deploy/james/apache-james-conf/keystore

@@ -1 +1 @@
-../../../keys/james/final/mail_mailiverse_com.jks
+../../../keys/james/final/mail_servers.jks

deploy/web/resources/nginx.conf

@@ -1,46 +1,10 @@
-
-#user  nobody;
-worker_processes  1;
-
-error_log  ~/run.log;
-#error_log  logs/error.log  notice;
-#error_log  logs/error.log  info;
-
-#pid        logs/nginx.pid;
-
-
-events {
-    worker_connections  1024;
-}
-
-
-http {
-    include       mime.types;
-    default_type  application/octet-stream;
-
-    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
-    #                  '$status $body_bytes_sent "$http_referer" '
-    #                  '"$http_user_agent" "$http_x_forwarded_for"';
-
-    access_log  ~/run.log  main;
-
-    sendfile        on;
-    #tcp_nopush     on;
-
-    #keepalive_timeout  0;
-    keepalive_timeout  65;
-
-    gzip  on;
-
-    # HTTPS server
-    #
     server {
         listen       443;
 #        server_name  mail.mailiverse.com;
 
         ssl                  on;
-        ssl_certificate      ~/resources/nginx-ssl.crt;
+        ssl_certificate      nginx-ssl.crt;
-        ssl_certificate_key  ~/resources/nginx-ssl.key;
+        ssl_certificate_key  nginx-ssl.key;
 
         ssl_session_timeout  5m;
 
@@ -50,7 +14,7 @@ http {
 
         location / {
 	    autoindex off;
-	    root   ~/www/;
+	    root   /var/local/www/;
-        }
         }
     }
+

deploy/web/run

@@ -4,5 +4,5 @@ echo "----------------------------------------------" >> ../run.log
 date >> ../run.log
 echo "----------------------------------------------" >> ../run.log
 
-/usr/bin/nginx -t -c ~/resources/nginx.conf
+sudo /etc/init.d/nginx start
 

deploy/web/setup

@@ -1,3 +1,6 @@
 cd ~
+ln -fs deploy/www
 ln -fs deploy/resources
+ln -fs /var/log/nginx/error.log run.log
 
+rsync -avz --delete www/ /var/local/www/

deploy/web/shutdown

@@ -1 +1 @@
-/usr/bin/nginx -c ~/resources/nginx.conf -s stop
+sudo /etc/init.d/nginx stop

install/enable-rsync

@@ -0,0 +1,8 @@
+if [ -z "$1" ]; then echo "Must supply name"; exit 0; fi
+
+set -x
+M_ROOT=`cat requirements/server_root_account`
+M_HOST=$1
+
+scp enable-rsync.remote $M_ROOT@$M_HOST:
+ssh $M_ROOT@$M_HOST ./enable-rsync.remote

install/enable-rsync.remote

@@ -0,0 +1,4 @@
+apt-get update
+apt-get install rsync
+
+

install/setup-server

@@ -10,4 +10,4 @@ M_HOST=$1
 ./setup-mysql $M_HOST
 ./setup-tomcat $M_HOST
 ./setup-web $M_HOST
-
+./enable-rsync $M_HOST

install/setup-web

@@ -3,5 +3,6 @@ set -x
 M_ROOT=`cat requirements/server_root_account`
 M_HOST=$1
 
+scp sudoers.d-web $M_ROOT@$M_HOST:
 scp setup-web.remote $M_ROOT@$M_HOST:
 ssh $M_ROOT@$M_HOST ./setup-web.remote

install/setup-web.remote

@@ -1,3 +1,18 @@
 set -x
 
 sudo apt-get install nginx --yes
+cp sudoers.d-web /etc/sudoers.d/web
+chmod 0440 /etc/sudoers.d/web
+
+cd /etc/nginx
+ln -s /home/web/resources/nginx-ssl.crt
+ln -s /home/web/resources/nginx-ssl.key
+
+cd conf.d
+ln -s /home/web/resources/nginx.conf
+
+cd /var/local
+mkdir www
+chown web www
+
+

install/sudoers.d-web

@@ -0,0 +1 @@
+web ALL = (root) NOPASSWD :/etc/init.d/nginx start,/etc/init.d/nginx stop

keys/james/clean

@@ -0,0 +1,2 @@
+rm mail_servers.*
+rm final/mail_servers.*

keys/james/final/post-process

@@ -1,4 +1,4 @@
-cp ../mail_mailiverse_com.jks .
+cp ../mail_servers.jks .
-#keytool -import -trustcacerts -alias root -file ../mail_mailiverse_com/AddTrustExternalCARoot.crt -keystore mail_mailiverse_com.jks
+#keytool -import -trustcacerts -alias root -file ../mail_servers/AddTrustExternalCARoot.crt -keystore mail_servers.jks
-keytool -import -trustcacerts -alias POSITIVESSL -file ../mail_mailiverse_com/PositiveSSLCA2.crt -keystore mail_mailiverse_com.jks
+keytool -import -trustcacerts -alias POSITIVESSL -file ../mail_servers/PositiveSSLCA2.crt -keystore mail_servers.jks
-keytool -import -trustcacerts -alias james -file ../mail_mailiverse_com/mail_mailiverse_com.crt -keystore mail_mailiverse_com.jks
+keytool -import -trustcacerts -alias james -file ../mail_servers/mail_servers.crt -keystore mail_servers.jks

keys/james/make

@@ -1,2 +1,7 @@
-keytool -genkeypair -alias james -keyalg RSA -keysize 2048 -keystore mail_mailiverse_com.jks
+echo -----------------
-keytool -certreq -alias james -keyalg RSA -file mail_mailiverse_com.csr -keystore mail_mailiverse_com.jks
+echo the default installation uses 'password' for password
+echo sorry about that, feel free to modify sources
+echo -----------------
+
+keytool -genkeypair -alias james -keyalg RSA -keysize 2048 -keystore mail_servers.jks
+keytool -certreq -alias james -keyalg RSA -file mail_servers.csr -keystore mail_servers.jks

keys/james/prepare-response-crt-for-final

@@ -0,0 +1,6 @@
+if [ -z "$1" ]; then echo "Must supply directory name of result"; exit 0; fi
+
+rm -rf mail_servers
+mkdir mail_servers
+cp $1/* mail_servers/
+mv mail_servers/mail_* mail_servers/mail_servers.crt

keys/nginx/convert

@@ -1,12 +1,12 @@
 set -x
 
-JKSSTORE=mail_mailiverse_com.jks
+JKSSTORE=mail_servers.jks
 cp $JKSSTORE store.jks
 
 keytool -importkeystore -srckeystore store.jks -srcstoretype JKS -deststoretype PKCS12 -destkeystore store.p12
 openssl pkcs12 -in store.p12 -nocerts -out store.key
 openssl rsa -in store.key -out final.key
 
-cat mail_mailiverse_com.crt > final.crt
+cat mail_servers.crt > final.crt
 cat PositiveSSLCA2.crt >> final.crt
 cat AddTrustExternalCARoot.crt >> final.crt