Install Apache Traffic Server From Source on Ubuntu 14.04.3/16.04.1

Sources:


Building and configuring the Apache Traffic Server is super easy. There is only a few configuration changes to consider and some useful documentation areas to read over before beginning the procedure. This tutorial will build the server form source, but downloaded from the Apache Traffic Server website. Alternatively you can download from the repository or some Linux package managers such as apt-get and yum have older versions of the Traffic Server themselves. This tutorial will be using Apache Traffic Server 6.2.0.

Download

download the program with wget by executing the following commands

wget http://apache.parentingamerica.com/trafficserver/trafficserver-6.2.0.tar.bz2

Build

Execute the following commands to configure the project. The command ./configure will check you have all of the required dependencies for the installation. You may have to run it several times to ensure everything is included. Much of the online documentation provides general names for what is being looked for, but not everything is included.

tar xvf trafficserver-6.2.0.tar.bz2  
cd trafficserver-6.2.0.tar.bz2  
./configre

One of the more difficult dependencies is the openssl dependency. This required installing openssl, libssl1.0.0, libssl-dev, libsslcommon2 and libsslcommon2-dev on unbuntu to pass the configuration check

Install

Execute the next command to install

sudo make install  

Then execute the following and restart

sudo ldconfig  
sudo shutdown -r now  

You system will restart. ldconfig is a command that resets a number of control systems on linux. I ran into issues with the application until i ran this

Now your server is ready to go. Upon installation, the trafficserver command will be available anywhere in console. Simply start/stop/restart/status with a call like this:

sudo trafficserver <start/stop/restart/status>  

Keep an eye on the right hand side after each command as that will give u a status as to whether the command executed properly or not.

Debug / Problems

Errors can occur if the traffic server doesn't have permissions to the log files, which can cause more issues if you have other issues as well. By default the traffic server logs to the syslog daemon and information should be located in /var/log/trafficserver. Note if the trafficserver folder does not exist and does not have read/write permissions, logging will not work. Additionally, traffic server may write its logs to its install location under /usr/local/var/log/trafficserver. You can explicitly set the path that logs will be written to in the records.config file. Add or edit under the logging section of the file the proxy.config.log.logfile_dir attribute with the path to the desired logging location

If there are still issues and writing logs is not working, try executing the command traffic_ctl. This is another helper command for configuring the traffic server while it is still running. If there are errors, this command typicaly will dump them to console

Compile Thread Safe PHP 5.6.20 from Source

Sources


Compiling PHP 5.6.20 with thread safety is surprisingly straight forward. The first link mentioned in the sources actually is a shell script that will execute the process for you on Ubuntu Linux. be warned, there is a bug in that script though. Basically on line 39 of the script, the script clones the master branch of the pthreads project. This branch though is now setup for compatibility with PHP7. In order to compile pthreads with PHP5, you need to checkout the PHP5 branch on the repo before compiling.

We will be going through all of these steps in the tutorial below. This tutorial will walk through basically the above mentioned script as well as give equivalent commands so as to execute the script in Fedora 22 if that is your system instead. Commenting will be added beside any code if there is any difference as to whether to execute it in Ubuntu or Fedora 22. Additionally I am presenting this tutorial as if you were going to run this script by hand in the terminal. So you may find if you are reading along that a few steps appear out of order, this is merely for preference and maybe more intuitive ?


Update The System And Essential Packages

You should always make sure you have the latest packages. run updates for your system with the following commands

sudo apt-get update #Ubuntu  
sudo dnf update # Fedora 22  

Then install some essential tools you will need for downloading and manipulating the build

sudo apt-get install -y build-essential git-core vim curl #Ubuntu  
sudo dnf install -y git-core vim curl #Fedora 22  

Setup Terminal And Directories

For this tutorial I am assuming your executing everything in terminal, and not just running the script in the source links. So we need to set a couple of variables now

sudo mkdir /etc/php5ts

PHP_DIRECTORY="/etc/php5ts"  
PHP_TIMEZONE="UTC"  

NOTE: At this point do not close your terminal before finishing this tutorial. The variables that have been set above are only kept as long as the current terminal session remains open. Closing the session will remove the PHP_DIRECTORY and PHP_TIMEZONE variables. You can test if the variables are set by entering the following commands

echo $PHP_DIRECTORY  
echo $PHP_TIMEZONE  

Build Thread Safe PHP 5.6.20

Download and extract PHP 5.6.20 from here: http://php.net/downloads.php#v5.6.20 and navigate to the extracted root.

Install additional packages with the following commands:

sudo apt-get install -y make autoconf re2c bison #Ubuntu  
sudo dnf install -y make autoconf re2c bison #Fedora 22


sudo apt-get install -y libicu-dev libmcrypt-dev libssl-dev libcurl4-openssl-dev libbz2-dev libxml2-dev libpng-dev libjpeg-dev libedit-dev #Ubuntu

sudo dnf install -y libicu-devel libmcrypt-devel openssl* libcurl libcurl-devel bzip2-devel libxml-devel libpng-devel libjpeg-devel libedit-devel #Fedora 22  

Now lets start building PHP! At the PHP folder root enter the following commands

./buildconf --force

./configure --prefix=$PHP_DIRECTORY --with-config-file-path=$PHP_DIRECTORY --with-config-file-scan-dir=$PHP_DIRECTORY/conf.d --disable-all --enable-maintainer-zts --with-curl --with-openssl --with-gd --enable-gd-native-ttf --enable-intl --enable-mbstring --with-mcrypt --with-mysqli=mysqlnd --with-zlib --with-bz2 --enable-exif --with-pdo-mysql=mysqlnd --with-libedit --enable-zip --enable-pdo --enable-pcntl --enable-sockets --enable-mbregex --with-tsrm-pthreads

These commands will do additional checks for any missing packages on your system. If either of these command return errors of missing packages, Install them and then rerun the command.

If both run successfully without errors. Execute this next

make  

This will compile 5.6.20 and could take awhile. After this has completed execute

make test  

This will test the compiled php that all functionality expected works. You should only see PASSES and SKIPS while it processes the tests. The test procedure will give you a summary at the end and may prompt you to send results to the php development team. You can say yes or no here. This testing process executes around 10k tests, so aswell, this step will take awhile.

Once complete, run the following command

sudo make install  
sudo cp php.ini-production /etc/php5ts/php.ini  

Great, you now have compiled php5 with threading capabilities. Lets now add the threading!

Add Threading

Download and clone the pthread project from Github. Also we need to checkout the PHP5 branch, since the master branch is being developed for PHP7

git clone https://github.com/krakjoe/pthreads.git  
git checkout PHP5  

Then install php5 dev tools

sudo apt-get install -y php5-dev #Ubuntu  
sudo dnf install -y php5-devel #Fedora 22  

Now lets build the pthread project

phpize  
./configure --with-php-config=$PHP_DIRECTORY/bin/php-config

Again, configure will check for any additional dependencies needed that are not on your system. Install them and then rerun the configure step if you are given any errors when executing that command. Then as done when building PHP 5.6.20; compile, test and install pthread with the following commands

make  
make test  
make install  

This process should not take as long. The test suite only has about 60 tests in comparison. Again you may be prompted to send your results to the php dev team. The choice is up to you.

Complete the installation with the following commands

mkdir $PHP_DIRECTORY/conf.d  
echo "extension=pthreads.so" > /etc/php5ts/conf.d/pthreads.ini  

Congratulations, you have now completed the instalation of the pthreads extension. Lets configure our PHP 5.6.20 we build to use it!

Configure PHP 5.6.20 with pthread

Execute the following commands to update a number of settings in our compiled php to make it aware of the pthread extension

sudo su

sed 's#;date.timezone\([[:space:]]*\)=\([[:space:]]*\)*#date.timezone\1=\2\"'"$PHP_TIMEZONE"'\"#g' $PHP_DIRECTORY/php.ini > $PHP_DIRECTORY/php.ini.tmp

mv $PHP_DIRECTORY/php.ini.tmp $PHP_DIRECTORY/php.ini

sed 's#display_errors = Off#display_errors = On#g' $PHP_DIRECTORY/php.ini > $PHP_DIRECTORY/php.ini.tmp

mv $PHP_DIRECTORY/php.ini.tmp $PHP_DIRECTORY/php.ini

sed 's#display_startup_errors = Off#display_startup_errors = On#g' $PHP_DIRECTORY/php.ini > $PHP_DIRECTORY/php.ini.tmp

mv $PHP_DIRECTORY/php.ini.tmp $PHP_DIRECTORY/php.ini

sed 's#error_reporting = E_ALL & ~E_DEPRECATED & ~E_STRICT#error_reporting = E_ALL#g' $PHP_DIRECTORY/php.ini > $PHP_DIRECTORY/php.ini.tmp

mv $PHP_DIRECTORY/php.ini.tmp $PHP_DIRECTORY/php.ini  

Execution of the mv steps may cause prompts to confirm whether to overwrite the php.ini file. Say yes to this.

Installation of PHP 5.6.20 has now been completed on your system. To use this thread safe version you will have to call it with /etc/php5ts/bin/php. Calling just php in your terminal will execute the dev php we downloaded as part of this compiling process, which is not thread safe. You can make your life easier by creating a symlink to the thread safe php.

Install MySQL on Ubuntu 14.04.3 When Changing Root Password Errors

I found this for some reason on my Digital Ocean droplet when trying to install mysql. The server would be placed in this broken state all due to the root password of the server failing. Fortunately some trial and error found a solution. I followed along with this tutorial for the most part.

https://www.digitalocean.com/community/tutorials/how-to-install-linux-apache-mysql-php-lamp-stack-on-ubuntu-14-04

I also noticed when the installation did complete it said MySQL was running on a 14.04.1 server, which is not true...I do not know why this appears but I suspect maybe there is some kind of unknown permissions bug with MySQL at the moment and 14.04.3

The Fix

Basically its the usual sudo apt-get install mysql-server as given in the link above except when it does prompt for the root password leave it blank...DO NOTHING. let the software install itself.

Then we hit the setup scripts. And these are your saviours

After instalation completes. Run this guy:

sudo mysql_install_db  

Then run this guy:

sudo mysql_secure_installation  

Within this second script you will be prompted with the option to reset the root password. At THIS point change the root password to what you would like. And you will have no errors

Side Notes

I don't know if it actually have any effect whatsoever but I also have mongodb running on the same server. So I also had it turned off while this whole installation process occurred in case it interfered. If you have issues still with the above and have mongo installed, try that aswell

sudo service mongod stop #turn off mongo  
sudo service mongod start #turn on mongo  

Setup A Basic Gateway Server on Linux

Sources


**Full Disclosure**

This was tested on Ubuntu 14.04.1, items referenced below will be relative to locations in Ubuntu 14.04.1. The setup though across Linux distros should be the same other then paths. I will update this posting with location paths as they become known to me

-UPDATE: Additional notes added for Fedora 22
-UPDATE: This his now been tested on Ubuntu Server 14.04.1, 14.04.3 and Fedora 22. Documentation is complete to allow full setups on either of the 3 systems


This setup is a basic setup of a Gateway server. Note that this is not a tutorial on how to actually secure or properly secure the gateway. The intention of this tutorial is simply to get it working. It will allow anything to go through and work. The idea of this tutorial is to cator to first-time setups and people who would like to just tinker around.

Terminology

For this tutorial we have 2 devices - the Gateway Host and the Internal Host. The Internal Host being the machine behind the Gateway Host and relies on it in order to filter/forward/network all of the Internal Hosts traffic to the outside world

Setup the Gateway Host

Assumingly your Gateway Host has 2 network cards or connections of some kind. This is needed so that traffic from your Internal Host will come in one card of the Gateway Host and then sent out the other of the Gateway Host.

1. Configure Network Cards

Enter the following command:

ifconfig -a  

This will display all enabled and disabled cards. Find the card that will service your Internal Host. This will likely have a name like eth0 or eth1 on Ubuntu.

Then execute the following command. eth0 with the name of the card that services the Internal Host. Note also that as a newly enabled card we have statically assigned it an IP address 192.168.10.1 so that it can be routed to by the Internal Host.

ifconfig eth0 192.168.10.1 up  

2. Enable Kernel Forwarding

Kernel forwarding needs to be enabled in order for data to travel between your different network cards. This can be done as so on Ubuntu:

sudo cp /etc/sysctl.conf /etc/sysctl.conf.bak && sudo nano /etc/sysctl.conf  

This will create a backup and open the sysctl.conf file in nano. Within this conf file, search for and uncomment the following line:

net.ipv4.ip_forward=1  

This will enable Kernel Forwarding. On Ubuntu you will need to restart your Gateway Host now for the change to take effect.

On Fedora, you can enable kernel forwarding in a similar way, or use the shortcut command:

echo "1" >/proc/sys/net/ipv4/ip_forward  

On fedora this may or may not need a restart

3. Setup NAT Routing

NAT Routing takes a few special commands using iptables. These commands will let everything back and forth through your Gateway Host so it is important to note that this will not secure your Internal Host whatsoever

Enter the following iptables commands in order:

# flush all rules in fulter and nat tables
iptables --flush  
iptables --table nat --flush  
iptables --delete-chain  
# delete all chains that are not in default filter and nat table
iptables --table nat --delete-chain

iptables --table nat --append POSTROUTING --out-interface eth0 -j MASQUERADE  
iptables --append FORWARD --in-interface eth1 -j ACCEPT  

Note again in the above commands to replace eth0 and eth1 with the appropriate card names. eth1 in the above example is the card serving the Internal Host and eth0 is connected to the outside network.

It is also important to note that the above rules only allows the Internal Host to have access to the outside world. If your Internal Host is offering services through the Gateway Host, additional rules will be needed.
To allow external machines to connect to the Internal Host add the following rules:

iptables --table nat -A PREROUTING -i eth0 -j DNAT --to-destination 192.168.10.2  
iptables --append FORWARD --in-interface eth0 -j ACCEPT #let everything through  

This will allow connections to come in to the Internal Host. For this example, our Internal Host's IP is 192.168.10.2 which you can see in the first rule above is the --to-destination meaning all traffic directed at the Gateway Host will be redirected to the Internal Host.
NOTE: With the second command listed above will allow everything to be redirected to the internal host. For a more secure procedure, replace this rule with more finite FORWARD table rules using iptables.

Setup the Internal Host

Now all we have to do is tell our Internal Host to resolve its IP's with our configured Gateway Host.

1. Configure Network Cards

Again execute the following command to determine the name of your network card that is connected to the Gateway Host

ifconfig -a  

Then execute the following command replacing eth0 with the name of the card connected to the Gateway Host

ifconfig eth0 192.168.10.2 up  

Note that the ip address we assigned here belongs to the subnet of our Gateway Host this is an important factor as our Gateway Host only knows how to route data from our 192.168.10.0 subnet to the outside world.

If your Internal Host only has a single network card, it may be already enabled and have an IP assigned to it. To change the IP to be part of the subnet run the following command

ifconfig eth0 down  

this will disable the network card. Use the command from earlier now to re-enable it and assign an IP.

If your Internal Host has multiple cards and some are connected to the internet, you will probably want to disable them so as to be able to test if your Gateway Host has been configured correctly. You can disable those cards with the ifconfig eth0 down command mentioned earlier, replacing the eth0 with the name of the card

2. Configure Routing

Now we want to route all traffic from our Internal Host to the Gateway Host. To do this we simply change the Internal Hosts default resolving IP to the Gateway Hosts IP. We do this with the following command:

route add default gw 192.168.10.1  

3. Check Nameserver Resolution Matches

Fedora 22

On Fedora you will need to make sure the Internal Host and the Gateway Host both have the same content written in the /etc/resolv.conf file. Most importantly you want to copy the contents of the Gateway Hosts resolve.conf into the Internal Hosts resolve.conf file. Otherwise you will have troubles making DNS calls from your Internal Host. A copy and paste and a possible reboot is all that is needed.

Ubuntu Server 14.04.1 - 14.04.3

On Ubuntu Server you will need to do the same as Fedora except Ubuntu automates the process more.
To view the nameservers on the Gateway Host enter the following command:

cat /etc/resolv.conf  

This will display the nameserver IP that needs to be copied to the Internal Host

To update the Internal Host's nameservers, run the following commands

cd /etc/resolvconf/resolv.conf.d  
sudo cp -p head head.orig  #create a backup copy  
sudo nano head  

In the now opened file type the following:

nameserver <ip-of-nameserver>  

Hit Ctrl+X to save and then type:

sudo resolvconf -u  

This will cause Ubuntu to reload its nameservers from the file that was edited.

Install MongoDB 3.0 on Fedora 22

Sources


There seems to be very little complete documentation on installing the latest versions of MongoDB on Fedora. My situation was specifically Fedora 22. I was able to get everything working fortunately after discovering a feed from someone else posting the question, and then some digging around the MongoDB documentation for start-up instructions. Oddly Mongo doesn't have anything specifically for Fedora, so I had to poke around and guess a bit with CentOS and RedHat instructions. Fortunately they are very much the same for the most part with Fedora

Add the Repository

This took a bit of mixing from the feed and the mongodb documentation. Taking from the feed doesn't end up completely adding the repository properly, so I found it worked better doing it the following way. cd to the repository directory in /etc/yum.repos.d/ and create a file called mongodb-org-3.0.repo. Then add the following in the file:

[mongodb-org-3.0]
name=MongoDB Repository  
baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/3.0/x86_64/  
gpgcheck=0  
enabled=1  

Note that there is a variable $releasever in the above code snippet. From the feed for Fedora 22 the $releasever should be replaced with 7. Since this may eventually become obsolete though, I have left the variable in. Replace with the appropriate value is 7 is no longer appropriate

As a side, the main difference between the method of adding the repository described in the MongoDB documentation to the feed is the gpgcheck. From the snippet above and the MongoDB docs you can see it is set to false, as the code is not signed. The feed though adds the repository with the dnf helper command:

dnf config-manager --add-repo https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/3.0/x86_64/  

Which the config-manager default enabled the gpgcheck. Now you can side step this when you install with the --nogpgcheck parameter in the install call but I thought it was a bit messy for my liking.

Install Mongo

This is the easy part. Run and execute:

dnf install -y mongodb-org  

Start Mongo

This step tripped me up initially as Fedora, from my experience so far, names its services the same as the program name or the installed package. To start mongoDB though you need to execute the following command:

systemctl start mongod  

This command may take awhile as mongo does a first time load. Mongo will then have loaded. You can restart, stop and get the status of mongo with the following commands:

Stop Mongo

systemctl stop mongod  

Restart Mongo

systemctl restart mongod  

Get Mongo Status

systemctl status mongod  

Use Mongo

You can now interact with mongo locally through the mongo shell client like so:

mongo  

This will automatically log you in and load the test database. To configure Mongo further for external access and such, checkout the above links. The MongoDB documentation has good information on where to go. There is a specific issue with dealing with SELinux that the documentation says will interfere with that kind of setup