Archive for ‘Unix’ Category
Posted on 17:17, March 10th, 2014 by Many Ayromlou
A little while ago our web developer asked me to look into proxmox containers and how we could take advantage of it to setup a development environment for him. The idea was to use the power of linux containers and enable him to develop fully functional/accessible sites in a private container. Here are the steps we will cover in this article:
To do all this you need to download proxmox ISO file and burn it to a CD. Go through the installation of proxmox and set up the “host” with the single pubic IP address. This is simple enough so I’m not gonna cover it here. Once you have this setup you should be able to point your browser at the IP address (https://aaa.bbb.ccc.ddd:8006). NOTE: I will use aaa.bbb.ccc.ddd as the representation of the publicly available IP throughout.
Next we need to secure access to the host to only allow connections from a specific IP address space. In my case that’s the University network — 220.127.116.11/16 — this is optional. We need to make sure ufw is installed. We also need to make sure ufw is allowing incoming connections by default and then block everything except access from the University network:
Note that I’m assuming your ssh connection to the host is via the University network (18.104.22.168/16). Make adjustments to this if it’s not, otherwise you might lock yourself out. These basic rules will plug all the holes accessible publicly and only allow connections from our University network (22.214.171.124/16).
Setting up users in proxmox is a bit weird. You have to add a regular Unix user to the proxmox host environment and then add the user to proxmox later and give it permissions and roles. Here I will use a user “myadmin” to create something for our web developer to use.
This will create a account “myadmin”, join it to primary group “myadmin”, assign it /bin/bash as shell and make it part of the group “sudo” — which will allow the user to use the sudo command in the future. Next on the proxmox web interface we need to create a Admin group called “Admin”. In the proxmox interface we click on the Datacentre in the left pane and go to Groups and click the Create button. Call the group “Admin”. Now go to Permissions tab in the right pane. We need to create a Administrator Group Permission to assign to our “Admin” group. Click Add Group Permission (right below the tabs in this pane) and fill it in like below:
In this window the path: / means the entire Datacentre (including the host and the containers/VM’s). You might want to adjust this. The Role “Administrator” is a predefined role that is pretty much the same as root. Now that our group “Admin” has the “Administrator” role for the entire Datacentre, we want to make the user “myadmin” — which is a unix account right now — be part of that, effectively creating another “root” account for our web developer. So back to the Users tab we click Add and create our new user (really just add the Unix user to proxmox):
Okay, so now test and make sure you can access the host via ssh using myadmin as user, also make sure you can sudo to root on the host and check the web interface and ensure the myadmin account can login and see all the goodies in the data centre. Otherwise stop and fix.
At this point login/ssh to the host as root or myadmin (plus “sudo -i” to become root). We need to modify the networking config in /etc/network/interfaces to setup all the masquerading jazz. Make a back up of your interfaces file first and note the public IP address that is in there (I’m gonna use aaa.bbb.ccc.ddd as my public address here). Once you have a backup replace everything in the file with the following:
So in the above I’m creating a separate private network (10.10.10.0/24) behind the publicly available IP address aaa.bbb.ccc.ddd and am doing some iptables commands to setup masquerading. This is sorta like setting up a home router to share a publicly available IP address you have at home. Once this is in place reboot the host and make sure you can log back into https://aaa.bbb.ccc.ddd:8006/ and get the proxmox interface. If you’re good to go, as next step spin off two Ubuntu containers (I won’t go into details on this…..lots of docs out there for this). Your OperVZ Container confirmation screen should look something like this:
The only really important thing here is that you setup the networking under Network tab as Bridged mode and select vmbr0 as your bridge. Once that’s done ssh back to your host (aaa.bbb.ccc.ddd). Assuming you have two containers 100 and 101, enter one of them by using the vzctl command:
Once inside the container you need to setup the networking. Again the file here is /etc/network/interfaces (assuming you’re container is Ubuntu/Debian flavoured). Backup this file first and replace the content with the following:
Note here that I’m using google’s name server. You can use that or substitute your own “real” name servers. Once you reboot the container and enter it again via the host, you should be able to ping just about any real host (www.google.com, www.yahoo.com or whatever). This gives us a basic NAT running on the host and you just need to increment the IP address (10.10.10.2 in the above case) in the setup of the second container. At this point you should be able to enter either containers and ping something outside.
So the rest of this article describes how to setup a secure reverse proxy using apache on the proxmox host (aaa.bbb.ccc.ddd). This way you can just point arbitrary DNS names at aaa.bbb.ccc.ddd and choose (via apache config) which one of your containers will answer the call. You can even get fancy and have multiple hostnames proxied to the same container and do standard “Name based” virtual hosting inside the container. I will just show the one-to-one proxied connection here. Start by installing apache on the host (apt-get install apache). First we need to activate the proxy module. If you don’t have time to finish this entire procedure DO NOT CONTINUE. Literally in the time it takes to install and configure the proxy, script kiddies will hit your site and use you as a proxy to attack other sites. DO THE PROXY INSTALL AND CONFIG/SECURING PROCEDURE IN ONE SHOT.
Assuming apache is installed go to http://aaa.bbb.ccc.ddd and ensure you’re getting the apache “hello” screen. Now you can enable the three modules needed by issuing the following:
Once that’s done you need to make some changes to your proxmox hosts default apache config which is in /etc/apache2/sites-available/default. For the sake of completeness I’ve included my entire file here. Compare it to yours and modify accordingly:
Pay particular attention to parts that have the comment (# IMPORTANT: YOU NEED THIS)……Guess what…..YOU NEED THIS. The first one loads libxml2 which is needed. The second block of code makes sure you are in reverse proxy mode (not in forward proxy) and makes sure the main apache instance can’t be used for proxing. The third and fourth block enable reverse proxy for a particular virtual host name. Now we need to reload apache on our proxmox host and do some testing. Reload apache with (service apache2 reload) and for sanity sake change the index.html file in both containers (under /var/www/index.html) to reflect hosta and hostb. I’ve basically just added the words hosta and hostb to the html file. Register hosta.domain.ca and hostb.domain.ca as “A” fields in your DNS and point them at the IP address of the proxmox host (aaa.bbb.ccc.ddd).
If everything is working properly you should be able to use your browser and point at http://hosta.domain.ca and get the index.html page specific to that container and the same for hostb. At this point you should be more or less good to go. If you need more containers addressable from internet, just keep adding this block of code to the proxmox hosts /etc/apache2/sites-available/default and change the hostname and increment the private IP addresses:
Optionally you can now go back and add a couple more ufw rules to only allow access from a particular IP address space (in my case the university network 126.96.36.199/16)
Again with this setup — since we’re preserving the request header and are passing it through the proxy back and forth — you can have hostd, hoste, hostf, all point to the same private IP address in the proxy and do a named virtual serving on the apache instance in the particular container, just like a standard named virtual host based setup. Hope this helps…..
Posted on 15:35, November 24th, 2013 by Many Ayromlou
Proving the Network is Not the Problem With iperf – Packet Life: “When an application fails to perform as expected, the network is often the first thing blamed. I suppose this is because end users typically view the network as the sole limiting factor with regard to throughput, unaware of the intricacies of application, database, and storage performance. For some reason, the burden of proof always seems to fall onto networkers to demonstrate that the network is not at fault before troubleshooting can begin elsewhere. This article demonstrates how to simulate user traffic between two given points on a network and measure the achievable throughput.”
Posted on 14:38, August 15th, 2012 by Many Ayromlou
Here are some quick tip(s) for copying a ton of files between unixy machines really fast. You’re probably thinking “why not use rsync?”…..well rsync can be miserably slow if your source or destination cpu is underpowered. You can always do a rsync after these commands to make 100% certain that everything checks out, but try using one of these methods for the initial copy:
Once your favourite process (above) is done you can do a quick rsync to tie up any loose ends.
Rsync will now fly through the filesystem as 99.9% of the time, 99.9% of the files on the destination are good. And as always make sure you understand the commands before you use them…..and keep backups just in case :-).
Posted on 23:03, November 26th, 2009 by Many Ayromlou
I recently upgraded netbook using the distribution upgrade and didn’t like the results, so I reinstalled Ubuntu Notebook Remix 9.10 Karmic Koala. Well, I’m sorry but I don’t think this Koala was ready for release. First there was the issue of where the heck are all the beloved Ubuntu tools. Gone is the Add/Remove software progy (you have to install manually), now we have Ubuntu Software Centre. Gone is being able to check off multiple packages for batch install, USC installs apps one at a time (which takes two mouse clicks per app).
To top it off — atleast in UNR 9.10 — there is no install button once you click on the arrow beside the packages. No, it’s not a problem with root/admin, I tried running it as root and same thing, NO INSTALL BUTTON on the install screen. Anyways it turns out once you get past the gargantuan Windows XP like update (125 updates) using the following two commands, the Ubuntu Software Centre magically comes back to life and gives you the “oh so important” install button. Come on Ubuntu…..I thought you were friendly. This Koala Bites HARD!!! :-). So the magic commands are….yeah you guessed it:
BTW. If at some point the upgrade asks to replace /etc/defaults/grub say “yes” and go with the newer version. It does not harm the system.
Posted on 13:58, October 31st, 2009 by Many Ayromlou
Downloaded and installed 9.10 yesterday and what do you know, someone decided to take away Ctrl-Alt-Backspace — or what I call “Three Finger Salute for Linux”. Whhhaaattt!!!!
How the heck are you supposed to kill and restart X without that…..A coworker suggested Alt-PrintScreen-K, but that just restarts GDM, not really useful when X decides to go south. Damit!!
The reason given on Ubuntu wiki is that “This is due to the fact that DontZap is no longer an option in the X server and has become an option in XKB instead.”
Well, fear not, whoever disabled it also created a easy way to reenable it again. Here is what you do:
Posted on 12:17, October 30th, 2009 by Many Ayromlou
It’s been a while since I’ve had the pleasure (read: pain) of working with Sloowaris, but now that we have two 48TB Sun X4540 Thumpers in house, I have to…..Uggghhhh :-). Here are some notes:
Not sure if this has already been mentioned somewhere…..It’s pretty old, but I happen to come across it today. It’s a great rendition of everyone’s favorite space opera done by Simon Jansen in ASCII. Telenetification (is that even a word?) by Snore, with improvements by Mike Edwards. Anyways, use the following command, sit back and enjoy…..Star Wars in all its ASCII glory :-)
If you don’t know how to telnet, click here to see it in your browser.
Posted on 17:22, September 9th, 2009 by Many Ayromlou
Posted on 13:54, June 19th, 2009 by Many Ayromlou
So after yesterdays rant, I went back and figured out how to install the Cacti monitoring software (OSS, Free) onto a Ubuntu 9.04 “Jaunty Jackalope” Desktop installation. This guide uses packages only, no compiling, no Makefiles or anything like that…..You should be able to just follow this and get a fully functioning Cacti installation in about 30 minutes. Here are the steps:
Posted on 13:55, February 18th, 2009 by Many Ayromlou
As good as Ubuntu (and linux) are in general, once in a while you just get to a point where you need a reinstall. That’s when the realization kicks in that you’ve got far too many packages installed since the initial Ubuntu install. It’s okay, there is a way out. Make sure you have a USB key.
On Ubuntu Workstation (with graphical interface):
Run Synaptic package manager. Once inside Synaptic, go to File/Save Markings As menu option and choose a filename and location (USB stick). MAKE SURE YOU ALSO CHECK THE BOX “SAVE FULL STATE, NOT ONLY CHANGES”. This will save a text file that contains every single package installed on your system (through apt system and all it’s variants….manual compile/installs are something else). Now you can go ahead and reinstall the machine and configure your repositories. Once the machine is up and running again, load up Synaptic and go to File/Read Markings and point it at the file you saved on the USB stick and press Apply.
This will start a download process that will set the machine up (as far as installed packages are concerned) just like it used to be. Configurations need to be done manually, but atleast you get all your packages back.
This is also super handy if you’re duping identical systems. Remember that you can not do this to upgrade from one version to another. This is strictly for “Restoring” installed software packages from the same version of Ubuntu.
On Ubuntu Server (command line):
First we need to create a list of all the installed APT packages and configurations and save them:
Now reinstall the OS, copy your backed up debconfsel.txt and dpkglist.txt file to your fresh system’s /tmp directory and execute the following: