|
At the crossroad of New media, Engineering, Research and Development
|
|
Archive for ‘Linux’ Category
May
20
2009
Writing Moblin (and Ubuntu) USB images using dd in OSXI came across this problem this morning, while writing the newly downloaded moblin USB image file. The concept is straight forward, plugin a 1GB+ USB stick into a functioning Linux or Windows box, make sure the stick is not mounted and use dd to write the disk image to the stick. Under OSX however the instructions for unmounting are slightly different, so here are the quick steps:
That should do it….. Heard of Moblin yet….Intel’s foray into designing a Linux distro. Moblin is an open source project focused on building a Linux-based platform optimized for the next generation of mobile devices including Netbooks, Mobile Internet Devices, and In-vehicle infotainment systems. I came across the promo video below and I have to say I’m impressed. I’m downloading the beta image file right now to give it a try on my brand new Aspire One D150. More to come soon….. May
13
2009
ROCKS Cluster: "Unable to run job: denied: host is no submit host"I ran into this a couple of weeks ago and it’s been driving me bonkers. I finally figured out what’s wrong. I was just trying to get my feet wet using the Sun Grid Engine and figured I follow their instruction page and try out the example shell script and submit it using “qsub” command. I was doing this on the frontend machine that’s been configured properly as a ROCKS cluster frontend. This was not working and the error I kept getting was “Unable to run job: denied: host “name_of_computer” is no submit host. Exiting.” After googling around for a couple of days I found the answer (atleast the answer in my case). Issuing the following command solved my problem:
Apparently the SGE roll does not setup the frontend node as a “submit host” during install. After this (the above command) everything seems to work properly. Now I can do “qstat -f” and “qsub”. May
05
2009
Openfiler install on large disk + failover setup + usb installOkay so I’ve been playing around with openfiler for the past couple of months. We’re trying to setup a Student homedirectory NAS device and have a mirror machine that would take over if our primary dies. Our machines are hand built 13-TB NAS servers using 16 x 1TB Seagate disks and a 16 channel sata2 raid controller from 3Ware. There are several problems that one needs to overcome in this type of setup so I will try to cover it, bit by bit as I finish confirming it at work. As I said we’re using a Super Micro case and motherboard (Dual Quadcore Xeon) and we’ve stuffed a 16 channel 3ware 9650 controller in there. The first issue we had was with hardware and the fact that we had some screwy new firmware on the controller that was not working nicely with our 16 x 1TB seagate drives. We downgraded the firmware and got the machine to post. Then we created a (roughly) 14 TB container in raid-6 mode (16 drives, less 2). We further devided up the space into a 20GB boot partition (using the bios setting in 3ware bios) and a giant (rougly) 13TB partition that will hold our student data. The 20 GB partition will later on hold our swap space and non essential (frequently updated) folders under /var (lock, log, etc.) We have physically 2 separate machines that are exact copies of each other hardware-wise. The plan initiallly was to use DRBD and heartbeat service to create a High availability NAS cluster, but since we are tying to authenticate (for smb) with our Windows system, we could not get that configuration working (and frankly I still don’t trust DRBD, as good as it is). So we decided to create two USB sticks images. One for master and another for slave. The master will be a machine enrolled into our Active directory domain and the slave will be a passive (private) rsync server. The master USB image is configured with all the AD stuff and two interfaces. One interface serves as the NAS and another runs rsync against our slave/rsync server. When/If the master fails (ie: motherboard failure) beyond recognition, we simply plug the master USB stick into our slave machine and reboot it. Since the machines are exact copies of one another the (old slave) will now be master and once the (old) master is fixed, it will become the new slave/rsync server. Real simple. So here is Chapter one – How do you get Openfiler 2.3 to boot off a USB stick: Before you start you’ll need the following:
Insert your USB stick, and boot from the OpenFiler 2.3 installation CD. At the boot prompt, type expert (for text mode type expert text, I used graphical mode). Manually configure your partitions. I just had one 2Gb partition (ext2) on /. I used ext2 since it has no journal and won’t constantly write to the USB stick. No Swap partition at this point. After the install I noticed that something between 600 and 700 Mb was used for the system, so you might be able to use about 200-300Mb for swap if really needed (however, I doubt the use for a swap partition, as USB storage is really slow). The installer will breeze through to the end. Note that it is realllyyy slow. It took more than an hour on my config. Reboot at the end and get the OF2.3 CD booting again in rescue mode by typing “linux rescue” at the prompt. Once you’re at the prompt mount the USB stick manually (fdisk -l might help as it will print out info about all the disks). My USB stick was /dev/sdc, hence the commands below:
Now you’ve got the partition mounted and your shell chrooted to the root of the USB stick. We next copy the initrd on the USB stick into a temporary directory (on the stick) and uncompress it so we can modify it. You need to do this so that grub can initialize the bootloader ram disk off the USB stick (ie: makes OF installation bootable from USB).
At this point we need to edit the “init” file (text file containing kernel module listings that are required during boot). I used vi to do this, not sure if there is another editor available during rescue mode. Find the line containing “insmod /lib/sd_mod.ko” and insert the following snippet under it:
Save the file and follow along with the following commands to physically copy the appropriate kernel modules to the temp directory.
IMPORTANT – Now adjust grub config (/boot/grub/grub.conf) to reflect the change to initrd filename. You should also repeat this on kernel upgrades (but then again, never touch a working system ;)). More than likely it’s a no go, since the installer did not install grub properly. Now take out your Ubuntu (or other favourite rescue CD) out and boot from it. Don’t use the OF2.3 CD in rescue mode…..IT DOES NOT WORK. Once booted, mount the USB stick on the system and use the following commands to re-install grub:
Reboot and you should be good to go (you will get a couple of Errors during boot about modules already loaded stuff…..ignore). At some point you do want to move some of those auxiliary directories (/tmp/ /var/log /var/lock and others) and swap file off the stick and onto the 20GB portion of our raid-6 we prepped earlier on. Below you find the fdisk -l listing of that “logical disk” (/dev/sdb in our system):
Device Boot Start End Blocks Id System here is a breakdown of what goes where (/dev/sdb6 is obviously swap which was prepared with “mkswap” command):
You need to make the above changes to your USB stick’s /etc/fstab, but before rebooting you need to use “cp -a” command to copy all the folders from the appropriate location on the USB stick to the above partitions (by mounting the partitions temporarily/one-at-a-time), just to make sure no process would go crazy if it didn’t find the lock directory (or cache, run, etc.). Next we want to make four copies of this stick. You can use a Mac or Win (using rawrite) or better yet Linux. It’s important that the stick your copying is not booted. Use the Ubuntu/whatever CD you used ealier and boot it into rescue mode. Go to command line and use “dd” command to create three more copies of the stick you just preped. Two copies (one for safe keeping) will become your Master USB sticks to boot the machine in Master mode (as described earlier in this article). The other two copies (one for safe keeping) will become your Slave sticks. NOTES: These notes have nothing to do with the installation. I’m just putting them down here for safe keeping. Only use these if you’re in trouble. - If you want to create a “Home Share” and you don’t get the “Make Home Share” button on the interface, something has gone wrong with one of the xml config files. No worries, find and edit the file /opt/openfiler/etc/homespath.xml . Inside it will look something like this: - If you have upgraded to a Windows 2008 R2 (Win2k8 r2) AD domain and you’re getting authentication errors when accessing your openfiler shares (although everything was working fine under R1) like the ones below: Nov 16 08:42:02 openfiler winbindd[3316]: [2009/11/16 08:42:02, 0] rpc_client/cli_pipe.c:rpc_api_pipe(789) [2009/11/16 08:43:12, 1] winbindd/winbindd_util.c:trustdom_recv(269) - If you’re having problems accessing a samba share you just created on your brand new openfiler, you might want to check the following. Lets say you have a Volume Group called “bigvg” and a Volume inside that called “studentvol” where you have a share called “test”. If you’re having problems accessing the share by just using something like - Couple of useful commands for Samba troubleshooting….. - There is another issue with this master/slave setup and that is UID/GID synchronization for samba. This comes into play since we’re rsyncing our files from master to slave. This process also transfers their respective UID/GID to the slave machine. If the master fails, our procedure is to turn if off and reboot the slave using the masters USB stick. This works, but all those rsync’ed UID/GID’s will not match when the slave machine is booted using the masters USB stick (samba voodoo that translates windows UID/GID’s to linux UID/GID is kinda random)…..UNLESS YOU DO THE FOLLOWING (taken from Samba How-To): The idmap_rid facility is a new tool that, unlike native winbind, creates a predictable mapping of MS Windows SIDs to UNIX UIDs and GIDs. The key benefit of this method of implementing the Samba IDMAP facility is that it eliminates the need to store the IDMAP data in a central place. The downside is that it can be used only within a single ADS domain and is not compatible with trusted domain implementations. Please note that the update version of SAMBA that gets installed after you do “conary updateall” (see above) has a option for this under “Advance” tab of the Accounts section. Mar
07
2009
Syncing Linux (Ubuntu) and Iphone/Ipod Touches with Firmware 2.xIf you’re using Ubuntu and have recently upgraded your IpodTouch or Iphone to 2.x firmware, you might be interested in this detailed tutorial. It basically outlines how you can setup syncing under Ubuntu with your 2.x device. The guide assumes that you have jailbroken your ipod/iphone . There is also a nice section for older 1.x devices. Feb
24
2009
Linux Server-in-a-Plug is here…..only $100Marvell Semiconductors is now shipping their SheevaPlug linux machines. Little tiny Linux boxes the size of a plugin adapter. The SheevaPlug draws about 5 Watts of power, comes with Linux, and boasts completely open hardware and software designs. At $100 the platform is available in single quantities, and is priced within reach of students, hobbyists, and tinkerers. This looks like the perfect embeded platform for all sorts of stuff. Think home automation, security monitoring, ultra low powered file servers, ad-hoc mini clusters, not to mention robots and such …..there is no end to it. Its hardware design is completely open — everything from schematics to Gerber files will be available on a marvell’s website — and ARM ports of several popular Linux distributions are already running, and included. More importantly, Marvell has committed to do everything it can to ensure the best Linux support for SheevaPlug going forward. The $100 SheevaPlug development platform and Plug Computer designs are built around the Marvell 88F6000, or “Kirkwood” SoC, which was introduced last year. The Plug Computer is based on the high-end 88F6281 version of the Kirkwood, with a Sheeva CPU core clocked to 1.2GHz. The Sheeva core combines elements of Marvell’s earlier Feroceon and XScale architectures, both of which implemented ARM Ltd.’s ARMv5 architecture, similar to ARM Ltd.’s own “ARM9″ cores. The SheevaPlug Plug Computer is further equipped with 512MB of DRAM and 512MB of flash. The tiny embedded PC also includes gigabit Ethernet and USB 2.0 ports. One early product based on the design is listed as measuring 4.0 x 2.5 x 2.0 inches. Plugging directly into a standard wall socket, the Plug Computer draws less than five watts under normal operation, compared to 25-100 watts for a PC being used as a home server, claims Marvell. Early supporters of the SheevaPlug Plug Computer design include the following companies, each with links to their respective websites:
Fun,fun,fun….. Thanks to Sugar Labs you can now have your Ubuntu 8.10 or Fedora 10 linux distributions with sugar on it…..Shweet :-). Yep you heard right, go here and grab your OLPC XO inspired 1GB USB stick image and boot all those old PC’s into sugar. According to Walter Bender (the creator of Sugar OS) a new version dubbed Sucrose 0.84 is on its way soon. Complete article (including interview with Mr. Bender) is over at XConomy. Feb
18
2009
Duplicate your Ubuntu Installation….As good as Ubuntu (and linux) are in general, once in a while you just get to a point where you need a reinstall. That’s when the realization kicks in that you’ve got far too many packages installed since the initial Ubuntu install. It’s okay, there is a way out. Make sure you have a USB key. On Ubuntu Workstation (with graphical interface):
Run Synaptic package manager. Once inside Synaptic, go to File/Save Markings As menu option and choose a filename and location (USB stick). MAKE SURE YOU ALSO CHECK THE BOX “SAVE FULL STATE, NOT ONLY CHANGES”. This will save a text file that contains every single package installed on your system (through apt system and all it’s variants….manual compile/installs are something else). Now you can go ahead and reinstall the machine and configure your repositories. Once the machine is up and running again, load up Synaptic and go to File/Read Markings and point it at the file you saved on the USB stick and press Apply. This will start a download process that will set the machine up (as far as installed packages are concerned) just like it used to be. Configurations need to be done manually, but atleast you get all your packages back. This is also super handy if you’re duping identical systems. Remember that you can not do this to upgrade from one version to another. This is strictly for “Restoring” installed software packages from the same version of Ubuntu. On Ubuntu Server (command line):
First we need to create a list of all the installed APT packages and configurations and save them: Now reinstall the OS, copy your backed up debconfsel.txt and dpkglist.txt file to your fresh system’s /tmp directory and execute the following: We also covered aptoncd program which basically does the same thing (it’s a extra install). Last but not least to make a costum Ubuntu install CD/DVD you want to check out our entry on Reconstructor. Feb
13
2009
Disk for Iphone….Turn your IPhone (IpodTouch) into a portable HD.MacFUSE project has grown a lot since we last covered it here and here. MacFuse 2.0 is here and it’s looking really nice. MacFUSE is the mac implementation of FUSE (File-system in USEr space) filesystem originally developed for Linux. For those of you who’ve not heard of this gem before, MacFUSE allows you to extend Mac OS X’s native file handling capabilities via 3rd-party file systems. Pretty much anything that has some order to it can be turned (viewed as) into a filesystem (ie: sshfs, youtubefs). As a user, installing the MacFUSE software package will let you use any 3rd-party file system written atop MacFUSE. Disk for iPhone is a MacFUSE based filesystem that allows you to read and write files on your iPhone. It uses the MobileDevice API (like iTunes) to access the filesystem of the iPhone over USB. You need to install MacFuse base system on your machine first and then grab Disk for iPhone module. Imagine if you could hide your 16 Core, Quad GPU, Nitro burning gaming PC in the closet in the basement. What if you could build the next University lab where all the highpowered CAD workstations where humming away in the machine room and you could actually “teach” in a quite classroom/lab. Fill in the rest of the scenarios yourself. The next wave in thin client computing is here and it’s name is Teradici. They don’t really make a end user product (they just make the custom compression chipsets), but companies like Leadtek have end user products on the market now. Integration of Teradici‘s PCoIP remote enterprise desktop technology in Leadtek‘s WinFast VP200 enables delivery of a high-definition graphic and multimedia experience across standard IP networks, unmatched by any thin client on the market today. For $800 per link, $400 on each side of the link, Teradici provides a PC-over-IP host card, to be embedded into the host unit (most likely a workstation), and a desktop portal, a device slightly bigger than a hardcover book, equipped with a Teradici processor chip, 4 USB ports, and an HD audio output, and dual DVI outputs. The desktop portal and the host unit are linked via LAN, WAN, or a wireless network, allowing the user to communicate with the back-end PC. The WinFast VP 200 system includes a WinFast VP200H host PCI Express card and a WinFast VP200P desktop portal. The Host Card allows you to centralize your computing in a data center as an add-in solution to your existing Workstation, Rack Mount, or Blade Server to secure all data and computers; On the desktop, a stateless device called a Portal connects over a standard enterprise IP network to the Host Card, eliminating heat, noise, and clutter at the end user’s desk for a comfortable working environment. AND DID WE SAY IT’S COMPLETELY GPU AND OS INDEPENDANT. SO GO GET ONE :-).
So head on over and grab the ISO file. University of Massachusetts Dartmouth: ATMC Physics Professor Gaurav Khanna and Principal Investigator Chris Poulin have created a great step-by-step guide that shows you how $4000 and a bit of elbow grease can get you a nice supercomputer cluster. They use Fedora Core 8 distribution, due to the prevalence of Fedora and its Cell SDK (3.0) compatibility Nov
11
2008
Energize your Apple TV based Media Centre machine with BOXEE….I was lucky enough to get a alpha Boxee account and I have to say…..It’s slick. If you haven’t tried it, head over to boxee.tv and sign up for a invite (Mac OS and Linux only for now). If you already have an account you might be interested in this tutorial video that shows you how to install boxee on your Apple TV box. Before you leave please make sure that when you create the patchstick there are no external drives connected. Oct
29
2008
Seam Carving Gui….Resize images just like the new Photoshop CS4So you’ve probably been hearing a lot lately about the new feature in Photoshop CS4 that allows you to resize an image in a way that the image will not get distorted. Dubbed Seam Carving (we covered it here, here and here), the new function is all the rage.Well, if that’s the only reason you’re going to upgrade to CS4, you might want to hang on and save a bit of $$$. You see, there is a opensource, free, standalone implementation of seam carving called Seam Carving GUI (which is itself a frontend GUI for CAIR…..Content Aware Image Resizer). It’s available for OSX, Windows and Linux so download it, read the manual and have fun saving $$$. Here is a example of what this little gem can do….. Original
Stretched/squeezed and black fish removed
Oct
26
2008
Synkron, because syncing can be such a pain….I know, rsync is the answer to all my prayers, at least as far as syncing data is concerned. I agree, BUT sometimes you just want a simple GUI (yes pointy/clicky) application to do the job. Well that’s were synkron comes in. A simple applications that does one thing and it does it well, synchronizing your files/folders. It’s multi platform and works really well. It uses tabs to setup multiple synching jobs. It supports 1-to-1 synching or what’s known as multi-sync were synkron synchronises the sources one by one with their representing folder in the destination. This can be used for backups for example. It also has a scheduler/filters/blacklist and the ability to restore as well. The analyze function is also very useful as it can tell you what will be backed up before its backed up. Oh, and it’s open source/free software, so no excuses :-). Sep
22
2008
How to mount CD/DVD images from command line under LinuxHere are some quick CLI commands to get those CD/DVD images mounted under Linux (without having to burn them first).
# MDF # NRG Okay, now for a bit of CLI goodness. Here is a quick list of basic apt commands. Debian and most derivatives (Ubuntu) use these for package maintenance. #package info #clean #check #get source code #get dep #update/install/remove/upgrade Aug
28
2008
Remote OSX server install via Apple Remote Desktop or VNC….Came across this issue a few weeks ago and never got around to posting it until today. Yes, you can avoid using the Server Assistant program and go straight to ARD to install your server. For this to work your server and remote desktop machines need to be on the same subnet. You boot the server using the 10.5 CD/DVD and note it’s serial number. Now you need to find out the IP address of the server and connect to it from your remote desktop machine. For this, you can run ARD and scan the subnet and find your new server. Once you’ve found it connect to it and login using the first eight characters of the machines serial number and leave the username blank. This even works from Windows/Linux machines using VNC which is great for those who use Windows/Linux on their desktop. Aug
28
2008
UNetbootin takes care of all your USB Linux installtion blues….I’d covered Fedora’s Live USB creator a while back. But for those of you wanting to install other flavours of linux on a USB stick (or a HD even), the process was lengthy and cumbersome. That was before UNetbootin, the Universal Netboot Installer. You see UNetbootin can be run from Linux or Windows and will enable you to install a fully functional linux distro to a USB stick or even a spare partition. The current version has built-in support for the following distributions:
So go ahead and try some of those acquired flavors of Linux, or that FreeBSD you always wanted to install, but were too scared to. After all it’s only a USB stick, if it screws up just format and reinstall in 5-10 minutes using UNetbootin again. UNetbootin can also be used to load various system utilities, including:
So have fun and happy installing….. Jun
27
2008
JumpBox: Super simple way of getting web services deployed.
I used their parallel configuration on the Mac — JumpBoxes will run on all of the popular virtualization platforms including VMWare, Parallels, Microsoft Virtual PC/Server, Virtual Iron and Xen — and the static IP was all I had to configure to get the server up and running. If you have DHCP on your subnet/homerouter it’s even easier…..no thinking involved. |