Upgrade Debian 3.1(sarge) to 4.0(etch)

It look like that to upgrade from Debian 3.1 to 4.0 should be fairly easy; however, the truth is that you will somehow trap by "libc6" loop. Here is a note to solve the problem.

1. Update 3.1 as best as possible. Modified the /etc/apt/sources.list
   

   #OldStable
   #deb http://security.debian.org/ lenny/updates main contrib non-free
   #deb http://ftp.us.debian.org/debian/ oldstable main contrib non-free

   
   sudo apt-get update

sudo apt-get upgrade
   


2. Switch to the etch sources, and update the kernel.
   

   #etch
   deb http://security.debian.org/ etch/updates main contrib non-free
   deb http://ftp.us.debian.org/debian/ etch main contrib non-free

   
   sudo apt-get update

sudo apt-get install linux-image-2.6.18-6-686
   


3. Pre-Processing
   apt-get install locales
   
   Make sure we handle all the dependency issue:
   apt-get install -f
   


4. Install the kernel.
   
   sudo apt-get install linux-image...
   


5. Update Grub by running: update-grub
   
   Edit /boot/grub/menu.lst, set the Default to correct kernel
   


6. Edit the tow links to the right kernel:

   /boot/initrd.img initrd.img -> initrd.img-2.6.8-3-686-smp
   /boot/vmlinuz  -> vmlinuz-2.6.8-3-686-smp

   , and then Reboot

Install Postfix on Ubuntu

A step by step instruction to install a postfix server on ubuntu 8.10:

1. Install package.
   sudo apt-get install postfix
   
   
   
   1. type of mail configuration: Internet Site
   
   
   2. enter your domain name: e.g. not.exist.edu (You have to register a domain name first) Free Register
   
   
   
   


2. Install mail utilities if you want to send a email through command line.
   sudo apt-get install mailutils
   


3. Write a testing email.
   echo testing_message | mail -s Subject_Not_Important youremail@some.domain.name
   You should get your email in a second. If not, check your SPAM filter/box.
   


4. By default, the log files locate at /var/log
   ls -l /var/log | grep mail
   Check log files if you encounter any errors.
   

How to move mysql datadir to new location

By default the mysql install the datadir at /var/lib/mysql. Unfortunately, people usually did partition their hard disk, or buy a new hard drive later. In such a case, you would like to move the *huge* datadir to another disk to both boost the speed by parallel in I/O and utilize space smartly. Here's what you should do:

1. Stop your mysql service.
   sudo /etc/init.d/mysql stop


2. Copying all files from old location to new location and preserver the ownership and timestamp information.
   sudo cp -r -p /var/lib/mysql /newlocation/


3. Update your mysql configuration file:
   sudo vim /etc/mysql/my.cnf
   

   datadir     = /newlocation/mysql #old: /var/lib/mysql
   



4. Update the second configuration file for apparmor:
   

     /newlocation/mysql/ r, #/var/lib/mysql/ r,
     /newlocation/mysql/** rwk, #/var/lib/mysql/** rwk,



5. (Optional) Rename the old folder to prevent confusing.
   sudo mv /var/lib/mysql /var/lib/mysql.bak


6. Restart your apparmor & mysql service.
   sudo /etc/init.d/apparmor reload
sudo /etc/init.d/mysql start


7. Delete the old location
   sudo rm -rf /var/lib/mysql.bak

Recursively download entire website from a ftp

The easiest way is the following command:
wget -r ftp://user:password @domain.name here

The only problem is that this method doesn't support "resume" function :)

Intall Python 3 on Ubuntu

0. Install related packages:
sudo apt-get install build-essential libncursesw5-dev libreadline5-dev libssl-dev libgdbm-dev libbz2-dev libc6-dev libsqlite3-dev tk-dev g++ gcc

1. Download the file to /tmp:
cd /tmp
wget http://www.python.org/ftp/python/3.1.1/Python-3.1.1.tgz

2. Unzip the tar zip file:
tar xzvf Python-3.1.1.tgz

3. Run configuration command ./configure
* In case you encounter some problem, install build-essential

4. make (optional)

5. Install the program to system:
sudo make install

Appending to Your Python Path

Question:

How do you append directories to your Python path?

Answer:

Your path (i.e. the list of directories Python goes through to search for modules and files) is stored in the path attribute of the sys module. Since path is a list, you can use the append method to add new directories to the path.

For instance, to add the directory /home/me/mypy to the path, just do:

    import sys
    sys.path.append("/home/me/mypy") 

src: Here

wget with proxy

As a researcher, we often need to crawl the web pages and save a dump in local disk for future use. However, sometimes we use a proxy to connect a specific website for security concerns or performance issue. There are two ways to assign this job, the first method:

• Create a ~/.wgetrc file: This file can be used as a configuration file for wget command.
  

  http_proxy=<your http proxy>

  
  Reference: wget manual
  


• Add a environment variable to your ~/.bashrc.

  export http_proxy="...url..."

  
  
  

Either way works well, but how about a proxy through socks proxy?

sorry... I cannot find a solution yet

Set password protection for you web pages

Here is an simple scenario that comes to my mind. I created a website but I would like to protect some sub-directories from viewing by anonymous users. For example, you might created web pages to store all your bookmarks but you don't want to share it to the public but only limit to some close friends. Or, you are a instructor and you would like to share course website only to the students who attend the class. Okay... the solution is simple, you would like to create a password-protect directory through http server.

You will need two files to achieve this goal. First, you need a file, .htaccess to store the user name and authentication type. Secondly, you need a file, .htpasswd, to store the password of the user you created in .htaccess file.

Filename:.htaccess

AuthType Basic
# You can customized your authentication name
AuthName "Chucheng's Research Authentication"
# Where the .htpasswd located
AuthUserFile /home/chucheng/www/Research/.htpasswd
# Assign user name here
Require user chucheng

You can then generate the .htpasswd by typing the following code:

# The same username you used in .htaccess
$ htpasswd -c /home/chucheng/www/Research/.htpasswd chucheng

You can update the .htpasswd by typing the following code:

# The same username you used in .htaccess
$ htpasswd /home/chucheng/www/Research/.htpasswd-users chucheng

You are done! Enjoy your password-protected website :)

VirtuaWin - The Virtual Desktop Manager

Those of you who had ever been Ubuntu must remember the multi-desktop support on GNOME. Unfortunately, neither Windows XP or Vista support this function by default. As an indulgent user of Windows, I soon understand that this function is a great plus to increase my working performance. After googling around, I found this open source project work almost perfect to meet my needs (with very light loading). Try it if you do need a multi-desktops working environment in Windows. Hopefully, Windows 7 can made this function built-in.


Official Site: VirtuaWin Official Site









Copyrights of above pictures belong to the official website

Free Python 3.x Tutorial

This article shall be constantly updated to collect useful resource for learning python 3.x .

1. Dive into Python 3
   
   I didn't buy this book because while I start this article, it hasn't be published yet, but definitely this is a very very good book. It's much better than the book I bought from amazon, Programming in Python 3 by Mark Summerfield. The book I bought is too difficult to understand, and the only reason I bought it is that it's one of the earliest book published for python 3.x. I'm still waiting for the publication of the Dive into Python 3, and I believe this book is better than the one I owned now.
   


2. Official Python v3.1 documentation
   
   I believe that you can find all the update information from here. Anyway, it's something that you would always check for answers.
   

Make vim easy to use

These are common VIM setting I use to make vim more easy to use.

The following lines should be inserted to ~/.vimrc

#This can make the comment color become lighter (easy to read)
set bg=dark

#Smartindent is very helpful if you use vim for coding purpose
set smartindent

#4 spaces = 1 tab
set tabstop=4
set shiftwidth=4
set expandtab

#Press F5 to enable spell check
map  :setlocal spell! spelllang=en_us

Make the "ls --color" lighter and more readable

In some Linux distribution, the default color skim is quite annoying. The "blue" is too dark of "ls --color" command, and it is almost invisible. Here is one tip to resolve the problem:

open ~/.bashrc and insert the following line:

#This works for BASH shell
export LS_COLORS="no=00:fi=00:di=01;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;00:cd=40;33;00:or=00;05;37;41:mi=00;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*.com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz=00;31:*.arj=00;31:*.taz=00;31:*.lzh=00;31:*.zip=00;31:*.z=00;31:*.Z=00;31:*.gz=00;31:*.bz2=00;31:*.bz=00;31:*.tz=00;31:*.rpm=00;31:*.cpio=00;31:*.jpg=00;35:*.gif=00;35:*.bmp=00;35:*.xbm=00;35:*.xpm=00;35:*.png=00;35:*.tif=00;35:"

Collection Types in Python 3

I would like to document some useful snippet for handling collections in Python 3.x.
set:

To add item to a set:

s.add(x)

To check whether item is in a set:

x in s #return True if set s contains item x

To remove item from a set:

s.remove(x)

Awk Command in Linux

awk is a very useful command. I would document some very simple but useful example here. As a rule of thumb: we use "condition {expression}" in awk.

#Extract first column while the lines are comma seperated

awk '{FS=","}{print $1}'

#Check whether second column is "Artists", if so, print first and second columns.

awk '{FS="\t"} $2 = "Artists" {print $1 "\t" $2}'

Randomly picks n lines in a text file

I try to google, but I cannot find the easy way to do this in command. Often times, I would like to randomly sample (without duplicate) lines in a text file. Here is my python code.

#!/bin/python3
# The goal of this script is to randomly choose n lines from a text files by line without duplicate

import sys, os, pdb, random

#The prefix will add to the front of each line
prefix = ""

if len(sys.argv) != 3 and len(sys.argv) != 4:
    print("syntax: randomlines.py <input_file> <num_of_sample>")
    sys.exit()
else:
    input_file = sys.argv[1]
    num_of_sample = int(sys.argv[2])

if len(sys.argv) == 4:
    prefix = sys.argv[3]

#pdb.set_trace()
lines = open(input_file, "r").read().splitlines()
if len(lines) < num_of_sample:
    print("The number of random lines you asked for is larger than the lines in the target file.\n" +
        "Shrink it to lines of target file automatically.\n" +
        "-----------------------------------------------\n")
    num_of_sample = len(lines)

chosen_lines_num = random.sample(range(0,len(lines)), num_of_sample)   #range(0,3) only generate 0,1,2
for i in chosen_lines_num:
    print(prefix + lines[i])

Enable Latex support in blogspot

Here is an example:
$$\pi = \int_{0}^{1} \frac{4}{1+x^{2}}$$
becomes:
<a latex equation in pic>

Here explains all the details you need.

*Update*: The previous method doesn't work because the http://www.forkosh.dreamhost.com refuse to service other public website :)

Sorting Python Dictionary by Value

This is a slower version:

sorted(adict.items(), key=lambda (k,v): v)

This is a faster version:

from operator import itemgetter
sorted(d.items(), key=itemgetter(1))

source: here

However, what if value is a list, and we would like to sort by the items inside the list?

For example, we have a dictionary variable x:

>>> x
{'a': [1, 2, 3], 'b': [0, 3, 1]}

We can sort them by the following command

>>> sorted(x.items(), key = (lambda k: k[0])) #sort by key
[('a', [1, 2, 3]), ('b', [0, 3, 1])]
>>> sorted(x.items(), key = (lambda k: k[1])) #sort by value
[('b', [0, 3, 1]), ('a', [1, 2, 3])]
>>> sorted(x.items(), key = (lambda k: k[1][0])) #sort by first item in value(a list)
[('b', [0, 3, 1]), ('a', [1, 2, 3])]
>>> sorted(x.items(), key = (lambda k: k[1][1])) #sort by second item in value(a list)
[('a', [1, 2, 3]), ('b', [0, 3, 1])]