Chu-Cheng's Public Note

Find all possible ways of representing a number n

2011-11-18T02:39:00.000-08:00

During one of my recent interview with a giant software company, I was asked this question:

Given an array of numbers, find all possible ways of representing a number n. For example, let's assume we have [2, 3, 5] in an array, and the target number is 10.

Well... I fail this interview in that I'm trying to work out a iterative solution given the stressful condition. I went home and I suddenly realize this is a manageable problem as long as I'm not greedy.

Tip 1: for all "combination" / "subset" questions, you should always start with hacking a recursive solution, because it's the most intuitive one.
Tip 2: never try to provide the "best" answer to your interviewer. Don't worry, they will challenge you and ask you to propose a better answer later.

Some people might argue that tip 2 looks stupid, especially if you know that recursive is usually awful in terms of space complexity. Don't get me wrong here, the time complexity for a permutation style question is O(N!), factorial, and for a subset style question is O(2^N), exponential. But hey...this is not the side effect of recursive, but it is the nature that the space you have to search !! Yes, the space complexity is awful, and you may fix by Dynamic Programming or Iterative solution. However, what you need in the interview is not a perfect answer, but is to show you ability of coding, and analyzing complexity. And don't worry, if you get this simple answer quickly, they will ask you to improve it. That's the time for "iterative" solution.

Here is the simple way of doing it:

import java.util.ArrayList;
import java.util.Arrays;

public class FindAllComb {
    public static void main(String[] args) {
        int[] ar = {3,5, 2};
        printCombForN(ar,10,0, new ArrayList());
    }


    //try = 0, meaning we start from the first element in the array
    public static boolean printCombForN(int[] ar, int target, int idx, ArrayList buffer){
        //base case: if no more choice
        if(target == 0) {
            return true;
        }
        else if (target < 0) {
            return false;
        }

        // for all possible choice
        for(int i=idx; i < ar.length; i++){
            //make a choice
            buffer.add(ar[i]);
            target-=ar[i];
            if (printCombForN(ar, target, i, buffer)){
                //print buffer
                System.out.println(Arrays.toString(buffer.toArray()));
            }
            //unmake the choice
            buffer.remove(buffer.size() -1);
            target += ar[i];
        }
        return false;
    }
}

It may looks hard at its first glance but essentially this is a simple back-tracking problem. For back-tracking problem, here is the template I follow:

bool Solve(configuration conf) //Credit: Zelenski, Julie
{
    if (no more choices) // BASE CASE
        return (conf is goal state);

    for (all available choices) {
        try one choice c;
        // solve from here, if works out, you're done
        if (Solve(conf with choice c made)) return true;
        unmake choice c;
    }
    return false; //tried all choices, no soln found
}

Can we do better?

Yes, with the help of DP.
The idea is that we can "memorize" things we did before.

For example, if we know th only possible choice of making 5 are (2,3) or (5), we can simple save it in a HashMap so that we don't have to go over same path again.

So let's slightly change the problem, what if we don't need you to print all possible combination, but we just want you to return number of possible combinations?

The idea here is Num(array, 10) = the sum of the following

Num(array, 8) + 1 (we deal with 2)
Num(array, 7) + 1 (we deal with 3)
Num(array, 5) + 1 (we deal with 5)

So you get the big picture now, if we have an array that index is the "target", which is the remainder after we recursively call the function , and the value is the "# of possible combinations" we already know. We can simple sum them up.

I will add answer later, but one thing you should keep in mind is that to make things faster, often times the price is space. In this example, we need extra space for tracking counts. Though, here (the revised question) the cost is acceptable.

Note that all back-tracking problem can be accelerated by some heuristic pruning. That is, you may "return false" earlier if you know there's no way you can get a goal state along the path.

Log (logging) in Python

2011-08-17T20:34:00.000-07:00

Logging is always an important topic when you try to develop a "real-world" app. I once wrote my own logging system, but apparently the best practice is to use something provided by Python Standard Library, not to reinvent the wheel.

Here, I have no intent to write a complete tutorial of teaching the python's logging framework. I extract the most useful/simple part(s) of logging from my viewpoints, which I believe it solves 90% cases I(you) need.

Firstly, you should try to build a customized logger. It is impractical to use the logging module directly via changing basicConfig() because you cannot tell where the log comes from if you have multiple modules in your app. This is one of the reason why I write this article.

To get a "customized" logger, firstly you should assign a name to it.

mlogging = logging.getLogger("test_log") #Get a logging for this module

If you would like to have some beautiful format on logging records, logging.Formatter is something you need to set it up. Sure you should do it because the default one seems to be a little clumsy. To change the date format (that is, %(asctime)s), please refer to things used in time.strftime(). And to create a format for a record, the related variables can be found in attributes of LogRecord.

# -- Set up Formatter --
log_format = '%(asctime)s(%(name)s--%(levelname)s):%(message)s'
log_date_format = '[%m/%d/%y %H:%M:%S]'
uniform_fmt = logging.Formatter(fmt=log_format, datefmt=log_date_format)

Mostly, we store logs in a file, so you need a file handler.

# -- Log to File --
hdlr = logging.FileHandler('OnlyOneFile.log')   # or "mlogger.name + 'log' if you would like to separate logs
hdlr.setFormatter(uniform_fmt)

If you would like to see log in your screen, say, for debugging, you have to link it to STDOUT.

# -- Log to Console --
hdlrConsole = logging.StreamHandler(sys.__stdout__)
hdlrConsole.setFormatter(uniform_fmt)

Do not forget to link your handler with your logger.

mlogging.addHandler(hdlr)
mlogging.addHandler(hdlrConsole)

Finally, you have to assign a log level so that you can get all the details. Otherwise, The default level is WARNING. (All default levels)

mlogging.setLevel(logging.DEBUG) #this is the most detailed level

Putting all together, here is an example:

import sys
import logging

mlogging = logging.getLogger("test_log") #Get a logging for this module
"""
logging.basicConfig(
    #filename=mlogging.name + '.log', #Use the name of logger as log file name
    filename='onefile.log', #Use the name of logger as log file name
    level=logging.DEBUG,
    format='%(asctime)s(%(name)s--%(levelname)s):%(message)s',
    datefmt='[%m/%d/%y %H:%M:%S]'
"""

# -- Set up Formatter --
log_format = '%(asctime)s(%(name)s--%(levelname)s):%(message)s'
log_date_format = '[%m/%d/%y %H:%M:%S]'
uniform_fmt = logging.Formatter(fmt=log_format, datefmt=log_date_format)

# -- Log to File --
hdlr = logging.FileHandler('OnlyOneFile.log')
hdlr.setFormatter(uniform_fmt)

# -- Log to Console --
hdlrConsole = logging.StreamHandler(sys.__stdout__)
hdlrConsole.setFormatter(uniform_fmt)

mlogging.addHandler(hdlr)
mlogging.addHandler(hdlrConsole)
mlogging.setLevel(logging.DEBUG)

mlogging.debug('This is debug msg')
mlogging.info('Ok~ info now')
mlogging.warning('Warning here')

try:
    x = 4/0
except Exception as e:
    mlogging.exception(e)

Before we leave the discussion, you should pay attention to mlogging.exception(e). This is the way we log an exception, including its trace-back information.

Git Version Control Resource

2011-07-12T11:19:00.000-07:00

I'm a heavily Subversion (SVN) developer. Recently, I join several projects that use git as their version control frameworks. Thus, I document some useful resource as follows:

* A quick tutorial
* A detail, through introduction: Git Book Recently, I happens to notice another good website:* Git Magic: This site is hosted in Stanford.

Firefox 5 on ubuntu 10.04 LTS

2011-07-12T10:27:00.000-07:00

For people who are using ubuntu 10.04, the default version of Firefox is 3.6. (Seriously?) It's suggested that you update to the latest stable version of ubuntu.

To install the latest "stable" version of ubuntu, add the following ppa repository:

sudo add-apt-repository ppa:mozillateam/firefox-stable

This is nothing more than add a mozillateam-firefox-stable-lucid.list to "/etc/apt/sources.list.d".

Than, do "sudo apt-get update && sudo apt-get upgrade" and you are done.

MySQL Remote Access

2011-03-08T18:59:00.000-08:00

By default, the mysql (5.x) allows login within localhost. What if you need to access a MySQL database from another computer?

I found this tutorial is quite helpful:
how-do-i-enable-remote-access-to-mysql-database-server

Unfortunately, I still get the following errors

chucheng@laptop:~$ mysql -ufoo -pcrap mysql.ucla.edu
ERROR 1044 (42000): Access denied for user 'foo'@'laptop' to...

After trying several solution, I finally realized that in the statement GRANT ALL ON db_name.* TO foo@'%' IDENTIFIED BY 'PASSWORD'; you cannot skip "IDENTIFIED BY password".

It turns out that you will "override" existing passwords with an empty password if you did not provide corresponding inputs.

Just in case that you still have problems:
Please make sure that
(1) Your bind-address in /etc/mysql/my.cnf has been changed to "real" IP.
(2) your port is listening
netstat -an | grep 3306
(3) You grant remote access permission to the user accounts.
p.s if you encounter any problems when login from localhost, instead of granting foo@'%', please grant foo@'localhost' as well.

Ubuntu Application

2011-02-04T09:54:00.000-08:00

I would like to document several apps that, I believe, are very useful as follows. Though there are many choices in WWW. The followings are really handy ones.

Programming Tools:

WingIDE (Python)
IntelliJ IDEA (Java)
Eclipse (C++/Java)
Rabbit VCS (Version Control)

MISC:

Mendeley
Stardict (Dictionary)

Brightness Control on T410 (T410s)

2011-02-01T17:16:00.000-08:00

I find the brightness control seems to be not working after installing ubuntu 10.04 on my Thinkpad T410s.

Here are two solutions:

(1) Switch to console (CTRL+ALT+F2) and adjust the brightness. Then switch back to GNOME (CTRL+ALT+F7)

(2) Edit /etc/X11/xorg.conf
Under the "DEVICE" section, add the following OPTION

Option "RegistryDwords" "EnableBrightnessControl=1"

Done! Enjoy it!

Install Sun Java SDK 6 on Ubutnu 10.04

2011-01-25T23:35:00.000-08:00

Since Ubuntu 10.04, Sun(Oracle?) SDK is not existed in multiverse anymore.

If you would like to install Sun Java SDK, you have to use "partner" repository.


sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"
sudo apt-get update
sudo apt-get install sun-java6-jdk

Is that all? Not quitely.

Choosing the default Java to use

Just installing new Java flavours does not change the default Java pointed to by /usr/bin/java. You must explicitly set this:

* Open a Terminal window
* Run sudo update-java-alternatives -l to see the current configuration and possibilities.
* Run sudo update-java-alternatives -s XXXX to set the XXX java version as default. For Sun Java 6 this would be sudo update-java-alternatives -s java-6-sun
* Run java -version to ensure that the correct version is being called.

You can also use the following command to interactively make the change;

* Open a Terminal window
* Run sudo update-alternatives --config java
* Follow the onscreen prompt

Ref: https://help.ubuntu.com/community/Java

Windows 7 HotKey in Ubuntu

2010-12-01T22:15:00.000-08:00

As a bi-OS user, I am always confused with the hot-key setting while I switch between Operating System. No offense, while I said bi-OS user, I refer to Windows 7 and Ubuntu Lucid. (Not Mac OS).

Some hotkeys introduced in Windows 7 are very userful for myself. For example, Windows + Left / Right / Up / Down

If you never try above hotkeys, try it on a Windows O/S, and you will know what would happen.
Long story short, I would like to resize my windows the the left/right window. What should I do?

(1) You have to install wmctrl package

 sudo apt-get install wmctrl

(2) Open "System"-->"Preference"-->"Keyboard Shortcuts".
Bound Windows + Left to ==> wmctrl -r :ACTIVE: -b remove,maximized_vert,maximized_horz;wmctrl -r :ACTIVE: -e 0,0,20,1920,1150
Bound Windows + Right to ==> wmctrl -r :ACTIVE: -b remove,maximized_vert,maximized_horz;wmctrl -r :ACTIVE: -e 0,960,20,960,1150

To Maximize a windows:
Bound Windows + Up to ==> wmctrl -r :ACTIVE: -b remove,maximized_vert,maximized_horz;wmctrl -r :ACTIVE: -e 0,0,20,1925,1150

Would you like to learn more?
Read here: wmctrl tutorial

Where is my "Ctrl + Alt + Backspace" ?

2010-11-23T11:27:00.000-08:00

After installing ubuntu 10.04, I quickly realized that CRTL-ALT-F1 (for opening a terminal) and CRTL-ALT-BACKSPACE (for killing X-server) are gone. Luckily, here is the solution I found:

Enabling Ctrl-Alt-Backspace for Ubuntu 10.04

Select “System”->”Preferences”->”Keyboard”
Select the “Layouts” tab and click on the “Layout Options” button.
Select “Key sequence to kill the X server” and enable “Control + Alt + Backspace”

Enjoy your hot keys :)

Move window buttons (Min, Max, Close) back to right in ubuntu 10.04

2010-10-14T00:00:00.001-07:00

One of the most controversial change in ubuntu 10.04 is to have window buttons on the left side. Although this change favors many mac(apple) users. As a user worked in dual O/S, windows 7 and ubuntu, the change is somehow inconvenient. Here I explain how to move them back. (The min, max, and close button)

1. Open the configuration editor of gnome.

gconf-editor

2. Click on the + button next to the “apps” folder, then click “metacity” in the list of folders expanded for apps, and then click on the “general” category. The button layout can be now changed by editing the “button_layout” key.

3. Change it to:

menu:minimize,maximize,close

Save, close, and you are done. (Restart X Window to take effect)

Weired ssh-agent error in Ubuntu

2010-08-24T13:50:00.000-07:00

To protect your private key, one might add a long pass-phrase for avoiding the burglary. However, somehow I see the following error while trying to add my private key to ssh-agent.

chucheng@ubuntuVM:~$ ssh-add
Could not open a connection to your authentication agent.

Here is the solution:
exec ssh-agent bash

Don't know why, but it just works :)

Results:

chucheng@ubuntuVM:~$ ssh-add
Enter passphrase for /home/chucheng/.ssh/id_rsa:
Identity added: /home/chucheng/.ssh/id_rsa (/home/chucheng/.ssh/id_rsa)

Rename mutliple files at once in ubuntu / linux

2010-07-06T20:31:00.000-07:00

I would like to rename multiple files at once in my Ubuntu 10.04.
I google around and I found people suggest the following command:

rename .bak .new *.bak

However, this magic does not work in my Ubuntu.
To rename a batch of files with their extension, the following syntax works:

rename 's/\.bak$/\.csv/' *.bak

The above command would rename all files with the extension ".bak" to the new extension ".csv".

Ubuntu 10.04 hang after login

2010-06-22T15:43:00.000-07:00

For some unknown reason(s), my newly install Ubuntu 10.04 went panic last Friday. Simply put, the ubuntu 10.04 hang after entering login information. I spend lost of hours on finding the problem. However, I still cannot see why it happens. It turns out that some package contains bugs and cause the o/s crashed after login.

I have tried the following means and none of them works:
- sudo apt-get install --reinstall ubuntu-desktop
- Update every package installed in the system
- Use the default xorg.conf setting (Without Xinerma)
- ... rm -rf home directory ...

Anyway, after keep trial and errors, here explains how I solve the problem.

1. Reboot to Recovery Model
It's a little tricky in Ubuntu 10.04 because by default the grub2 won't show up a start menu for you. The "default" in Ubuntu 10.04 always try to select the default booting item after power on the system. In addition, there's no more /boot/grub/menu.lst in ubuntu 10.04.

Here are solutions:
(1) Comment out "#GRUB_HIDDEN_TIMEOUT=0" through editing /etc/default/grub
(2) Run update-grub (or update-grub2 ?)
(3) Reboot and you will see the grub menu. Select the recovery mode and you are done.

2. Installed the following two package

sudo aptitude install libpam-gnome-keyring=2.92.92.is.2.30.0-0ubuntu3
sudo aptitude install gnome-keyring=2.92.92.is.2.30.0-0ubuntu3

If it works for you as well, please simply leave a comment here :)

Upgrade Debian 3.1(sarge) to 4.0(etch)

2009-11-18T20:36:00.000-08:00

It look like that to upgrade from Debian 3.1 to 4.0 should be fairly easy; however, the truth is that you will somehow trap by "libc6" loop. Here is a note to solve the problem.

Update 3.1 as best as possible. Modified the /etc/apt/sources.list

#OldStable
#deb http://security.debian.org/ lenny/updates main contrib non-free
#deb http://ftp.us.debian.org/debian/ oldstable main contrib non-free

sudo apt-get update

sudo apt-get upgrade

Switch to the etch sources, and update the kernel.

#etch
deb http://security.debian.org/ etch/updates main contrib non-free
deb http://ftp.us.debian.org/debian/ etch main contrib non-free

sudo apt-get update

sudo apt-get install linux-image-2.6.18-6-686

Pre-Processing
apt-get install locales

Make sure we handle all the dependency issue:
apt-get install -f

Install the kernel.

sudo apt-get install linux-image...

Update Grub by running: update-grub

Edit /boot/grub/menu.lst, set the Default to correct kernel

Edit the tow links to the right kernel:

/boot/initrd.img initrd.img -> initrd.img-2.6.8-3-686-smp
/boot/vmlinuz  -> vmlinuz-2.6.8-3-686-smp

, and then Reboot

Install Postfix on Ubuntu

2009-11-10T01:30:00.000-08:00

A step by step instruction to install a postfix server on ubuntu 8.10:

Install package.
sudo apt-get install postfix
1. type of mail configuration: Internet Site
2. enter your domain name: e.g. not.exist.edu (You have to register a domain name first) Free Register

Install mail utilities if you want to send a email through command line.
sudo apt-get install mailutils

Write a testing email.
echo testing_message | mail -s Subject_Not_Important youremail@some.domain.name
You should get your email in a second. If not, check your SPAM filter/box.

By default, the log files locate at /var/log
ls -l /var/log | grep mail
Check log files if you encounter any errors.

How to move mysql datadir to new location

2009-11-09T16:55:00.000-08:00

By default the mysql install the datadir at /var/lib/mysql. Unfortunately, people usually did partition their hard disk, or buy a new hard drive later. In such a case, you would like to move the *huge* datadir to another disk to both boost the speed by parallel in I/O and utilize space smartly. Here's what you should do:

Stop your mysql service.
sudo /etc/init.d/mysql stop

Copying all files from old location to new location and preserver the ownership and timestamp information.
sudo cp -r -p /var/lib/mysql /newlocation/

Update your mysql configuration file:
sudo vim /etc/mysql/my.cnf datadir = /newlocation/mysql #old: /var/lib/mysql


      Update the second configuration file for apparmor:
           /newlocation/mysql/ r, #/var/lib/mysql/ r,
  /newlocation/mysql/** rwk, #/var/lib/mysql/** rwk,

      (Optional) Rename the old folder to prevent confusing.
         sudo mv /var/lib/mysql /var/lib/mysql.bak

      Restart your apparmor & mysql service.
         sudo /etc/init.d/apparmor reload
         sudo /etc/init.d/mysql start

      Delete the old location
         sudo rm -rf /var/lib/mysql.bak



Recursively download entire website from a ftp
2009-09-29T21:22:00.000-07:00
The easiest way is the following command:
wget -r ftp://user:password @domain.name here


The only problem is that this method doesn't support "resume" function :)


Intall Python 3 on Ubuntu
2009-09-25T13:03:00.000-07:00
0. Install related packages:
   sudo apt-get install build-essential libncursesw5-dev libreadline5-dev libssl-dev libgdbm-dev libbz2-dev libc6-dev libsqlite3-dev tk-dev g++ gcc 

1. Download the file to /tmp:
   cd /tmp
   wget http://www.python.org/ftp/python/3.1.1/Python-3.1.1.tgz

2. Unzip the tar zip file:
   tar xzvf Python-3.1.1.tgz 

3. Run configuration command ./configure
   * In case you encounter some problem, install build-essential

4. make (optional)

5. Install the program to system:
   sudo make install


Appending to Your Python Path
2009-08-17T13:50:00.000-07:00

Question:
How do you append directories to your Python path?
Answer:
Your path (i.e. the list of directories Python goes through to search for modules and files) is stored in the path attribute of the sys module. Since path is a list, you can use the append method to add new directories to the path.

For instance, to add the directory /home/me/mypy to the path, just do:
    import sys
    sys.path.append("/home/me/mypy") 


src: Here



wget with proxy
2009-08-13T15:11:00.000-07:00
As a researcher, we often need to crawl the web pages and save a dump in local disk for future use. However, sometimes we use a proxy to connect a specific website for security concerns or performance issue. There are two ways to assign this job, the first method:

   Create a ~/.wgetrc file: This file can be used as a configuration file for wget command.
http_proxy=<your http proxy>

Reference: wget manual 
   

   Add a environment variable to your ~/.bashrc. export http_proxy="...url..."

   

Either way works well, but how about a proxy through socks proxy?

sorry... I cannot find a solution yet



Set password protection for you web pages
2009-08-10T16:48:00.000-07:00

Here is an simple scenario that comes to my mind. I created a website but I would like to protect some sub-directories from viewing by anonymous users. For example, you might created web pages to store all your bookmarks but you don't want to share it to the public but only limit to some close friends. Or, you are a instructor and you would like to share course website only to the students who attend the class. Okay... the solution is simple, you would like to create a password-protect directory through http server. 

You will need two files to achieve this goal. First, you need a file, .htaccess to store the user name and authentication type. Secondly, you need a file, .htpasswd, to store the password of the user you created in .htaccess file. 

Filename:.htaccess
AuthType Basic
# You can customized your authentication name
AuthName "Chucheng's Research Authentication"
# Where the .htpasswd located
AuthUserFile /home/chucheng/www/Research/.htpasswd
# Assign user name here
Require user chucheng


You can then generate the .htpasswd by typing the following code:
# The same username you used in .htaccess
$ htpasswd -c /home/chucheng/www/Research/.htpasswd chucheng


You can update the .htpasswd by typing the following code:
# The same username you used in .htaccess
$ htpasswd /home/chucheng/www/Research/.htpasswd-users chucheng


You are done! Enjoy your password-protected website :)




VirtuaWin - The Virtual Desktop Manager
2009-08-10T13:00:00.000-07:00
Those of you who had ever been Ubuntu must remember the multi-desktop support on GNOME. Unfortunately, neither Windows XP or Vista support this function by default. As an indulgent user of Windows, I soon understand that this function is a great plus to increase my working performance. After googling around, I found this open source project work almost perfect to meet my needs (with very light loading). Try it if you do need a multi-desktops working environment in Windows. Hopefully, Windows 7 can made this function built-in.


Official Site: VirtuaWin Official Site









Copyrights of above pictures belong to the official website



Free Python 3.x Tutorial
2009-08-04T18:00:00.000-07:00
This article shall be constantly updated to collect useful resource for learning python 3.x .

   Dive into Python 3

   I didn't buy this book because while I start this article, it hasn't be published yet, but definitely this is a very very good book. It's much better than the book I bought from amazon, Programming in Python 3 by Mark Summerfield. The book I bought is too difficult to understand, and the only reason I bought it is that it's one of the earliest book published for python 3.x. I'm still waiting for the publication of the Dive into Python 3, and I believe this book is better than the one I owned now.
   

   Official Python v3.1 documentation

   I believe that you can find all the update information from here. Anyway, it's something that you would always check for answers.




Make vim easy to use
2009-08-04T17:52:00.000-07:00
These are common VIM setting I use to make vim more easy to use. 

The following lines should be inserted to ~/.vimrc
#This can make the comment color become lighter (easy to read)
set bg=dark

#Smartindent is very helpful if you use vim for coding purpose
set smartindent

#4 spaces = 1 tab
set tabstop=4
set shiftwidth=4
set expandtab

#Press F5 to enable spell check
map  :setlocal spell! spelllang=en_us




Make the "ls --color" lighter and more readable
2009-08-04T17:46:00.000-07:00
In some Linux distribution, the default color skim is quite annoying. The "blue" is too dark of "ls --color" command, and it is almost invisible. Here is one tip to resolve the problem:

open ~/.bashrc and insert the following line:
#This works for BASH shell
export LS_COLORS="no=00:fi=00:di=01;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;00:cd=40;33;00:or=00;05;37;41:mi=00;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*.com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz=00;31:*.arj=00;31:*.taz=00;31:*.lzh=00;31:*.zip=00;31:*.z=00;31:*.Z=00;31:*.gz=00;31:*.bz2=00;31:*.bz=00;31:*.tz=00;31:*.rpm=00;31:*.cpio=00;31:*.jpg=00;35:*.gif=00;35:*.bmp=00;35:*.xbm=00;35:*.xpm=00;35:*.png=00;35:*.tif=00;35:"



Collection Types in Python 3
2009-07-31T16:06:00.000-07:00
I would like to document some useful snippet for handling collections in Python 3.x.
set:
 To add item to a set:
s.add(x)


 To check whether item is in a set:
x in s #return True if set s contains item x


 To remove item from a set:
s.remove(x)




Awk Command in Linux
2009-07-31T14:36:00.000-07:00
awk is a very useful command. I would document some very simple but useful example here. As a rule of thumb: we use "condition {expression}" in awk.

#Extract first column while the lines are comma seperated
awk '{FS=","}{print $1}'


#Check whether second column is "Artists", if so, print first and second columns.
awk '{FS="\t"} $2 = "Artists" {print $1 "\t" $2}'




Randomly picks n lines in a text file
2009-07-31T14:20:00.000-07:00
I try to google, but I cannot find the easy way to do this in command. Often times, I would like to randomly sample (without duplicate) lines in a text file. Here is my python code.

#!/bin/python3
# The goal of this script is to randomly choose n lines from a text files by line without duplicate

import sys, os, pdb, random

#The prefix will add to the front of each line
prefix = ""

if len(sys.argv) != 3 and len(sys.argv) != 4:
    print("syntax: randomlines.py <input_file> <num_of_sample>")
    sys.exit()
else:
    input_file = sys.argv[1]
    num_of_sample = int(sys.argv[2])

if len(sys.argv) == 4:
    prefix = sys.argv[3]

#pdb.set_trace()
lines = open(input_file, "r").read().splitlines()
if len(lines) < num_of_sample:
    print("The number of random lines you asked for is larger than the lines in the target file.\n" +
        "Shrink it to lines of target file automatically.\n" +
        "-----------------------------------------------\n")
    num_of_sample = len(lines)

chosen_lines_num = random.sample(range(0,len(lines)), num_of_sample)   #range(0,3) only generate 0,1,2
for i in chosen_lines_num:
    print(prefix + lines[i])



Enable Latex support in blogspot
2009-07-29T18:18:00.001-07:00
Here is an example:
$$\pi = \int_{0}^{1} \frac{4}{1+x^{2}}$$
becomes:
<a latex equation in pic>

Here explains all the details you need.

*Update*: The previous method doesn't work because the http://www.forkosh.dreamhost.com refuse to service other public website :)


Sorting Python Dictionary by Value
2009-07-29T17:39:00.000-07:00
This is a slower version:

sorted(adict.items(), key=lambda (k,v): v)


This is a faster version:

from operator import itemgetter
sorted(d.items(), key=itemgetter(1))


source: here

However, what if value is a list, and we would like to sort by the items inside the list?

For example, we have a dictionary variable x:
>>> x
{'a': [1, 2, 3], 'b': [0, 3, 1]}


We can sort them by the following command
>>> sorted(x.items(), key = (lambda k: k[0])) #sort by key
[('a', [1, 2, 3]), ('b', [0, 3, 1])]
>>> sorted(x.items(), key = (lambda k: k[1])) #sort by value
[('b', [0, 3, 1]), ('a', [1, 2, 3])]
>>> sorted(x.items(), key = (lambda k: k[1][0])) #sort by first item in value(a list)
[('b', [0, 3, 1]), ('a', [1, 2, 3])]
>>> sorted(x.items(), key = (lambda k: k[1][1])) #sort by second item in value(a list)
[('a', [1, 2, 3]), ('b', [0, 3, 1])]