kenics.net

Technical notes on perl, python, php, sql, cgi, c/c++, q/kdb+, unix/shell, revision control tools, data structures & algorithms, and their applications into web services and other various forms of software engineering.

unix general concepts shell regex cmd

 
################################## 
###   metasyntactic variable   ### 
################################## 
 
it is an example variablelike like foo bar 
 
 
########################### 
####    bit wise ops    ### 
########################### 
 
xor;  (~x|~y) & (x|y) 
 
use >> << in place of multiply/divide by 2^n or wipe certain bits. 
 
 
################ 
###   bash   ### 
################ 
 
$$        # current shell PID 
!!        # repeates the prev command 
!$        # output of the prev command 0=success 
!<num>    # history 
^foo^bar  # substitution 
$!        # PID of the most recent background process (a cmd that got executed with &) 
 
tty  - print the file name of the terminal connected to standard input 
 
escape brackets  \[  \] 
escape char  \033 \e \E 
 
 
for((i=0;i<6;i++)); do echo `expr $i \* 2`; done     ## for i in {1..6};  works too 





10 
 
 
## to format better; 
 
$ for i in 0{7..9} {10..12}; do echo $i; done 
07 
08 
09 
10 
11 
12 
 
 
## when needing to monitor the file size 
 
while /bin/true ; do ls -s afile; sleep 5; done 
 
 
## a simple if else conditional statement 
 
FOO=0 
if [ $FOO -eq 34 ]; then echo 1; elif [$FOO -eq 789]; then echo 123; else echo 0; fi 
if [ $FOO -eq 34 ] && [ $USER = 'kenics']; then echo 1; else echo 0; fi 
if [ $FOO -eq 34 ] || [ $USER = 'kenics']; then echo 1; else echo 0; fi 
 
 
  Operator             | Description 
----------------------------------------------------------------------------- 
! EXPRESSION           | negate EXPRESSION 
-z STRING              | STRING length is zero (i.e. empty) 
-n STRING              | STRING length is non zero 
STRING1 = STRING2      | STRING1 is equal to STRING2 (notice syntax, it is a single =) 
STRING1 != STRING2   | STRING1 is not equal to STRING2 
INTEGER1 -eq INTEGER2  | INTEGER1 == INTEGER2 (notice syntax) 
INTEGER1 -gt INTEGER2  | INTEGER1 > INTEGER2 
INTEGER1 -ge INTEGER2  | INTEGER1 >= INTEGER2 
INTEGER1 -lt INTEGER2  | INTEGER1 < INTEGER2 
INTEGER1 -le INTEGER2  | INTEGER1 <= INTEGER2 
-e FILE                | FILE exists. 
-d FILE                | FILE is a directory 
-s FILE                | FILE size is non zero (i.e. non-empty) 
-r FILE                | FILE is readable 
-w FILE                | FILE is writable 
-x FILE                | FILE executable 
 
 
################################################################## 
####   shell meta-characters (not to be confused with regex)   ### 
################################################################## 
 
?       ## any single char 
*       ## any string including zero or many 
[abcd]  ## matches a single char 'a' 'b' 'c' or 'd'    e.g. [a-zA-Z0-9] 
[!abcd] ## matches a single char NOT 'a' 'b' 'c' 'd' 
 
# how many words in the online dictionary contain a sequence of vowels; a,e,i,o,u? 
# (the vowels must appear in order but not consecutively) 
 
grep '.*a[^eiou].*e[^iou].*i[^ou].*o[^u].*u.*'  /usr/share/dict/words 
 
 
 
################### 
###    regex    ### 
################### 
 
(1) meta-characters 
.             # any single char but new line 
[a-zA-Z0-9]   # any single char in the brackets 
[^a-z] 
 
(2) anchors 
^             # beginning 
$             # end 
 
e.g. 
grep -v '^$' afile   ## look for non empty lines 
grep '^[ ]*$' afile  ## look for whitespace lines 
 
(3) quantifiers 
*       # zero or more occurence of preceding char 
?       # zero or one time of preceding char 
+       # one or more times of preceding char 
\{n\}   # repeats n times 
\{n,\}  # repeats at least n times 
\{n,m\} # repeats between n and m times 
 
e.g. 
grep 'd\{2\}' afile 
 
(4) groups 
\(\) 
 
#### 
#### protection of regex from shell interpretation 
#### 
==> use quotes 
==> ""  # protects from *|><  but not $ 
==> ''  # protects from everything 
==> \   # backslash works 
==> ``  # executes the embedded cmd 
 
e.g. 
grep '\$' afile               # looks for "$" mark 
grep '^...$' afile            # look for 3-char string 
grep '^.\{3\}$' afile         # the same 
grep '^[a-zA-Z]\{3\}$' afile  # look for 3-letter string 
 
 

#  grep for a tab 
#  cut by a tab 

 
grep $'\t' afile               # notice the dollar sign 
cut -d $'\t' -f 2,4 afile 
 
 
############################# 
###   general unix cmds   ### 
############################# 
 
### 
### /usr/bin/man <cmd>     #  or google 
### 
 
#### 
####  sort / uniq 
#### 
 
sort alist              # alphabetical 
sort -d alist           # dictionary 
sort -n alist           # numerical 
sort -r -k 3 alist      # reverse, by column 3 
sort -k4nr -k5nr alist  # sort by column 4, then by column 5 
sort -t, alist          # "-t" is for delimiter 
 
sort afile | uniq   #  uniq -u(only unique lines) -d(only dupe lines) 
                    #  "sort" the content first is critical. otherwise, uniq does not work 
 
e.g. (you can list how many connections each host has to a particular port on the host) 
lsof -iTCP | grep <some_port_name> | awk <some_column> | uniq -c 
34 hostA 
25 hostB 
17 hostC 
 
 
#### 
####  wc 
#### 
 
wc <file>    # prints how many (1)lines (2)words (3)chars 
wc -l <file> # prints how many lines 
 
 
#### 
####  head / tail 
#### 
 
head <file> -n XXX   # print first XXX lines.  without "-n" then XXX=10 by default 
tail <file> -n XXX   # we can alternatively write  "head -15 <file>"  and  "tail -15 <file>" 
 
tail -f <file>       # keep monitoring the file as it grows. 
 
e.g. 
head -200 <file> | tail -150     # prints line 50 to 200 
 
 
#### 
####  ls 
#### 
 
ls -s # size in block 
ls -S # sort by size 
ls -F # format.  shows dir and files. executables are followed by "*" 
ls -h # human readable (size) 
ls -i # inode info 
 
# Devise a cmd which lists the files in the current direcotry in the order of largest first, but only displays their files names. 
 
ls -l  | sort -k5nr | awk '{print $9}' 
ls -lS | awk '{print $9}' 
 
 
#### 
####  df/du 
#### 
 
$ df -h / 
Filesystem            Size  Used Avail Use% Mounted on 
/dev/sda2             118G  6.9G  105G   7% / 
 
$ du -h ./ --exclude='foobar*' 
-> shows how much disk space is used by files under current directory excluding any files whose names start with foobar 
 
 
#### 
#### diff/cmp/comm cmd 
#### 
 
diff file1 file2 
< aaa   # only in file1 
--- 
> bbb   # only in file2 
 
diff -c file1 file2  # shows the whole thing and highlights the diff parts with "!" 
diff -wb file1 file2 # ignores whitespace difference  -b only ignores tab-whitespace diff and trailing space, but -w completely ignores whitespace/tab, so "test" and "te st" will match 
 
(ref) https://superuser.com/questions/613556/diff-b-and-w-difference 
 
diff <(ls dirA) <(ls dirB)      # this is how to compare the output of two commands 
 
cmp file1 file2    # binary level comparison 
 
comm file1 file2   # displays (left) only-in-file1, (middle) only-in-file2, (right) in-both-files 
                   # contents have to be sorted in the first place 
 
#### 
#### cut/awk/tr/fold 
#### 
 
cut -c10-30 afile   # print 10th to 30th chars from every line of afile 
cut -b10-30 afile   # the same assuming it is a byte char encoding 
cut -c-30 afile     # 1 to 30 
cut -c10- afile     # 10 to the last 
cut -d : -f 1,3     # -d delimitor=":" and -f field 1 and 3 
cut -d' ' -f3       # delimit by a whitespace 
 
NOTE: how to delimit by multiple chars (e.g. delimite by a whitespace or a comma) ? - just sed and convert them to one same char 
 
cat afile | awk -F":" '{print $2}' 
 
cat afile | tr -d ns     ## -d deletes any occurence of 'n' and 's' 
tr -d ns < afile         ## the same as above 
tr -s ns < afile         ## -s suppresses any repeats of 'n' and 's' 
tr -s " " < afile        ## suppresses any multiple occurence of whitespace into single space 
tr ab xy < afile         ## swap 'a' to 'x', and 'b' to 'y' 
tr [a-z] [A-Z] < afile   ## lower case to upper case 
echo hello | tr -d '\n'  ## same as $ echo -n hello    you can alternatively do tr '\n' '' 
 
fold -w 20 README  # fold the width of each line to 20 chars 
 
 
e.g.   often we have some directory like /proc/log  where we keep logs of various processes/jobs. 
       as a high level check, you may want to see if any jobs missed some scheduled run. 
 
$ ls -1 /proc/log 
foo.20141229.01 
foo.20141229.02 
foo.20141229.03 
foo.20141229.04 
foo.20141230.01 
foo.20141230.02 
foo.20141230.03 
foo.20141230.04 
foo.20141231.01 
foo.20141231.02 
foo.20141231.03 
foo.20141231.04 
var.20141231.01 
var.20141231.02 
var.20141231.03 
var.20141231.04 
 
$ for i in `ls /proc/log`; do echo -n $i | grep foo | awk -F'.' '{print $2}'; done | sort| uniq -c 
    4 20141229 
    4 20141230 
    4 20141231 
 
 
e.g.  if-else conditional statement with awk 
e.g.  calculating an average of a column with awk 
 
suppose you have a csv file, and you want the average value, rounded to int, of the second column where it shows more than 200. 
here is how 
 
$ awk -F',' '{if($2 > 200) {sum+=$2 ;n++ ;} } END {print int(sum/n);}' some_data_file.csv 
 
$ awk -F'|' '{if($0 == "foobar"){print $3}}' some_file 
 
 
#### 
####  column 
#### 
 
useful to format the output 
 
e.g. 
 
cat somefile.csv | column -s, -t 
 
 
#### 
####  last / w          # to check who logged in when / who is still logged in/ what proc they are runnning 
#### 
 
#### 
####  who -b / uptime    # one way to know last reboot time 
#### 
 
$ uptime 
 19:37:33 up 89 days,  9:33, 13 users,  load average: 0.02, 0.03, 0.00 
 
 
#### 
####  bc -l            # basic calculator 
#### 
 
$ echo 123 + 3242| bc -l 
3365 
 
$ echo 123 / 3242| bc -l 
.03793954349167180752 
 
$ echo 123 \* 3242.3| bc -l    # notice escape * is a must 
398802.9 
 
 
#### 
####  find 
#### 
 
find . -name "*init*"           # find any files that contains "init"  recursively under current dir 
find . -iname "*init*"          # case insensitive 
find . -name "foo*" -type f     #  "f" is regular file, "d" is directory 
find -L . -name "*init*"        #  -L means follow symlinks (notice the position of -L. it has to be before the location) 
find -P . -name "*init*"        #  -P means NOT follow symlinks 
 
 
find ./foo -mtime -3    # finds all files under 'foo' dir, with modified time within last 3 days. 
find ./foo -mtime +5    # finds all files under 'foo' dir, with modified time older than 5 days. 
 
find . -name "*temp*" -mtime +365 -print               # print is chosen by default 
find . -name "*temp*" -mtime +365 --delete             # notice delete doesn't prompt you to confirm. so be careful. 
find . -name "*temp*" -mtime +365 -exec rm -i {} \;    # equivalent but prompts for confirmation 
 
find . -name "*.pl" -perm 755 --user kenics -size +100   # finds perl files owned by user 'kenics' with perm 755, with size larger than 100 blocks (1 block is 512 bytes, i.e. 2 blocks are 1KB) 
 
# then you can do all sorts of stuff with -exec 
 
find ./foo -type f -name "*.py" -exec chmod 755 {} \; 
find -L ./foo -type d -exec chmod g+s {} \; -ls 
find ./ -type f -name "*.txt" -exec sed -i 's/foo/bar/g' {} \; 
find ./ -type f -name "*.txt" -exec grep 'foo'  {} \; 
 
#  '-not'  lets you negates options 
 
find . -not -name "*.exe" -not -type d -not -user kenics     # find all fines recursively under current dir, except directories, except files ending with .exe owned by kenics 
 
 
############## 
####  ed  #### 
############## 
 
ed afile        # open a file 
a               # append 
this is a test 
hello world 
.               # terminates 
w               # save 
1p              # print the first line 
$               # print the last line 
1,$p            # print the content from line 1 to the last line 
%p              # the same 
3i              # insert to line 3 
hello line 3 
3d              # delete line 3 
 
/hogehoge/c 
foobar          # grep for "hogehoge" and replace the entire line with "foobar" 
 
s/foo/bar       # replace "foo" with "bar" for the first instance 
s/foo/bar/g     # replace "foo" with "bar" for the first line 
1,$s/foo/bar    # replace "foo" with "bar" for all lines # but without "g" it will only change the first instance of each line 
1,$s/foo/bar/g 
 
/hoge           # search for hoge. only the first instance 
g/hoge          # search globally for hoge. all instance 
g/hoge/d        # search hoge-contained lines and delete them all 
 
1,3t            # copy line 1-3 to the end of file 
1,3t 4          # copy line 1-3 to the line 4 
 
/jack/,/tom/d   # find the first occurence of "jack" till the first occurence of "tom", delete all lines 
 
w README3       # output to README3 
 
3x              # paste whatever was deleted last after line 3 
 
 
 
################### 
###     sed     ### 
################### 
 
"-i" in-place edit of the input file 
"-e" is needed when supplying multiple cmds 
"-n" means only print the matching line. otherwise it prints everything 
"$p" means last line 
"p"  means print 
"w"  means outout to a file 
 
sed -n -e '1,4p' -e '6,$p' afile   ## print all lines of afile except line 5 
sed -n '5p' afile                  ## print line 5 
sed -n '/foobar/p' afile           ## print all lines containing 'foobar' 
 
sed -n 's/foobar/matrix/p' afile   ## replace 'foobar' with 'matrix' in afile and print the applicable line 
 
sed -n '/^foobar/w bfile' afile    ## write all lines starting with "foobar" to afile 
 
sed -n '/foobar[0-9]/s/foobar/foobarTAB/p' afile 
## look for "foobar" followed by a number then insert TAB in between and print 
 
sed -i 's/foo/bar/' afile          ## replace "foo" with "bar" once in every line in afile (and save) 
sed -i 's/foo/bar/g' afile         ## replace "foo" with "bar" (all occrences) in every line in afile (and save) 
sed -i 's/\s\{1,\}/ /g'  afile     ## replace any contiguous whitespace into a whitespace char 
 
sed '/foobar/d' afile    # delete lines that contain foobar 
                         # if you only want to delete string foobar, then you do sed 's/foobar//' afile 
 
 
 
#### 
#### file/type/which cmd 
#### 
 
file <file_name>    # tells if it is text/executable/directory, etc 
type <cmd_name>     # tells alias, shell-built-in function, etc 
which <cmd_name>    # tells where the cmd is defined 
stat <file_name>    # shows inode info 
 
 
#### 
#### chmod cmd - file permission 
#### 
 
    (u) (g) (o) 
sst rwx rwx rwx 
421 421 421 421 
 
chmod 1755 afile 
chmod u-x,g+rx,o-rw afile 
 
### 
###  chgrp <group_name> <dir|file> 
### 
 
chgrp -R newGroupName ./someDir      # you can do this to all files in a dir recursively 
                                     # notice in redhat, i saw this changes timestamp and owner of the files/dirs too 
 
 
 
############################### 
###   user administration   ### 
############################### 
 
/etc/passwd 
/etc/shadow 
/etc/group 
 
uid=0 means super user. not user name being root. 
uid <= 500 are admin accounts 
 
man 5 passwd  # shows the format of the content 
man 5 shadow 
man 5 group 
 
groupadd -g GID <group_name> 
useradd -u UID -g GID -c "some desription" -d /home/mel -s /bin/bash <user_name> 
 
passwd -n min -x max -w warn <user_name>  # -l = lock the account, -u = unlock 
chage  <user_name> -d 2013-09-13          # expire the pw on that day 
 
userdel -r <user_name> 
 
 
### 
###  id, getent 
### 
 
$ id root 
uid=0(root) gid=0(root) groups=0(root) 
 
$ id mel 
uid=10645(mel) gid=1000(rg) groups=1000(rg) 
 
# lets you find out textual user name via uid 
$ getent passwd 10645 
mel:x:10645:1000:K Sugimoto, s06532ks:/home/mel:/bin/tcsh 
 
 
#### 
#### od/strings cmd 
#### 
 
stinrgs afile   # displays in strings, not in binary 
 
od afile      # octed dump 
od -x afile   # hex 
od -o afile   # octal 
od -d afile   # decimal 
od -c afile   # newline='\r' '\n' 
od -c afile   # newline-'CR' 'LF' 
 
 
### 
###  rm 
### 
 
-i  # promts for confirmation 
-r  # force remove 
-f  # force remove folders and its contents 
 
### 
### less / more 
### 
 
/<string>  # search for "string" 
n          # look for next occurrence of "string" 
shift + G  # end of file 
%          # beginning of file 
 
 
### 
### /usr/bin/script 
### 
 
from the moment it's run, until "exit" is run, it records everything into "./typescript" file 
 
 
 
 
### AFS ### 
 
fs la 
fs sa 
 
 
########################### 
####   process/shell   #### 
########################### 
 
everything in unix is a process or a file. 
 
ls ; ls ; ls              ## sequential composition 
cd tmep && pwd            ## conjunctive composition 
( sleep 5 ; ls ; ls ) &   ## subshell background command execution 
ls & ls & ls              ## parallel composition 
ls || ls || ls            ## disjunctive composition 
 
tee -a afile | ksh -i 2>&1 | tee -a afile 
 
ps -ef | grep "ksh|bash"  ## the startup shell will show "-" as in "-ksh" 
                          ## this is how you identify the startup shell 
 
 
set   # shows all the local variables 
env   # shows all the global variables 
 
echo $$    # print current shell PID (bash) 
 
 
##################### 
###   capacity    ### 
##################### 
 
/usr/bin/top       # various options available to sort by CPU, mem, etc 
 
# top 15 mem consumer procs 
$ ps alx | awk '{print $8,$2,$13}' | sort -nr | head -15 | column -t    # column 8 = RSS (consumed mem size) 
                                                                        # column 2 = UID 
                                                                        # column 13 = cmd 
$ du -a ./ | sort -nr | head -n 10 
 
$ uptime 
$ who -b 
 
 
####################### 
###   redirection   ### 
####################### 
 
stdin  0 
stdout 1 
stderr 2 
 
cat afile > bfile           ## i.e. $ cat afile 1> bfile 
tr 'hoge' 'foo' < ./afile   ## i.e. $ tr 'hoge' 'foo' 0< ./afile 
2>&1                        ## redirecting stderr to stdout 
ls 1>log_file 2>&1          ## all output to log_file 
 
 
##################### 
###   proc ctrl   ### 
##################### 
 
ctr-z 
bg 
fg 
jobs 
kill %1    ### job number from jobs cmd 
ps -ef 
kill PID   ### pid from ps cmd 
kill -l    ### shows all signals on OS 
 
e.g. 
$ kill -l 
 1) SIGHUP       2) SIGINT       3) SIGQUIT      4) SIGILL       5) SIGTRAP 
 6) SIGABRT      7) SIGBUS       8) SIGFPE       9) SIGKILL     10) SIGUSR1 
11) SIGSEGV     12) SIGUSR2     13) SIGPIPE     14) SIGALRM     15) SIGTERM 
16) SIGSTKFLT   17) SIGCHLD     18) SIGCONT     19) SIGSTOP     20) SIGTSTP 
21) SIGTTIN     22) SIGTTOU     23) SIGURG      24) SIGXCPU     25) SIGXFSZ 
26) SIGVTALRM   27) SIGPROF     28) SIGWINCH    29) SIGIO       30) SIGPWR 
31) SIGSYS      34) SIGRTMIN    35) SIGRTMIN+1  36) SIGRTMIN+2  37) SIGRTMIN+3 
38) SIGRTMIN+4  39) SIGRTMIN+5  40) SIGRTMIN+6  41) SIGRTMIN+7  42) SIGRTMIN+8 
43) SIGRTMIN+9  44) SIGRTMIN+10 45) SIGRTMIN+11 46) SIGRTMIN+12 47) SIGRTMIN+13 
48) SIGRTMIN+14 49) SIGRTMIN+15 50) SIGRTMAX-14 51) SIGRTMAX-13 52) SIGRTMAX-12 
53) SIGRTMAX-11 54) SIGRTMAX-10 55) SIGRTMAX-9  56) SIGRTMAX-8  57) SIGRTMAX-7 
58) SIGRTMAX-6  59) SIGRTMAX-5  60) SIGRTMAX-4  61) SIGRTMAX-3  62) SIGRTMAX-2 
63) SIGRTMAX-1  64) SIGRTMAX 
 
 
e.g. 
jobs 
[1]  Stopped  sleep 30 
[2]  Stopped  sleep 35 
[3]- Stopped  sleep 20    # "-" is the second most recently suspended 
[4]+ Stopped  sleep 10    # "+" is the most recently suspended. this gets invoked by "fg" 
 
kill -TERM PID   # Ctr-C 
kill -STOP PID   # Ctr-Z 
kill -9    PID   # absolution 
 
 

#  tricky EPIPE/SIGPIPE error 

 
(assume bash) 
when you do "cat a_big_file | head -n 1"  then, in addition to your expected result, you may also get this bash error msg "cat: write error: Broken pipe" 
this is because while cat is still outputting the file content to the pipe, the receiving end head did its job and exitted, at that point it sends the signal to the transmitting side that there is no longer a recepient, thus cat terminates with the write error broken pipe. 
this is not necessarily a bad thing, and in fact, in the above example, it is perfectly fine, no harm. but obviously the msg itself can confuse you to think something went wrong. 
 
i first saw the error when i did the below in perl 
------------ 
open (IN,"cat $file |"); 
while(<IN>){ 
   next if($_ =~ /header/);  # skip header 
   last if($_ =~ /foobar/);  # some keyword that indicates the rest of the content is not needed. 
   ... 
   .. 
   _process_content_as_needed_ 
   .. 
   .. 

close IN; 
------------ 
 
===> i got "cat: write error: Broken pipe" whenever i broke out of the loop with "last" because as you can see, it's actually the same as the above cat-head example. 
 
 
### 
###  tar/compress/uncompress/gzip/gunzip/bzip2/bunzip2/zip/unzip/lha/ar 
### 
 
$ tar cvf /tmp/archive.tar afile bfile  ## arhives the afile & bfile into arhive.tar 
                                        ## (never specify the abs path to the target file = afile, bfile) 
$ tar tvf /tmp/archive.tar              ## list the contents 
$ tar xvf /tmp/archive.tar              ## de-compress. be careful where you are de-compressing. it might overwrite files. 
                                        ## especially if you specified the abs path of target file when you arhived, it will expand to that very path. 
 
$ tar cvf stu.tar student --exclude='ken*'  ## exclude names that start with ken 
 
$ compress afile           # creates afile.Z 
$ uncompress afile.Z       # output = afile 
 
$ gzip afile               # afile.gz 
$ gunzip afile.gz          # afile 
 
$ bzip2 afile              # afile.bz2 
$ bunzip2 afile.bz2        # afile 
 
$ zip archv.zip afile bfile # archv.zip  containing afile & bfile 
# zip -r archv.zip afolder  # recursive 
$ unzip archv.zip           # afile bfile 
$ unzip -l archv.zip        # list its contents 
$ unzip -p archv.zip afile  # preserve the zip and only output to standard out stream. 
 
$ lzh -c archv.lzh afile   # "-c" new archive 
$ lzh -x archv.lza         # afile "-x" for unzip 
 
$ ar -r arch.a afile bfile # "-r" for archiving 
$ ar -x arch.a             # "-x" for unzip 
 
 
### 
###  nslookup/whois/host 
### 
 
nslookup <hostname/ipaddr>   ## does DNS lookup 
host <hostname/ipaddr> ## does DNS lookup 
whois <URL/ipaddr>           ## searches for an object in a RFC 3912 database(directory) 
 
e.g. 
$ nslookup ntt7-ppp890.tokyo.sannet.ne.jp 
Server:         dns-hoge.foobar.com         ## DNS server name 
Address:        203.178.142.130             ## DNS server ipaddr 
 
Non-authoritative answer: 
Name:   ntt7-ppp890.tokyo.sannet.ne.jp      ## hostname 
Address: 101.102.62.128                     ## ipaddr 
 
$ host cpu 
cpu.sfc.wide.ad.jp has address 203.178.142.142 
cpu.sfc.wide.ad.jp has IPv6 address 2001:200:0:8803:203:178:142:142 
 
 
######################## 
####     inode      #### 
######################## 
 
a proper place to discuss inode is in the context of file systems. in unix, everything is either a process or a file. 
even a directory is just a file that has a list of file names. a sym link is another file. 
 
inode is a data struct that contains all the attributes(owner user id, size, timestamp, link counts, etc) about a file. 
a file just contains data. 
 
NOTE: inode does not contain a filename as its attribute. this is important to note, see the below Soft vs Hard link discussion. 
 
 
#### 
####  ln -s   Soft vs Hard links 
#### 
 
(SOFT) 
ln -s <fileA> <fileB>    # create a symlink/alias fileB to an actual file fileA 
                         # fileA can be a symlink to another file 
                         # fileA can be a non-existent file name (a symlink to a non-existent file) 
                         # soft link is file system level link. 
e.g. 
you edit fileB content, then obviously fileA gets updated. 
you remove fileA, then fileB is a symlink to a non-existent file. 
 
NOTE: man ln. commonly used options with -s are  -f for force, -n for overwrite. 
 
(HARD) 
ln afile bfile           # creates a hard copy bfile to an original file afile. 
                         # inode number is the same (i.e. same disk space). check "link count" in its inode, it is increasing 
                         # because link count tracks how many hard links. 
                         # because inode is the same, hard link is a kernel level (its directory table) link. 
                         # thus hard link cannot be made across diff file systems. 
 
e.g. 
you edit bfile, then afile content will sync with bfile. 
you remove afile, bfile will persist obviously, it is just another link to that inode. 
this means, to remove a file, you must remove ALL hard links. 
 
suppose user A creates a file, and user B wants to create a hard link to it. 
then the file must be writable to user B. (big diff from soft link) 
this gets interesting, because suppose each user only has 10GB quota. 
then if user A's 9GB file is writable to user B who creates a hard link to it. 
then even if user A deletes it, user B owns 9GB of user A's space. duh. so hard link is tricky. 
 
for the same reason, chown is dangerous and usually reserved to root. 
 
overall, hard link is not as useful, because every user who can create a hard link has write-access. 
 
You can not create a hard link to a directory. All links are owned by the original file's owner and are subject to the permissions on that file. The user that creates the link can remove the link. The file information (inode) will not be removed from the disk until all of the links that are pointing to it are removed. 
 
https://unix.stackexchange.com/questions/172285/hard-linking-other-users-files 
 
 
############################################## 
####     multiple slash in a dir path     #### 
############################################## 
 
they are treated as a single slash. 
 
cd /tmp/foo/bar 
cd /tmp/foo//bar    # the same 
 
 
 
############################################## 
####    real  VS  effective  uid/gid      #### 
############################################## 
 
often doesnt matter. but lets say you invoke /usr/bin/passwd which lets you change /etc/passwd & /etc/group. 
( to kernel, user name or group name are mapped to some numbers uid and gid. ) 
The effective uid then becomes that of root, i.e. 0 
but you as a user must have some non-zero uid number. that is real VS effective uid. 
 
 
(ref) http://www.lst.de/~okir/blackhats/node23.html 

  1. 2013-11-25 00:49:49 |
  2. Category : unix
  3. Page View:

Google Ads