PL00, System programming
Back to the previous page |page management
List of posts to read before reading this article
Contents
- Introduction
- Shell
- Key factors on system programming
- Management of process
- IPC, Inter-Process Communication
- Signal action
- Shell script
- Thread
- System programming
- Linux Kernel
- Microcomputer structure
- Kernel programming langugage and environment
- 80X86 protection mode and its programming
- Linux kernel architecture
- Booting system
- Initialization programming
- Kernel code
- Block device driver
- Character device driver
- Math coprocessor
- File system
- Memory management
- Header files
- Library files
- Building kernel
- Experimental environment setting and usage
Introduction
Resource
Type | Command | Resource Management | Directory |
---|---|---|---|
existence | htop, ps | cpu, ram | /proc/ |
existence | df | storage, devices | /dev/, /mnt/, /media/ |
existence | netstat | network card | |
interaction | ~ | /bin/, /sbin/, /sys/ | |
function | ~ | /usr/ |
Environmental variables
/home/user/.bashrc
$ echo $PATH
$ sudo echo $PATH
$ whereis [command]
$ which [command]
$ sudo which [command]
export PATH=/usr/local/cuda-10.0/bin${PATH:+:${PATH}}
Execute
$ [command] # $echo $PATH;whereis [command];which [command] : through environmental variables
$ ./[command] # pwd : on current directory
File on Linux
working directory
$ pwd
$ cd ~ # move user directory
$ cd / # move root directory
$ cd - # move previous working directory
$ ls
$ ls -l
$ ls -a
$ ls -al
$ ls | grep [pattern]
$ ls -i
ls wildcard example
$ ls host*
host.conf hostname hosts hosts.allow hosts.deny
$ ls host?
hosts
$ ls host[stu]
hosts
$ rm * # remove all files
$ rm [file_name]
$ rm -rf [folder_name]
Search contents
$ cat [file_name]
$ cat [file_name] | sort {-r}
$ cat [file_name] | sort {-r} | grep [pattern]
$ head [file_name]
$ head -n[number_of_lines] [file_name]
$ tail [file_name]
$ tail -n[number_of_lines] [file_name]
$ more [file_name] # spacebar : next page, enter : next line
$ less [file_name]
$ grep [pattern] [file_name] # search conformable pattern(string) in file
$ grep [pattern] * # search conformable pattern(string) only in present directory
$ grep -i [pattern] [file_name] # search conformable pattern(string) in file, regardless of capital and small letter
$ grep -v [pattern] [file_name] # search unconformable pattern(string) in file
$ grep -n [pattern] [file_name] # search conformable pattern(string) in file, numbering line on results for searching
$ grep -l [pattern] [file_name] # search conformable pattern(string) in file, displaying only file name including pattern
$ grep -c [pattern] [file_name] # search conformable pattern(string) in file, displaying the number of consistant pattern line on a file
$ grep -c [pattern] * # search conformable pattern(string) in file, displaying the number of consistant pattern line on present directory
$ grep -c [pattern] [folder_name] # search conformable pattern(string) in file, displaying the number of consistant pattern line per file on a folder
$ grep -r [pattern] * # search conformable pattern(string) in present directory including sub-directory
$ grep -E "[pattern_1]|[pattern_2]|[pattern_3]" [file_name] # search pattern 1 or pattern 2 or pattern 3 in file
Search file
$ find / -name [file_name] # from root directory
$ find / -size [file_size] # from root directory
$ find / -name [file_name] -size [file_size] # from root directory
$ find . -name [file_name] # from current directory
$ find . -size [file_size] # from current directory
$ find . -name [file_name] -size [file_size] # from current directory
$ cmp [file_name] [file_name]
$ diff [file_name] [file_name]
$ file [file_name]
Zip, Unzip
$ tar -cf [file_name.tar] a b c # tie and zip a b c files
$ tar -zcf [file_name.tar.gz] a b c # zip a b c files
$ tar -xvf [file_name.tar] # unzip .tar
$ tar -zxvf [file_name.tar.gz] # unzip .tar.gz
$ gzip [file_name] # file_name to file_name.gz
$ gzip -d [file_name].gz # file_name.gz to file_name
$ zip -r [folder_name].zip folder_a # zip folder_a folder_name.zip
$ unzip [file_name].zip
$ unzip [file_name].zip [path]
Split zip file
$ cat *.tar.gz.* | tar xvfz - # unzip .tar.gz
$ cat *.tar.* | tar xvfz - # unzip .tar
SUPPLEMENT
- f : set name of file that will be tied
- c : tie files with .tar
- x : unzip .tar(.gz) file
- v : print list detaily
- z : zip to gzip
- t : print list
- p : save authroity of files
- C : set path
Process on Linux
Authority on Linux
- OS manages permissions of users and resources
- Linux manages permissions with users and groups
- root is super user(manager)
- On each files, for owner, owner group, and others(all users)
- management of ‘read’, ‘write’, ‘execute’
- save access authority information in data structure of inode
Shell
Multiuser system
Systemd
Global
/lib/systemd : General
/etc/systemd : Customizing
Local
~/.config/systemd/user
$ systemctl –-user
User
all user name currently accessed on linux
$ who
$ users
$ last
my user name currently accessed on linux
$ whoami
detail
$ id
change password
$ passwd [user_name] # change or create
$ passwd -d [user_name] # delete
$ passwd -u [user_name] # unlock
$ passwd -l [user_name] # lock
when error
$ mount -rw -o remount /
# or
$ mount -o remount,rw /
$ ls -l /etc/shadow
OUTPUT
-rw-r----- 1 root shadow 1025 Feb 11 22:11 /etc/shadow
$ passwd
create another user id on linux
/etc/passwd : ID
/etc/shadow : Password
$ adduser [user_name]
If you concretely set user id
$ useradd [user_name]
$ passwd [user_name]
$ mkdir /home/[user_name]
$ chown [user_name]:[user_name] /home/user_name
$ usermod -a -G [group_name] [user_name]
$ userdel -r [user_name]
SUPPLEMENT : delete user
userdel
$ userdel [user_name] # only account
$ userdel -r [user_name] # account, home directory
deluser
$ deluser [user_name] # only account
$ deluser --remove [user_name] # account, home directory
$ deluser --remove-all-files [user_name] # account, home directory, all files
script file
$ touch adduser
$ vim adduser
useraddd $1
tail -n2 /etc/passwd
mkdir /home/$1
chown $1:$1 /home/$1
echo "$1 user added"
$ sudo ./adduser [user_name]
change user
$ su [user_name] # change id including my present .bashrc, .profile files.
$ su - [user_name] # change id excluding my present .bashrc, .profile files.
$ exit
authorize superuser to user
$ usermod -a -G sudo [user_name]
Another way: Edit
:~$ cd /etc
:/etc$ vim sudoers # change id excluding my present .bashrc, .profile files.
root ALL=(ALL:ALL) ALL
root ALL=(ALL:ALL) ALL
[username] ALL=(ALL:ALL) ALL
Group
Group management
/etc/group
$ groupadd [group_name] # create group
$ usermod -a -G [group_name] [user_name] # add user to group
$ groups [user_name] # list of groups including user
file permission | URL
for file
$ chmod u=rwx, g=rw, o=rx [file_name]
$ chmod 765 [file_name]
for folder
$ chmod u=rwx, g=rw, o=rx [folder_name]
$ chmod -R u=rwx, g=rw, o=rx [folder_name]
$ chmod 765 [folder_name]
$ chmod -R 765 [folder_name]
change owner
for file
$ chown [owner:owner_group] [file_name]
$ chown [owner:] [file_name]
$ chown [:owner_group] [file_name]
$ chgrp [owner_group] [file_name]
for folder
$ chown [owner:owner_group] [folder_name]
$ chown -R [owner:owner_group] [folder_name]
$ chown [owner:] [folder_name]
$ chown -R [owner:] [folder_name]
$ chown [:owner_group] [folder_name]
$ chown -R [:owner_group] [folder_name]
$ chgrp [owner_group] [folder_name]
$ chgrp -R [owner_group] [folder_name]
Super user
Password
$ passwd root : create password
$ passwd -d root : delete password
Super user
$ su
$ su -l
$ su --login
$ sudo
Standard Stream
- 0 : stdin : standard input stream from keyboard
- 1 : stdout : standard output stream to screen
- 2 : stderr : standard error stream to screen
$ echo [arbitary_stdout]
arbitary_stdout
Redirection
$ [command] > [file_name] # overwrite with stdout
$ [command] 1> [file_name] # overwrite with stdout
$ [command] 2> [file_name] # overwrite with stderr
$ [command] >> [file_name] # add text of stdout at the end
$ [command] 1>> [file_name] # add text of stdout at the end
$ [command] 2>> [file_name] # add text of stderr at the end
$ [command] 1>&2 # stdout to stderr
$ [command] 2>&1 # stderr to stdout
$ [command] >& [file_name] # overwrite with stdout and stderr
$ [command] < [file_name]
$ [command] < [file_name_1] > [file_name_2]
Pipe
:/etc$ ls | grep issue
issue
issue.net
Process and Binary
background
$ ./[file_name] & # execute on background
$ jobs # list of process
$ bg # re-execute last stoped process on background
$ bg [job_number_of_stoped_process] # re-execute stoped process on background
$ kill -9 %[number_of_process] # terminate process
foreground
- ctrl + c : terminate process
- ctrl + z : puase process
$ jobs
$ fg # execute process denoted '+' on foreground
$ fg %[number_of_process] # execute process corresponding number on foreground
management of process
URL
process structure
- stack
- heap
- BSS : uninitialized
- DATA : initialized
- TEXT : code
$ ps # displaying list of process for me
$ ps -a # displaying list of process for all users(Select all processes except both session leaders and processes not associated with a terminal.)
$ ps -u # displaying details about owner of process
$ ps -l # displaying details about process
$ ps -x # displaying deamon process
$ ps aux # frequently used(-a, -u, -x)
# ps aux | more
# ps aux | grep [pattern]
# '$ top' or '$ htop'
$ ps -e # displaying list of all process
$ ps -f # displaying relationship about process
$ kill -9 [pid]
process status
code | Description |
---|---|
D | Uninterruptible sleep (usually IO) |
R | Running or runnable (on run queue) |
S | Interruptible sleep (waiting for an event to complete) |
T | Stopped, either by a job control signal or because it is being traced. |
W | paging (not valid since the 2.6.xx kernel) |
X | dead (should never be seen) |
Z | Defunct (“zombie”) process, terminated but not reaped by its parent. |
$ top
shift + t
shift + p
shift + m
shift + a
f
1
Summary for management of resource
$ top
$ htop
$ df -h
$ du -sh
$ vmstat
$ vmstat 1
deamon process
/etc/init.d/
$ service --status-all
$ service [process] start
$ service [process] status
$ service [process] stop
$ service [process] restart
/etc/rc3.d/
auto-start : linking
File System
linking
$ copy [origin_file_name] [new_file_name] # copy : different inode for origin file,
# be live after deleting origin file
$ ln [origin_file_name] [new_file_name] # hard : equal inode for origin file,
# be live after deleting origin file
$ ln -s [origin_file_name] [new_file_name] # soft : different inode for origin file,
# be die after deleting origin file
Specific files
device
- block device
- character device
Key factors on system programming
System call
C compiler
gcc(GNU cc) installation
$ apt update
$ apt upgrade
$ apt install gcc
$ gcc --version
gcc usage
$ gcc -o [.c_file_name] [.exe_file_name]
API and ABI(Application Binary Interface)
Standard
Linux supports POSIX and ANSI C
- POSIX, Portable Operating System Interface
- ANSI C, American National Standards Institute C Language standard
Management of process
Process ID
/proc/
- program
- process : executing program
- thread
maximun of PID, usually signed 16 bit = \(2^{15}\) = 32768
$ cat /proc/sys/kernel/pid_max
First process, PID1, init process(/sbin/init)
$ ps -fe
parent process(ppid) <-> child process(pid)
origin
#include <sys/types.h>
#include <unistd.h>
pid_t getpid (void);
pid_t getppid (void);
practice
#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>
int main(){
printf("pid=%d\n", getpid());
printf("ppid=%d\n", getppid());
return 0;
}
// $ ps
OUTPUT
Create process(system call) 1 : fork() and exec()
- fork() : copy and paste a process space / parent process is conserved
- exec() : overwrite TEXT, DATA, BSS on process space / parent process dissappear
Create process(system call) 2 : wait() and copy()
Terminate process(system call) : exit()
IPC, Inter-Process Communication
Signal action
signal | action |
---|---|
SIGKILL | |
SIGALARM | |
SIGSTP | |
SIGCONT | |
SIGINT | |
SIGSEGV |
$ kill -l
Transfer signal
#include <sys/types.h>
#include <signal.h>
// pid : 프로세스의 PID
// sig : 시그널 번호
int kill(pid_t pid, int sig);
$ ./loop &
$ ./sigkill 1806 2
$ ps
Signal action
#include <signal.h>
void (*signal(int signum, void (*handler)(int)))(int);
// 예1
// void (*handler)(int): SIG_IGN - 시그널 무시, SIG_DFL - 디폴트 동작
signal(SIGINT, SIG_IGN);
// 예2
// SIGINT 시그널 수신 시, signal_handler 함수를 호출
signal(SIGINT, (void *)signal_handler);
Signal handler
static void signal_handler (int signo) {
printf("Catch SIGINT!, but no stop\n");
}
int main (void) {
if (signal (SIGINT, signal_handler) == SIG_ERR) {
printf("Can't catch SIGINT!\n");
exit (1);
}
for (;;)
pause();
return 0;
}
$ ./sigloop &
$ ./sigkill 1894 2
$ ps
$ kill -9 1894
Process PCB(Process control block)
Context switching
Shell script
.sh file
Basic
#!/bin/bash
$ chmod u+x [file_name]
Example
#!/bin/bash
echo "Hello bash"
Comment
#!/bin/bash
# comment
$Variable
#!/bin/bash
mysql_id='root'
mysql_directory='/etc/mysql'
echo $mysql_id
echo $mysql_directory
Caution : do not put space
List = ${Variable[Index]}
#!/bin/bash
daemons=("a" 2 "2" "c")
echo ${daemons[0]} # a
echo ${daemons[1]} # 2
echo ${daemons[2]} # 2
echo ${daemons[2]} # c
echo ${daemons[*]} # a 2 2 c
echo ${daemons[@]} # a 2 2 c
echo ${#daemons[*]} # 4
echo ${#daemons[@]} # 4
filelist1=( $(ls) ) # ls="file1" "file2" "" "" ...
# (ls)=("file1" "file2" "" "" ...)
# $(ls)=$("file1" "file2" "" "" ...)
echo ${filelist1[*]} # file1 file2 file3
filelist2=$(ls)
echo ${filelist2[*]}
#!/bin/bash
daemons=("httpd" "mysqld" "vsftpd")
echo ${daemons[1]} # $daemons 배열의 두 번째 인덱스에 해당하는 mysqld 출력
echo ${daemons[@]} # $daemons 배열의 모든 데이터 출력
echo ${daemons[*]} # $daemons 배열의 모든 데이터 출력
echo ${#daemons[@]} # $daemons 배열 크기 출력
filelist=( $(ls) ) # 해당 쉘스크립트 실행 디렉토리의 파일 리스트를 배열로 $filelist 변수에 입력
echo ${filelist[*]} # $filelist 모든 데이터 출력
Pre-defined local variables
Local variables | Description |
---|---|
$$ | PID of shell script |
$0 | name of shell script |
$1~$9 | arguments |
$* | list of arguments on all of command |
$# | number of arguments |
$? | value corresponded about latest executed command |
- 0(success), 1~125(error)
- 126(not executable)
- 128~ 255(generated signal)
SUPPLEMENT
$ ls -al -z
- $0 : ls
- $1 : -al
- $2 : -z
- $* : -al -z
- $# : 2
Example
#!/bin/bash
echo $$ $0 $1 $* $#
Operator, `expr`
#!/bin/bash
num=`expr \( 3 \* 5 \) / 4 + 7`
echo $num
Caution 1 : do put space among all of words, numbers, symbols
Caution 2 : Arithmetics operator [ +, -, \*, / ]
Caution 3 : associative symbol \(, \)
Selection statements
#!/bin/bash
# if 1
if [ conditional sentence ]
then
[command]
fi
# if 2
if [ conditional sentence ]; then [command]; fi
# if else
if [ conditional sentence ]
then
[command]
else
[command]
fi
Example
#!/bin/bash
ping -c 1 192.168.0.1 1> /dev/null # option -c 1 : only 1 ping test
if [ $? == 0 ]
then
echo "ping of gateway is sucessive"
else
echo "ping of gateway has been fail"
fi
Conditional sentence
Compare words
- word1 == word2 # word1 and word2 are same
- word1 != word2 # word1 and word2 are not same
Example
Compare numbers
- value1 -eq value2 # value1 = value2(equal)
- value1 -ne value2 # value1 != value2(not equal)
- value1 -lt value2 # value1 < value2(less than)
- value1 -le value2 # value1 <= value2(less or equal)
- value1 -gt value2 # value1 > value2(greater than)
- value1 -ge value2 # value1 >= value2(greater or equal)
Example
#!/bin/bash
if [ $1 -gt $2 ]
then
echo "$1 is greater than $2"
exit
fi
Inspect files
- -z word # if word is null, True
- -n word # if word is not null, True
- -e[file_name] #파일이존재하면참
- -d[file_name] #파일이디렉토리면참
- -h[file_name] #심볼릭링크파일
- -f[file_name] #파일이일반파일이면참
- -r[file_name] #파일이읽기가능이면참
- -s[file_name] #파일크기가0이아니면참
- -u[file_name] #파일이set-user-id가설정되면참
- -w[file_name] #파일이쓰기가능상태이면참
- -x[file_name] #파일이실행가능상태이면참
Example
#!/bin/bash
if [ -e $1 ]
then
echo "$1 is exist"
fi
$ if [ -z $1 ]; then echo "Insert arguments"; fi
Logical operation, C(conditional sentence)
- C1 -a C2 # AND
- C1 -o C2 # OR
- C1 && C2 # 양쪽 다 성립
- C1 || C2 # 한쪽 또는 양쪽다 성립
- !C # 조건이 성립하지 않음
- true # 조건이 언제나 성립
- false # 조건이 언제나 성립하지 않음
Example
Iteration statements
#!/bin/bash
# for 1
for [variable] in value1 value2 ...
do
[command]
done
# for 2
for [variable] in value1 value2 ...; do [command]; done
# while
while [ conditional sentence ]
do
[command]
done
for, Example
#!/bin/bash
for database in $(ls)
do
echo $database
done
#!/bin/bash
for database in $(ls); do
echo $database
done
#!/bin/bash
for database in $(ls); do echo $database; done
while, Example
#!/bin/bash
lists=$(ls)
num=${#lists[@]}
index=0
while [ $num -ge 0 ]
do
echo ${lists[$index]}
index=`expr $index + 1`
num=`expr $num - 1`
done
Examples
backup
#!/bin/bash
if [ -z $1 ]||[ -z $2 ]; then
echo usage: $0 sourcedir targetdir
else
SRCDIR=$1
DSTDIR=$2
BACKUPFILE=backup.$(date +%y%m%d%H%M%S).tar.gz
if [ -d $DSTDIR ]; then
tar -cvzf $DSTDIR/$BACKUPFILE $SRCDIR
else
mkdir $DSTDIR
tar -cvzf $DSTDIR/$BACKUPFILE $SRCDIR
fi
fi
clean log file
/var/log
#!/bin/bash
LOGDIR=/var/log
GZIPDAY=1
DELDAY=2
cd $LOGDIR
echo "cd $LOGDIR"
sudo find . -type f -name '*log.?' -mtime +$GZIPDAY -exec bash -c "gzip {}" \; 2> /dev/null
sudo find . -type f -name '*.gz' -mtime +$DELDAY -exec bash -c "rm -f {}" \; 2> /
Thread
Pthread
System programming
Linux Kernel
Microcomputer structure
I/O Port addressing & access control : /proc/ioports
I/O port address assignment
$ cat /proc/ioports
Kernel programming langugage and environment
Object file format
80X86 protection mode and its programming
Linux kernel architecture
Booting system
Initialization programming
Kernel code
Block device driver
Character device driver
Math coprocessor
File system
Memory management
Header files
Library files
Building kernel
Experimental environment setting and usage
List of posts followed by this article
Reference
- A Heavily Commented Linux Kernel Source Code
- Implementation with C on web
- Implementation with bash on web
- zetawiki
- 생활코딩
- tech on the net