6626070
2997924

PL00, System programming

Back to the previous pagepage management
List of posts to read before reading this article


Contents


Introduction

linux kernel map
Interactive+map+of+Linux+kernel b080063dbe7f9cf817972c558ef96e78 a476e5ac785fa192712b24316bfaf3c3


Resource

Type Command Resource Management Directory
existence htop, ps cpu, ram /proc/
existence df storage, devices /dev/, /mnt/, /media/
existence netstat network card  
interaction ~ /bin/, /sbin/, /sys/  
function ~ /usr/  




Environmental variables

/home/user/.bashrc

$ echo $PATH
$ sudo echo $PATH
$ whereis [command]
$ which [command]
$ sudo which [command]
export PATH=/usr/local/cuda-10.0/bin${PATH:+:${PATH}}




Execute

$ [command]           # $echo $PATH;whereis [command];which [command]  : through environmental variables
$ ./[command]         # pwd : on current directory 





File on Linux

Linux directory structure

working directory

$ pwd
$ cd ~     # move user directory
$ cd /     # move root directory
$ cd -     # move previous working directory
$ ls
$ ls -l
$ ls -a
$ ls -al
$ ls | grep [pattern]
$ ls -i
ls wildcard example

image

$ ls host*
host.conf  hostname  hosts  hosts.allow  hosts.deny


$ ls host?
hosts


$ ls host[stu]
hosts

$ rm *                    # remove all files
$ rm [file_name]
$ rm -rf [folder_name]




Search contents

$ cat [file_name]
$ cat [file_name] | sort {-r}
$ cat [file_name] | sort {-r} | grep [pattern]
$ head [file_name]
$ head -n[number_of_lines] [file_name]
$ tail [file_name]
$ tail -n[number_of_lines] [file_name]
$ more [file_name]                            # spacebar : next page, enter : next line
$ less [file_name]                        
$ grep [pattern] [file_name]         # search conformable pattern(string) in file
$ grep [pattern] *                   # search conformable pattern(string) only in present directory

$ grep -i [pattern] [file_name]      # search conformable pattern(string) in file, regardless of capital and small letter
$ grep -v [pattern] [file_name]      # search unconformable pattern(string) in file
$ grep -n [pattern] [file_name]      # search conformable pattern(string) in file, numbering line on results for searching
$ grep -l [pattern] [file_name]      # search conformable pattern(string) in file, displaying only file name including pattern

$ grep -c [pattern] [file_name]      # search conformable pattern(string) in file, displaying the number of consistant pattern line on a file 
$ grep -c [pattern] *                # search conformable pattern(string) in file, displaying the number of consistant pattern line on present directory 
$ grep -c [pattern] [folder_name]    # search conformable pattern(string) in file, displaying the number of consistant pattern line per file on a folder

$ grep -r [pattern] *                # search conformable pattern(string) in present directory including sub-directory
$ grep -E "[pattern_1]|[pattern_2]|[pattern_3]" [file_name]        # search pattern 1 or pattern 2 or pattern 3 in file
grep example

URL





Search file

$ find / -name [file_name]                        # from root directory
$ find / -size [file_size]                        # from root directory
$ find / -name [file_name] -size [file_size]      # from root directory
$ find . -name [file_name]                        # from current directory
$ find . -size [file_size]                        # from current directory
$ find . -name [file_name] -size [file_size]      # from current directory
$ cmp [file_name] [file_name]
$ diff [file_name] [file_name]
$ file [file_name]




Zip, Unzip

$ tar -cf [file_name.tar] a b c        # tie and zip a b c files 
$ tar -zcf [file_name.tar.gz] a b c    # zip a b c files 
$ tar -xvf [file_name.tar]             # unzip .tar
$ tar -zxvf [file_name.tar.gz]         # unzip .tar.gz
$ gzip [file_name]                     # file_name to file_name.gz
$ gzip -d [file_name].gz               # file_name.gz to file_name
$ zip -r [folder_name].zip folder_a    # zip folder_a folder_name.zip
$ unzip [file_name].zip
$ unzip [file_name].zip [path]

Split zip file

$ cat *.tar.gz.* | tar xvfz -       # unzip .tar.gz
$ cat *.tar.* | tar xvfz -          # unzip .tar
SUPPLEMENT
  • f : set name of file that will be tied
  • c : tie files with .tar
  • x : unzip .tar(.gz) file
  • v : print list detaily
  • z : zip to gzip
  • t : print list
  • p : save authroity of files
  • C : set path




Process on Linux





Authority on Linux

  • OS manages permissions of users and resources
  • Linux manages permissions with users and groups
  • root is super user(manager)
  • On each files, for owner, owner group, and others(all users)
    • management of ‘read’, ‘write’, ‘execute’
    • save access authority information in data structure of inode





Shell

Multiuser system

Systemd

Global
/lib/systemd : General
/etc/systemd : Customizing




Local
~/.config/systemd/user

$ systemctl –-user




User

all user name currently accessed on linux

$ who
$ users
$ last




my user name currently accessed on linux

$ whoami
detail
$ id




change password

$ passwd [user_name]         # change or create
$ passwd -d [user_name]      # delete
$ passwd -u [user_name]      # unlock
$ passwd -l [user_name]      # lock
when error

https://www.codevoila.com/post/26/fix-authentication-token-manipulation-error-when-changing-user-password-in-ubuntu

$ mount -rw -o remount /
# or
$ mount -o remount,rw /
$ ls -l /etc/shadow
OUTPUT
-rw-r----- 1 root shadow 1025 Feb  11 22:11 /etc/shadow


$ passwd




create another user id on linux
/etc/passwd : ID
/etc/shadow : Password

$ adduser [user_name]
If you concretely set user id

image

$ useradd [user_name]
$ passwd [user_name]
$ mkdir /home/[user_name]
$ chown [user_name]:[user_name] /home/user_name
$ usermod -a -G [group_name] [user_name]
$ userdel -r [user_name]
SUPPLEMENT : delete user

userdel

$ userdel [user_name]         # only account
$ userdel -r [user_name]      # account, home directory


deluser

$ deluser [user_name]                    # only account
$ deluser --remove [user_name]           # account, home directory
$ deluser --remove-all-files [user_name] # account, home directory, all files




script file

$ touch adduser
$ vim adduser
useraddd $1
tail -n2 /etc/passwd
mkdir /home/$1
chown $1:$1 /home/$1
echo "$1 user added"
$ sudo ./adduser [user_name]




change user

$ su [user_name]        # change id including my present .bashrc, .profile files.
$ su - [user_name]       # change id excluding my present .bashrc, .profile files.
$ exit




authorize superuser to user

$ usermod -a -G sudo [user_name]
Another way: Edit
:~$ cd /etc
:/etc$ vim sudoers       # change id excluding my present .bashrc, .profile files.
root    ALL=(ALL:ALL) ALL

image

root    ALL=(ALL:ALL) ALL
[username]    ALL=(ALL:ALL) ALL

image





Group

Group management
/etc/group

$ groupadd [group_name]                     # create group
$ usermod -a -G [group_name] [user_name]    # add user to group
$ groups [user_name]                        # list of groups including user




file permissionURL
for file

$ chmod u=rwx, g=rw, o=rx [file_name]
$ chmod 765 [file_name]

for folder

$ chmod u=rwx, g=rw, o=rx [folder_name]
$ chmod -R u=rwx, g=rw, o=rx [folder_name]

$ chmod 765 [folder_name]
$ chmod -R 765 [folder_name]




change owner
for file

$ chown [owner:owner_group] [file_name]

$ chown [owner:] [file_name]

$ chown [:owner_group] [file_name]
$ chgrp [owner_group] [file_name]


for folder

$ chown [owner:owner_group] [folder_name]
$ chown -R [owner:owner_group] [folder_name]

$ chown [owner:] [folder_name]
$ chown -R [owner:] [folder_name]

$ chown [:owner_group] [folder_name]
$ chown -R [:owner_group] [folder_name]
$ chgrp [owner_group] [folder_name]
$ chgrp -R [owner_group] [folder_name]




Super user

Password

$ passwd root       : create password
$ passwd -d root    : delete password




Super user

$ su
$ su -l
$ su --login
$ sudo








Standard Stream

URL

  • 0 : stdin : standard input stream from keyboard
  • 1 : stdout : standard output stream to screen
  • 2 : stderr : standard error stream to screen
$ echo [arbitary_stdout]
arbitary_stdout




Redirection

redirection(>,<) change stream's direction for a process
$ [command] > [file_name]        # overwrite with stdout
$ [command] 1> [file_name]       # overwrite with stdout
$ [command] 2> [file_name]       # overwrite with stderr

$ [command] >> [file_name]       # add text of stdout at the end
$ [command] 1>> [file_name]      # add text of stdout at the end
$ [command] 2>> [file_name]      # add text of stderr at the end

$ [command] 1>&2                 # stdout to stderr
$ [command] 2>&1                 # stderr to stdout
$ [command] >& [file_name]       # overwrite with stdout and stderr

$ [command] < [file_name] 
$ [command] < [file_name_1] > [file_name_2]




Pipe

pipe(|) connect a process to another process
:/etc$ ls | grep issue
issue
issue.net





Process and Binary

background

$ ./[file_name] &                       # execute on background
$ jobs                                  # list of process
$ bg                                    # re-execute last stoped process on background
$ bg [job_number_of_stoped_process]     # re-execute stoped process on background
$ kill -9 %[number_of_process]          # terminate process  




foreground

  • ctrl + c : terminate process
  • ctrl + z : puase process
$ jobs
$ fg                                 # execute process denoted '+' on foreground
$ fg %[number_of_process]            # execute process corresponding number on foreground




management of process

URL
process structure
1_CtL4OiSQFvSrK8mIOkb5zQ (1) kernelUserMemorySplit

  • stack
  • heap
  • BSS : uninitialized
  • DATA : initialized
  • TEXT : code
$ ps             # displaying list of process for me
$ ps -a          # displaying list of process for all users(Select all processes except both session leaders and processes not associated with a terminal.)
$ ps -u          # displaying details about owner of process
$ ps -l          # displaying details about process
$ ps -x          # displaying deamon process
$ ps aux         # frequently used(-a, -u, -x)
                 # ps aux | more
                 # ps aux | grep [pattern]
                 # '$ top' or '$ htop' 
$ ps -e          # displaying list of all process
$ ps -f          # displaying relationship about process
$ kill -9 [pid]
process status
code Description
D Uninterruptible sleep (usually IO)
R Running or runnable (on run queue)
S Interruptible sleep (waiting for an event to complete)
T Stopped, either by a job control signal or because it is being traced.
W paging (not valid since the 2.6.xx kernel)
X dead (should never be seen)
Z Defunct (“zombie”) process, terminated but not reaped by its parent.




$ top
shift + t
shift + p
shift + m
shift + a
f
1

URL




Summary for management of resource

$ top
$ htop
$ df -h
$ du -sh
$ vmstat
$ vmstat 1




deamon process

/etc/init.d/

$ service --status-all
$ service [process] start
$ service [process] status
$ service [process] stop
$ service [process] restart


/etc/rc3.d/
auto-start : linking



File System

linking

$ copy [origin_file_name] [new_file_name]       # copy : different inode for origin file,
                                                #        be live after deleting origin file
$ ln [origin_file_name] [new_file_name]         # hard : equal inode for origin file,
                                                #        be live after deleting origin file
$ ln -s [origin_file_name] [new_file_name]      # soft : different inode for origin file,
                                                #        be die after deleting origin file




Specific files

device

  • block device
  • character device





Key factors on system programming

System call

C compiler

gcc(GNU cc) installation

$ apt update
$ apt upgrade
$ apt install gcc
$ gcc --version
gcc usage
$ gcc -o [.c_file_name] [.exe_file_name]





API and ABI(Application Binary Interface)





Standard

Linux supports POSIX and ANSI C

  • POSIX, Portable Operating System Interface
  • ANSI C, American National Standards Institute C Language standard





Management of process

image image


Process ID

/proc/

  • program
  • process : executing program
  • thread


maximun of PID, usually signed 16 bit = \(2^{15}\) = 32768

$ cat /proc/sys/kernel/pid_max


First process, PID1, init process(/sbin/init)

$ ps -fe


parent process(ppid) <-> child process(pid)

origin

#include <sys/types.h>
#include <unistd.h>
pid_t getpid (void);
pid_t getppid (void);


practice

#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>

int main(){
        printf("pid=%d\n", getpid());
        printf("ppid=%d\n", getppid());
        return 0;
}

// $ ps
OUTPUT

image



Create process(system call) 1 : fork() and exec()

  • fork() : copy and paste a process space / parent process is conserved
  • exec() : overwrite TEXT, DATA, BSS on process space / parent process dissappear





Create process(system call) 2 : wait() and copy()


Terminate process(system call) : exit()





IPC, Inter-Process Communication





Signal action

signal action
SIGKILL  
SIGALARM  
SIGSTP  
SIGCONT  
SIGINT  
SIGSEGV  
$ kill -l




Transfer signal

#include <sys/types.h>
#include <signal.h>
// pid : 프로세스의 PID
// sig : 시그널 번호
int kill(pid_t pid, int sig);
$ ./loop &
$ ./sigkill 1806 2
$ ps




Signal action

#include <signal.h>

void (*signal(int signum, void (*handler)(int)))(int);

// 예1
// void (*handler)(int): SIG_IGN - 시그널 무시, SIG_DFL - 디폴트 동작
signal(SIGINT, SIG_IGN);

// 예2
// SIGINT 시그널 수신 시, signal_handler 함수를 호출
signal(SIGINT, (void *)signal_handler);




Signal handler

static void signal_handler (int signo) {
    printf("Catch SIGINT!, but no stop\n");
}

int main (void) {
    if (signal (SIGINT, signal_handler) == SIG_ERR) {
        printf("Can't catch SIGINT!\n");
        exit (1);
    }
    for (;;)
    pause();
    return 0;
}
$ ./sigloop &
$ ./sigkill 1894 2
$ ps
$ kill -9 1894




Process PCB(Process control block)

image

Context switching

URL1, URL2





Shell script

Bash Scripting Tutorial

.sh file

Basic

#!/bin/bash
$ chmod u+x [file_name]
Example
#!/bin/bash

echo "Hello bash"




Comment

#!/bin/bash

# comment




$Variable

#!/bin/bash

mysql_id='root'
mysql_directory='/etc/mysql'

echo $mysql_id
echo $mysql_directory

Caution : do not put space




List = ${Variable[Index]}

#!/bin/bash

daemons=("a" 2 "2" "c")
echo ${daemons[0]}             # a
echo ${daemons[1]}             # 2
echo ${daemons[2]}             # 2
echo ${daemons[2]}             # c
echo ${daemons[*]}             # a 2 2 c
echo ${daemons[@]}             # a 2 2 c
echo ${#daemons[*]}            # 4
echo ${#daemons[@]}            # 4

filelist1=( $(ls) )            # ls="file1" "file2" "" "" ...
                               # (ls)=("file1" "file2" "" "" ...)
                               # $(ls)=$("file1" "file2" "" "" ...)
                               
echo ${filelist1[*]}           # file1 file2 file3

filelist2=$(ls)             
echo ${filelist2[*]}            
#!/bin/bash

daemons=("httpd" "mysqld" "vsftpd")
echo ${daemons[1]}             # $daemons 배열의 두 번째 인덱스에 해당하는 mysqld 출력
echo ${daemons[@]}             # $daemons 배열의 모든 데이터 출력
echo ${daemons[*]}             # $daemons 배열의 모든 데이터 출력
echo ${#daemons[@]}            # $daemons 배열 크기 출력

filelist=( $(ls) )             # 해당 쉘스크립트 실행 디렉토리의 파일 리스트를 배열로 $filelist 변수에 입력
echo ${filelist[*]}            # $filelist 모든 데이터 출력




Pre-defined local variables

Local variables Description
$$ PID of shell script
$0 name of shell script
$1~$9 arguments
$* list of arguments on all of command
$# number of arguments
$? value corresponded about latest executed command
  • 0(success), 1~125(error)
  • 126(not executable)
  • 128~ 255(generated signal)
SUPPLEMENT
$ ls -al -z
  • $0 : ls
  • $1 : -al
  • $2 : -z
  • $* : -al -z
  • $# : 2

Example
#!/bin/bash

echo $$ $0 $1 $* $#

image





Operator, `expr`

#!/bin/bash

num=`expr \( 3 \* 5 \) / 4 + 7`
echo $num

Caution 1 : do put space among all of words, numbers, symbols
Caution 2 : Arithmetics operator [ +, -, \*, / ]
Caution 3 : associative symbol \(, \)


Selection statements

#!/bin/bash


# if 1
if [ conditional sentence ]
then
    [command]
fi

# if 2
if [ conditional sentence ]; then [command]; fi


# if else
if [ conditional sentence ]
then
    [command]
else
    [command]
fi
Example
#!/bin/bash

ping -c 1 192.168.0.1 1> /dev/null           # option -c 1 : only 1 ping test
if [ $? == 0 ]
then
        echo "ping of gateway is sucessive"
else
        echo "ping of gateway has been fail"
fi

image


Conditional sentence

Compare words

  • word1 == word2 # word1 and word2 are same
  • word1 != word2 # word1 and word2 are not same
Example




Compare numbers

  • value1 -eq value2 # value1 = value2(equal)
  • value1 -ne value2 # value1 != value2(not equal)
  • value1 -lt value2 # value1 < value2(less than)
  • value1 -le value2 # value1 <= value2(less or equal)
  • value1 -gt value2 # value1 > value2(greater than)
  • value1 -ge value2 # value1 >= value2(greater or equal)
Example
#!/bin/bash

if [ $1 -gt $2 ]
then
        echo "$1 is greater than $2"
        exit
fi

image





Inspect files

  • -z word # if word is null, True
  • -n word # if word is not null, True
  • -e[file_name] #파일이존재하면참
  • -d[file_name] #파일이디렉토리면참
  • -h[file_name] #심볼릭링크파일
  • -f[file_name] #파일이일반파일이면참
  • -r[file_name] #파일이읽기가능이면참
  • -s[file_name] #파일크기가0이아니면참
  • -u[file_name] #파일이set-user-id가설정되면참
  • -w[file_name] #파일이쓰기가능상태이면참
  • -x[file_name] #파일이실행가능상태이면참
Example
#!/bin/bash

if [ -e $1 ]
then
        echo "$1 is exist"
fi

image


$ if [ -z $1 ]; then echo "Insert arguments"; fi




Logical operation, C(conditional sentence)

  • C1 -a C2 # AND
  • C1 -o C2 # OR
  • C1 && C2 # 양쪽 다 성립
  • C1 || C2 # 한쪽 또는 양쪽다 성립
  • !C # 조건이 성립하지 않음
  • true # 조건이 언제나 성립
  • false # 조건이 언제나 성립하지 않음
Example





Iteration statements

#!/bin/bash


# for 1
for [variable] in value1 value2 ...
do
    [command]
done

# for 2
for [variable] in value1 value2 ...; do [command]; done


# while
while [ conditional sentence ]
do
    [command]
done
for, Example
#!/bin/bash

for database in $(ls)
do
    echo $database
done
#!/bin/bash

for database in $(ls); do
    echo $database
done
#!/bin/bash

for database in $(ls); do echo $database; done

while, Example
#!/bin/bash

lists=$(ls)
num=${#lists[@]}
index=0
while [ $num -ge 0 ]
do
    echo ${lists[$index]}
    index=`expr $index + 1`
    num=`expr $num - 1`
done





Examples

backup

#!/bin/bash

if [ -z $1 ]||[ -z $2 ]; then
    echo usage: $0 sourcedir targetdir
else
    SRCDIR=$1
    DSTDIR=$2
    BACKUPFILE=backup.$(date +%y%m%d%H%M%S).tar.gz
    if [ -d $DSTDIR ]; then
        tar -cvzf $DSTDIR/$BACKUPFILE $SRCDIR
    else
        mkdir $DSTDIR
        tar -cvzf $DSTDIR/$BACKUPFILE $SRCDIR
    fi
fi




clean log file

/var/log

#!/bin/bash

LOGDIR=/var/log
GZIPDAY=1
DELDAY=2
cd $LOGDIR
echo "cd $LOGDIR"

sudo find . -type f -name '*log.?' -mtime +$GZIPDAY -exec bash -c "gzip {}" \; 2> /dev/null
sudo find . -type f -name '*.gz' -mtime +$DELDAY -exec bash -c "rm -f {}" \; 2> /





Thread

Pthread





System programming

Linux Kernel

Microcomputer structure

I/O Port addressing & access control : /proc/ioports

I/O port address assignment

$ cat /proc/ioports

image





Kernel programming langugage and environment

Object file format

image image image





80X86 protection mode and its programming





Linux kernel architecture





Booting system





Initialization programming





Kernel code





Block device driver





Character device driver





Math coprocessor





File system





Memory management





Header files





Library files





Building kernel





Experimental environment setting and usage






List of posts followed by this article


Reference


OUTPUT