2021-12-09 21:18:43 +01:00
#+TITLE: bash commands
#+DATE: 2021-08-30
#+STARTUP: indent
#+OPTIONS: num:nil
* DONE work in bash tutorial
:LOGBOOK:
CLOCK: [ 2021-09-08 mié 18:52] --[ 2021-09-08 mié 19:22] = > 0:30
CLOCK: [ 2021-09-04 sáb 16:04] --[ 2021-09-04 sáb 16:34] = > 0:30
CLOCK: [ 2021-09-04 sáb 15:19] --[ 2021-09-04 sáb 15:49] = > 0:30
:END:
* Variables
System variables are stored in users environment
- ~$PATH ~ Finds the content of a variable ( "PATH" )
- "env" variables defined in shell environment
* Add directory to path
[ [ https://wiki.archlinux.org/title/Environment_variables] [ "Environment variables" in: Arch wiki] ]
To add a directory to the PATH for local usage, put
the following in "~/.bash_profile" :
#+begin_example bash
export PATH = " ${ PATH } :/home/my_user/bin "
#+end_example
This will add the directory at the end of $PATH . To update the variable, re-login or source the file:
#+begin_example bash
source ~/.bash_profile
#+end_example
Then check directories in " $PATH "
#+begin_example bash
echo $PATH
#+end_example
* Adding a new user
Check users that are logged
#+begin_src bash
users
#+end_src
#+RESULTS:
: myuser
New user called "test"
#+begin_example bash
sudo useradd -p -m test
#+end_example
The configuration file "useradd" sets options for this command:
#+begin_src bash
sudo less /etc/default/useradd
#+end_src
#+RESULTS:
| # useradd defaults file for ArchLinux |
| # original changes by TomK |
| GROUP = users |
| HOME = /home |
| INACTIVE = -1 |
| EXPIRE = |
| SHELL = /bin/bash |
| SKEL = /etc/skel |
| CREATE_MAIL_SPOOL = no |
| ( END) |
PS: should use "sudo" to see the file. And ~less( ) ~ or ~more( ) ~ will show the file
The following file contains a list of fields about each user:
#+begin_src bash
head -4 /etc/passwd
#+end_src
#+RESULTS:
| root:x:0:0::/root:/bin/bash |
| bin:x:1:1::/:/sbin/nologin |
| daemon:x:2:2::/:/sbin/nologin |
| mail:x:8:12::/var/spool/mail:/sbin/nologin |
PS: ~head( ) ~ is used to show only the first four lines.
* Describe hardware
Here using piping to show only the first four lines in the output
#+begin_src bash
lshw | head -4
#+end_src
#+RESULTS:
| myuser | | |
| description: | Computer | |
| width: | 64 | bits |
| capabilities: | smp | vsyscall32 |
* Execute stuffs
** Streaming from terminal
To watch a stream use video player such as 'mpv' or 'vlc' that supports streaming
And pass the streaming address as argument to the name of the media player
#+begin_example bash
mpv https://live0.emacsconf.org/main.webm
#+end_example
** Run a bash script
*** Write script
After writing, save the file with '.sh' extension. This script will
echo the words within quotes
#+begin_src bash
#!/bin/bash
echo "Hellow hu-hu"
#+end_src
#+RESULTS:
: Hellow hu-hu
*** Option a: make it executable
Use the command "chmod" to change permissions for the file
#+begin_example bash
chmod +x script.sh
#+end_example
Then, execute
#+begin_example bash
./script.sh
#+end_example
*** Option b: run from terminal
#+begin_example bash
bash script.sh
#+end_example
Or
#+begin_example bash
sh script.sh
#+end_example
PS: to execute the script from any place in the machine include
the script in a directory that can be read from ~$PATH ~
*** Option c: run from graphical interface
2022-01-26 20:40:43 +01:00
** Run R scripts
Write the script and save it. Then go to the directory where the script
was saved and run this code from terminal
#+begin_src bash
Rscript script.R
#+end_src
2021-12-09 21:18:43 +01:00
* Manuals
2022-01-26 20:40:43 +01:00
To explore the manuals use the command ~man~ or ~info~ followed by the name of the function or program
2021-12-09 21:18:43 +01:00
#+begin_src bash
man
#+end_src
#+RESULTS:
| What manual page do you want? |
| For example, try 'man man' . |
2022-01-26 20:40:43 +01:00
Search for information about the manual interface within ~man~
2021-12-09 21:18:43 +01:00
: man man
2022-01-26 20:40:43 +01:00
Description of the system hierarchy
2021-12-09 21:18:43 +01:00
: man hier
2022-01-26 20:40:43 +01:00
Showing only the first four lines of "info" interface
2021-12-09 21:18:43 +01:00
#+begin_src bash
info | head -4
#+end_src
#+RESULTS:
| File: | dir, | Node: | Top, | This | is | the | top | of | the | INFO | tree. |
| | | | | | | | | | | | |
| This | is | the | Info | main | menu | ( aka | directory | node) . | | | |
| A | few | useful | Info | commands: | | | | | | | |
2022-01-26 20:40:43 +01:00
Search for ~info~ page about Bash shell features
2021-12-09 21:18:43 +01:00
: info bash
2022-01-26 20:40:43 +01:00
The manuals can also be accessed from emacs using "M-x" ( using the "meta" key or CTRL + x)
: M-x info
: M-x man
: M-x woman
2021-12-09 21:18:43 +01:00
* Show command history
: history
The file can be found in the user home:
: /home/user/.bash_history
* Translators and dictionaries
** Apertium translator
Installing in Parabola GNU-linux
: sudo pacman -S apertium-eng-spa-git
: sudo pacman -S apertium-lex-tools
: sudo pacman -S apertium-viewer-2.5.3-1
Apertium can be used in the command line
: echo Hola | apertium spa-eng
: echo Hello | apertium eng-spa
2022-01-26 20:40:43 +01:00
The pipe operator "~|~" is used to pass the input to apertium
2021-12-09 21:18:43 +01:00
* Files and directories
** Create empty file
Create "file.txt" with command ~touch~
: touch file.txt
Create a file redirecting the shell output using "~>~" operator
: echo I will put this text wthin a file > file2.txt
2022-01-26 20:40:43 +01:00
** Redirect input, output and standard error
Everything in GNU systems is a file. The keyboard and the terminal are also a file.
The keyboard is used for input and the terminal is used by commands as output.
The files can be found in directory "fd" , which stands for file descriptor)
#+begin_src bash
cd /dev/fd/
ls -l
#+end_src
#+RESULTS:
| total | 0 | | | | | | | | | |
| lr-x------ | 1 | user | user | 64 | dec | 15 | 10:31 | 0 | -> | /tmp/babel-yl1RTd/ob-input-izpXuk |
| l-wx------ | 1 | user | user | 64 | dec | 15 | 10:31 | 1 | -> | pipe:[ 42063] |
| lr-x------ | 1 | user | user | 64 | dec | 15 | 10:31 | 17 | -> | /dev/pts/0 |
| l-wx------ | 1 | user | user | 64 | dec | 15 | 10:31 | 2 | -> | /tmp/emacsA1M18o |
* standard input stream is bound to /dev/fd/0
* standard output ( stout) stream is bound to /dev/fd/1
* standard errror ( stderr) stream is bound to /dev/fd/2
During interaction a terminal gets written to stdin ( dev/fd/0) , which
a command read. The command then do an action and writes the output
( stout or stderr) where it will read by the terminal and is displayed
to the user.
To redirect use the operator ` >` as follows:
* ` command > file_to_redirect`
The operator ` >` redirects stout, i.e., is the same that 1>. This code
creates a new file with the list of files present in the actual directory:
#+begin_src bash
ls -l > file_redirect.txt
cat file_redirect.txt
#+end_src
#+RESULTS:
| total | 32 | | | | | | | |
| -rw-r--r-- | 1 | user | user | 15782 | dec | 15 | 10:49 | bash_tutorial.org |
| -rw-r--r-- | 1 | user | user | 0 | dec | 15 | 10:49 | file_redirect.txt |
| -rw-r--r-- | 1 | user | user | 11895 | dec | 9 | 17:44 | markdown.md |
| -rw-r--r-- | 1 | user | user | 121 | dec | 9 | 17:17 | README.org |
Use "2>&1" to redirect stderr with the stout. Using an arbitrary word and redirect it
to the file should print a warning message that is redirected to the file:
#+begin_src bash
ls -l > file_redirect.txt
paco >> file_redirect.txt 2>& 1
cat file_redirect.txt
#+end_src
#+RESULTS:
| total | 32 | | | | | | | |
| -rw-r--r-- | 1 | user | user | 15995 | dec | 15 | 12:39 | bash_tutorial.org |
| -rw-r--r-- | 1 | user | user | 0 | dec | 15 | 13:41 | file_redirect.txt |
| -rw-r--r-- | 1 | user | user | 11895 | dec | 9 | 17:44 | markdown.md |
| -rw-r--r-- | 1 | user | user | 121 | dec | 9 | 17:17 | README.org |
| bash: | line | 2: | paco: | command | not | found | | |
Use the "<" operator to pass commands to the file:
#+begin_src bash
ls -l > file_redirect.txt
paco >> file_redirect.txt 2>& 1
sort < file_redirect.txt
cat file_redirect.txt
#+end_src
#+RESULTS:
| bash: | line | 2: | paco: | command | not | found | | |
| -rw-r--r-- | 1 | user | user | 0 | dec | 15 | 13:44 | file_redirect.txt |
| -rw-r--r-- | 1 | user | user | 121 | dec | 9 | 17:17 | README.org |
| -rw-r--r-- | 1 | user | user | 16501 | dec | 15 | 13:41 | bash_tutorial.org |
| total | 24 | | | | | | | |
| total | 24 | | | | | | | |
| -rw-r--r-- | 1 | user | user | 16501 | dec | 15 | 13:41 | bash_tutorial.org |
| -rw-r--r-- | 1 | user | user | 0 | dec | 15 | 13:44 | file_redirect.txt |
| -rw-r--r-- | 1 | user | user | 121 | dec | 9 | 17:17 | README.org |
| bash: | line | 2: | paco: | command | not | found | | |
2021-12-09 21:18:43 +01:00
** List files
Use ~ls~ in the directory you want to explore
: ls
List with properties using options ~-l~ and ~-la~
: ls -l
: ls -la
List using wildcards
List all files ending in ".csv"
: ls *.csv
List all files containing the characters "moda" within name, e.g.,
"acomoda" , "comoda.txt" ...
: ls *moda*
Remove "x"
: rm x
Copy a file using "~cp~ source destination"
: cp /home/text.csv /home/myuser/text.csv
Copy multiple files, directories or disk using "rsync" ( source, destination) .
The command accepts arguments:
- "-r" recursive
- "-v" verbose
- "-a" archive ( keeps info about files)
: rsync -rav /home/user/Documents /run/media/user/disk/backup/
~rsync~ can be also used with selective copyng based on file type
: rsync /home/user/Desktop/*.jpg /home/user/Desktop/backupdata/
Create directory
: mkdir new_dir
2022-01-26 20:40:43 +01:00
Remove ( unlink) a directory
Some arguments:
* -r: recursively unlink
* -v: verbose
* -f: force
2021-12-09 21:18:43 +01:00
: rm -r new_dir
2022-01-26 20:40:43 +01:00
Remove a file
2021-12-09 21:18:43 +01:00
: rm file.txt
Space used by a directory
: du -hs /usr
Display directory
: pwd
Change directory you are working from terminal.
Go to the home of the user
: cd
Go one level up in the directory tree
: cd ..
Go to "Documents" directory
: cd /home/myuser/Documents
** Use rmlint to search for duplicated or empty files
Install
: sudo pacman -S rmlint
Run in the directory you want to check for duplicated or empty files
: rmlint -g
This will create two files:
* rmlint.sh
* rmlint.json
The shell script can be used to delete duplicated files
OPTIONS:
- h Show this message.
- d Do not ask before running.
- x Keep rmlint.sh; do not autodelete it.
- p Recheck that files are still identical before removing duplicates.
- r Allow deduplication of files on read-only btrfs snapshots. ( requires sudo)
- n Do not perform any modifications, just print what would be done . ( implies -d and -x)
- c Clean up empty directories while deleting duplicates.
- q Do not show progress.
- k Keep the timestamp of directories when removing duplicates.
- i Ask before deleting each file
: chmod +x rmlint.sh
: bash -xcp rmlint.sh
** Convert pdf to text or html using poppler
Use poppler and redirect the text to a org file.
~pdftotext~ converts a pdf to "txt" if no other format is
specified
: pdftotext foo.pdf foo
: pacman -S poppler
: pdftotext foo.pdf ->> foo.org # first option
: pdftotext foo.pdf foo.org # second option
Convert pdf to html
: pfdftohtml foo.pdf foo.html
** Convert files using pandoc
Use pandoc with "-s" flag to produce a standalone document and "-o" to
redirect output to a file.
In the example a file "foo" is converted from ~odt~ to ~org~
: pandoc -s -o foo.org foo.odt
** Convert files to pdf using libreoffice
Convert odt file "tcl_online3.odt" to pdf
2022-01-26 20:40:43 +01:00
: libreoffice --headless --convert-to pdf tcl_online3.odt
2021-12-09 21:18:43 +01:00
** Using 'sed' to substitute text
"~sed~" is a stream editor for filtering and transforming text" ...
This command in emacs editor
: M-x woman [ RET] sed
Visits the following file with info about ~sed~
usr/share/man/man1/sed.1.gz
A brief example:
I have a long comma separated text file with strings and
need to include a line break after each comma ( \n )
: cat filetest
#+RESULTS:
| 'string1' ,| 'string2' , | 'string3' |
Using ~sed~ and redirecting the output to "filetest1"
: $ sed 's/\,/&\n/g' filetest > filetest1
#+RESULTS:
| 'string1' , |
| 'string2' , |
| 'string3' |
** Send files to another machine with "croc"
: sudo pacman -S croc
Then run:
: croc send clown.txt
This will give a code
: > Code is 34-04-paprika-sugar-micro
: > On the other computer run
: > croc 34-04-paprika-sugar-micro
More info about croc in [ [ https://github.com/schollz/croc] [ Github repo] ]
** Move or rename
Use "~mv~ source destination" command
: mv /home/myuser/text.txt /home/myuser/Documents/text.csv
Example: moving the ".config" files to ".config3" , then moving
".config1" to ".config" to arrange configuration archives.
This creates a copy of each file in the same directory
: pwd # check where are you now
: # moving files
: mv file1.txt file3.txt
*Moving multiple files* can be possible. Selecting all the files ending
in ".mp3" within Downloads
: mv ~/Downloads/*.mp3 ~/Music/
*Moving multiple* files included within a text file. Files are included in "filesmv2.txt" and are moved to
"tmpfiles" directory. "t" argument stands for "target directory"
Each line in the "txt" file looks like this:
'/home/user/file1.pdf' # original
: $ cat filesmv2.txt | xargs mv -t ~/tmpfiles
Source: [ [ https://unix.stackexchange.com/questions/115734/move-file-by-list-in-file-with-leading-whitespace] [ stackexchange] ]
*Moving multiple files* using wildcards. This moves all files containing the sequence "nasa" in the middle of
the name. '*' indicates that there are characters or numbers before and after the word "nasa"
: mv *roco* -t ./direct
** Flashing USB with "dd"
"dd" ( convert and copy a file) is useful to copy information from a disk. It can be used
to create bootable USB. An example with parabola:
: sudo dd if = /home/user/Documents/parabola-2016.11.03-dual.iso of = /dev/sdb1
The ISO image is copied to "sdb1" ( USB volume)
More files with examples of use of "dd" [ ../bash/dd_command.org] [ here] ] and [ [ ~/Documentos/GNU/bash/dd_command_wiki.org] [ here] ]
** Display files
Show content of a file. Concatenates and writes file.
Using "tac" reverses the result of "cat" command
: cat file.txt
Using head, tail and less to show parts of files
Showing the first two entries
#+begin_src bash
head -n 2 /etc/hosts
#+end_src
#+RESULTS:
| 127.0.0.1 | localhost |
| 127.0.1.1 | myuser |
Showing the last two entries
#+begin_src bash
tail -n 2 /etc/hosts
#+end_src
#+RESULTS:
| ff02::1 | ip6-allnodes |
| ff02::2 | ip6-allrouters |
#+begin_src bash
less /etc/hosts
#+end_src
#+RESULTS:
| 127.0.0.1 | localhost | | |
| 127.0.1.1 | myuser | | |
| ::1 | localhost | ip6-localhost | ip6-loopback |
| ff02::1 | ip6-allnodes | | |
| ff02::2 | ip6-allrouters | | |
** Backup a file
Using "cp"
: cp bbdb{ ,.bak}
: bbdb.bak
Backups with tar ( tape archiver)
Back up of "home"
2022-01-26 20:40:43 +01:00
* c = create file
* v = verbose
* f = write to a file/device
* z = compress the file ( gzip) "tar.bz"
* j = compress the file ( bzip2)
2021-12-09 21:18:43 +01:00
: tar -czvf /tmp/home.tar.gz /home
** Simple code to encrypt
Encrypt:
: gpg -c X
: gpg X
Use the code to remove the original file:
: shred --remove X
** Searching
2022-01-26 20:40:43 +01:00
*** Search for files
'find' can be used with multiple arguments such as:
* -name: search for name
* -iname: search for case -insensitive name
* -type: type of file
* f = file
* d = directory
Search within all the directory and subdirectories by name
#+begin_src bash
find ./ -name "gitea*"
#+end_src
#+RESULTS:
./imagens/gitea-sm.png
Search for all the files ending in '.png'
#+begin_src bash
find ./ -name "*.png"
#+end_src
#+RESULTS:
./imagens/gitea-sm.png
./files/plot.png
Search for empty files in 'home'
#+begin_src bash
find /home -type f -empty
#+end_src
'find' can also be used with grep to match patterns:
#+begin_src bash
cd ~/Documentos/dat/intro_r/answers
find . | grep -E '.js|.png'
#+end_src
#+RESULTS:
| ./plot.png |
| ./vizjs.js |
| ./Rplot.png |
| ./bipartiteD3Script.js |
| ./demo1.js |
To find file with different name patters, 'find' can also be used as follows:
#+begin_src bash
find . -type f \( -name "*.org" -o -name "*.md" \)
#+end_src
#+RESULTS:
| ./README.org |
| ./bash_tutorial.org |
*** Matching patterns
Print lines within files that match patterns
2021-12-09 21:18:43 +01:00
: grep
Example: searching for all entries that have "root"
#+begin_src bash
grep 'root' /etc/passwd
#+end_src
#+RESULTS:
: root:x:0:0::/root:/bin/bash
2022-01-26 20:40:43 +01:00
Additional arguments can be used with ~grep~:
* r: recursive search ( within directories)
* n: print line number
* i: case -insensitive search
* '*' Wildcards to search in all the directories
Search within files in the actual directory with wildcards
#+begin_src bash
grep -rni IFELSE *
#+end_src
#+RESULTS:
| answers/repl_values.R:137: mutate( Status = ifelse( Status = = "almost peak" & |
| answers/categ_colum.R:40:df$lead_likely <- ifelse( grepl( "UNCOVERED CURB | SERVICE" , df$remarks ) , "non-lead" , "unknown" ) |
| docs/vectorization.org:354:( counting number of repeated value) , and = ifelse = ( vectorized if ...else|
'grep' can be used also to search for files in directories as shown in the anterior
example using grep with extended regular expressions ( "-E" ) and 'find'
: find . | grep -E '.js|.png'
In this example, 'find' will be passed to 'grep' and any pattern matching
will be printed
2021-12-09 21:18:43 +01:00
** Download files
This code download the file and the argument ~-O~ is to rename the file
#+begin_example
wget https://filetodownload.org -O new_file
#+end_example
2022-01-26 20:40:43 +01:00
2021-12-09 21:18:43 +01:00
* Devices
- mount /what /where
- what = device name
- where = directory
- /mnt = devices that mount occasionally
- /media = devices that mount frequently
: run/$USER /media/$LABEL
* List process
Use ~ps~, ~top~ or ~htop~ to list process:
* ~ps~ reports a snapshot of the current process
: ps aux | less
* ~top~ display dynamic real-time view of linux process
: top
* ~htop~ displays an interactive process viewer
: htop
To kill a process use:
: kill [ PID]
or
: pkill [ PID]
PID = process identificator
* Evaluate time to load
In emacs:
: time emacs --eval '(save-buffers-kill-terminal)'
* Cool commands
: cal # calendar
: clear # clear output
: uname # display system information (-r, -p, -a)
: wc # wordcount (file name)
: date # check the date
* References:
* [ [ https://wiki-dev.bash-hackers.org/] [ The bash hackers wiki] ]
* [ [ https://link.springer.com/content/pdf/bfm%3A978-1-4302-6829-1%2F1.pdf?error= cookies_not_supported& code = 2e41714e-ca8f-4796-a077-3243c836ec90] [ Beginning the linux command line] ]
* [ [ https://pandoc.org/] [ Pandoc manual] ]
* [ [ https://stackoverflow.com/] [ stackoverflow] ]