multifarm
/
bash-tutorial
7
2
Fork 0
Code Issues 1 Pull Requests Projects Releases Wiki Activity
bash-tutorial/bash_tutorial.org

819 lines
19 KiB
Org Mode
Raw Blame History

#+TITLE: bash commands
#+DATE: 2021-08-30
#+STARTUP: indent
#+OPTIONS: num:nil
* DONE work in bash tutorial
:LOGBOOK:
CLOCK: [2021-09-08 mié 18:52]--[2021-09-08 mié 19:22] => 0:30
CLOCK: [2021-09-04 sáb 16:04]--[2021-09-04 sáb 16:34] => 0:30
CLOCK: [2021-09-04 sáb 15:19]--[2021-09-04 sáb 15:49] => 0:30
:END:
* Variables
System variables are stored in users environment
- ~$PATH~ Finds the content of a variable ("PATH")
- "env" variables defined in shell environment
* Add directory to path
[[https://wiki.archlinux.org/title/Environment_variables]["Environment variables" in: Arch wiki]]
To add a directory to the PATH for local usage, put
the following in "~/.bash_profile":
#+begin_example bash
export PATH="${PATH}:/home/my_user/bin"
#+end_example
This will add the directory at the end of $PATH. To update the variable, re-login or source the file:
#+begin_example bash
source ~/.bash_profile
#+end_example
Then check directories in "$PATH"
#+begin_example bash
echo $PATH
#+end_example
* Adding a new user
Check users that are logged
#+begin_src bash
users
#+end_src
#+RESULTS:
: myuser
New user called "test"
#+begin_example bash
sudo useradd -p -m test
#+end_example
The configuration file "useradd" sets options for this command:
#+begin_src bash
sudo less /etc/default/useradd
#+end_src
#+RESULTS:
| # useradd defaults file for ArchLinux |
| # original changes by TomK |
| GROUP=users |
| HOME=/home |
| INACTIVE=-1 |
| EXPIRE= |
| SHELL=/bin/bash |
| SKEL=/etc/skel |
| CREATE_MAIL_SPOOL=no |
| (END) |
PS: should use "sudo" to see the file. And ~less()~ or ~more()~ will show the file
The following file contains a list of fields about each user:
#+begin_src bash
head -4 /etc/passwd
#+end_src
#+RESULTS:
| root:x:0:0::/root:/bin/bash |
| bin:x:1:1::/:/sbin/nologin |
| daemon:x:2:2::/:/sbin/nologin |
| mail:x:8:12::/var/spool/mail:/sbin/nologin |
PS: ~head()~ is used to show only the first four lines.
* Describe hardware
Here using piping to show only the first four lines in the output
#+begin_src bash
lshw | head -4
#+end_src
#+RESULTS:
| myuser | | |
| description: | Computer | |
| width: | 64 | bits |
| capabilities: | smp | vsyscall32 |
* Execute stuffs
** Streaming from terminal
To watch a stream use video player such as 'mpv' or 'vlc' that supports streaming
And pass the streaming address as argument to the name of the media player
#+begin_example bash
mpv https://live0.emacsconf.org/main.webm
#+end_example
** Run a bash script
*** Write script
After writing, save the file with '.sh' extension. This script will
echo the words within quotes
#+begin_src bash
#!/bin/bash
echo "Hellow hu-hu"
#+end_src
#+RESULTS:
: Hellow hu-hu
*** Option a: make it executable
Use the command "chmod" to change permissions for the file
#+begin_example bash
chmod +x script.sh
#+end_example
Then, execute
#+begin_example bash
./script.sh
#+end_example
*** Option b: run from terminal
#+begin_example bash
bash script.sh
#+end_example
Or
#+begin_example bash
sh script.sh
#+end_example
PS: to execute the script from any place in the machine include
the script in a directory that can be read from ~$PATH~
*** Option c: run from graphical interface
** Run R scripts
Write the script and save it. Then go to the directory where the script
was saved and run this code from terminal
#+begin_src bash
Rscript script.R
#+end_src
* Manuals
To explore the manuals use the command ~man~ or ~info~ followed by the name of the function or program
#+begin_src bash
man
#+end_src
#+RESULTS:
| What manual page do you want? |
| For example, try 'man man'. |
Search for information about the manual interface within ~man~
: man man
Description of the system hierarchy
: man hier
Showing only the first four lines of "info" interface
#+begin_src bash
info | head -4
#+end_src
#+RESULTS:
| File: | dir, | Node: | Top, | This | is | the | top | of | the | INFO | tree. |
| | | | | | | | | | | | |
| This | is | the | Info | main | menu | (aka | directory | node). | | | |
| A | few | useful | Info | commands: | | | | | | | |
Search for ~info~ page about Bash shell features
: info bash
The manuals can also be accessed from emacs using "M-x" (using the "meta" key or CTRL + x)
: M-x info
: M-x man
: M-x woman
* Show command history
: history
The file can be found in the user home:
: /home/user/.bash_history
* Translators and dictionaries
** Apertium translator
Installing in Parabola GNU-linux
: sudo pacman -S apertium-eng-spa-git
: sudo pacman -S apertium-lex-tools
: sudo pacman -S apertium-viewer-2.5.3-1
Apertium can be used in the command line
: echo Hola | apertium spa-eng
: echo Hello | apertium eng-spa
The pipe operator "~|~" is used to pass the input to apertium
* Files and directories
** Create empty file
Create "file.txt" with command ~touch~
: touch file.txt
Create a file redirecting the shell output using "~>~" operator
: echo I will put this text wthin a file > file2.txt
** Redirect input, output and standard error
Everything in GNU systems is a file. The keyboard and the terminal are also a file.
The keyboard is used for input and the terminal is used by commands as output.
The files can be found in directory "fd", which stands for file descriptor)
#+begin_src bash
cd /dev/fd/
ls -l
#+end_src
#+RESULTS:
| total | 0 | | | | | | | | | |
| lr-x------ | 1 | user | user | 64 | dec | 15 | 10:31 | 0 | -> | /tmp/babel-yl1RTd/ob-input-izpXuk |
| l-wx------ | 1 | user | user | 64 | dec | 15 | 10:31 | 1 | -> | pipe:[42063] |
| lr-x------ | 1 | user | user | 64 | dec | 15 | 10:31 | 17 | -> | /dev/pts/0 |
| l-wx------ | 1 | user | user | 64 | dec | 15 | 10:31 | 2 | -> | /tmp/emacsA1M18o |
* standard input stream is bound to /dev/fd/0
* standard output (stout) stream is bound to /dev/fd/1
* standard errror (stderr) stream is bound to /dev/fd/2
During interaction a terminal gets written to stdin (dev/fd/0), which
a command read. The command then do an action and writes the output
(stout or stderr) where it will read by the terminal and is displayed
to the user.
To redirect use the operator `>` as follows:
* `command > file_to_redirect`
The operator `>` redirects stout, i.e., is the same that 1>. This code
creates a new file with the list of files present in the actual directory:
#+begin_src bash
ls -l > file_redirect.txt
cat file_redirect.txt
#+end_src
#+RESULTS:
| total | 32 | | | | | | | |
| -rw-r--r-- | 1 | user | user | 15782 | dec | 15 | 10:49 | bash_tutorial.org |
| -rw-r--r-- | 1 | user | user | 0 | dec | 15 | 10:49 | file_redirect.txt |
| -rw-r--r-- | 1 | user | user | 11895 | dec | 9 | 17:44 | markdown.md |
| -rw-r--r-- | 1 | user | user | 121 | dec | 9 | 17:17 | README.org |
Use "2>&1" to redirect stderr with the stout. Using an arbitrary word and redirect it
to the file should print a warning message that is redirected to the file:
#+begin_src bash
ls -l > file_redirect.txt
paco >> file_redirect.txt 2>&1
cat file_redirect.txt
#+end_src
#+RESULTS:
| total | 32 | | | | | | | |
| -rw-r--r-- | 1 | user | user | 15995 | dec | 15 | 12:39 | bash_tutorial.org |
| -rw-r--r-- | 1 | user | user | 0 | dec | 15 | 13:41 | file_redirect.txt |
| -rw-r--r-- | 1 | user | user | 11895 | dec | 9 | 17:44 | markdown.md |
| -rw-r--r-- | 1 | user | user | 121 | dec | 9 | 17:17 | README.org |
| bash: | line | 2: | paco: | command | not | found | | |
Use the "<" operator to pass commands to the file:
#+begin_src bash
ls -l > file_redirect.txt
paco >> file_redirect.txt 2>&1
sort < file_redirect.txt
cat file_redirect.txt
#+end_src
#+RESULTS:
| bash: | line | 2: | paco: | command | not | found | | |
| -rw-r--r-- | 1 | user | user | 0 | dec | 15 | 13:44 | file_redirect.txt |
| -rw-r--r-- | 1 | user | user | 121 | dec | 9 | 17:17 | README.org |
| -rw-r--r-- | 1 | user | user | 16501 | dec | 15 | 13:41 | bash_tutorial.org |
| total | 24 | | | | | | | |
| total | 24 | | | | | | | |
| -rw-r--r-- | 1 | user | user | 16501 | dec | 15 | 13:41 | bash_tutorial.org |
| -rw-r--r-- | 1 | user | user | 0 | dec | 15 | 13:44 | file_redirect.txt |
| -rw-r--r-- | 1 | user | user | 121 | dec | 9 | 17:17 | README.org |
| bash: | line | 2: | paco: | command | not | found | | |
** List files
Use ~ls~ in the directory you want to explore
: ls
List with properties using options ~-l~ and ~-la~
: ls -l
: ls -la
List using wildcards
List all files ending in ".csv"
: ls *.csv
List all files containing the characters "moda" within name, e.g.,
"acomoda", "comoda.txt"...
: ls *moda*
Remove "x"
: rm x
Copy a file using "~cp~ source destination"
: cp /home/text.csv /home/myuser/text.csv
Copy multiple files, directories or disk using "rsync" (source, destination).
The command accepts arguments:
- "-r" recursive
- "-v" verbose
- "-a" archive (keeps info about files)
: rsync -rav /home/user/Documents /run/media/user/disk/backup/
~rsync~ can be also used with selective copyng based on file type
: rsync /home/user/Desktop/*.jpg /home/user/Desktop/backupdata/
Create directory
: mkdir new_dir
Remove (unlink) a directory
Some arguments:
* -r: recursively unlink
* -v: verbose
* -f: force
: rm -r new_dir
Remove a file
: rm file.txt
Space used by a directory
: du -hs /usr
Display directory
: pwd
Change directory you are working from terminal.
Go to the home of the user
: cd
Go one level up in the directory tree
: cd ..
Go to "Documents" directory
: cd /home/myuser/Documents
** Use rmlint to search for duplicated or empty files
Install
: sudo pacman -S rmlint
Run in the directory you want to check for duplicated or empty files
: rmlint -g
This will create two files:
* rmlint.sh
* rmlint.json
The shell script can be used to delete duplicated files
OPTIONS:
- h Show this message.
- d Do not ask before running.
- x Keep rmlint.sh; do not autodelete it.
- p Recheck that files are still identical before removing duplicates.
- r Allow deduplication of files on read-only btrfs snapshots. (requires sudo)
- n Do not perform any modifications, just print what would be done. (implies -d and -x)
- c Clean up empty directories while deleting duplicates.
- q Do not show progress.
- k Keep the timestamp of directories when removing duplicates.
- i Ask before deleting each file
: chmod +x rmlint.sh
: bash -xcp rmlint.sh
** Convert pdf to text or html using poppler
Use poppler and redirect the text to a org file.
~pdftotext~ converts a pdf to "txt" if no other format is
specified
: pdftotext foo.pdf foo
: pacman -S poppler
: pdftotext foo.pdf ->> foo.org # first option
: pdftotext foo.pdf foo.org # second option
Convert pdf to html
: pfdftohtml foo.pdf foo.html
** Convert files using pandoc
Use pandoc with "-s" flag to produce a standalone document and "-o" to
redirect output to a file.
In the example a file "foo" is converted from ~odt~ to ~org~
: pandoc -s -o foo.org foo.odt
** Convert files to pdf using libreoffice
Convert odt file "tcl_online3.odt" to pdf
: libreoffice --headless --convert-to pdf tcl_online3.odt
** Using 'sed' to substitute text
"~sed~" is a stream editor for filtering and transforming text"...
This command in emacs editor
: M-x woman [RET] sed
Visits the following file with info about ~sed~
usr/share/man/man1/sed.1.gz
A brief example:
I have a long comma separated text file with strings and
need to include a line break after each comma (\n)
: cat filetest
#+RESULTS:
|'string1',|'string2', |'string3'|
Using ~sed~ and redirecting the output to "filetest1"
: $ sed 's/\,/&\n/g' filetest > filetest1
#+RESULTS:
| 'string1', |
| 'string2', |
| 'string3' |
** Send files to another machine with "croc"
: sudo pacman -S croc
Then run:
: croc send clown.txt
This will give a code
: > Code is 34-04-paprika-sugar-micro
: > On the other computer run
: > croc 34-04-paprika-sugar-micro
More info about croc in [[https://github.com/schollz/croc][Github repo]]
** Move or rename
Use "~mv~ source destination" command
: mv /home/myuser/text.txt /home/myuser/Documents/text.csv
Example: moving the ".config" files to ".config3", then moving
".config1" to ".config" to arrange configuration archives.
This creates a copy of each file in the same directory
: pwd # check where are you now
: # moving files
: mv file1.txt file3.txt
*Moving multiple files* can be possible. Selecting all the files ending
in ".mp3" within Downloads
: mv ~/Downloads/*.mp3 ~/Music/
*Moving multiple* files included within a text file. Files are included in "filesmv2.txt" and are moved to
"tmpfiles" directory. "t" argument stands for "target directory"
Each line in the "txt" file looks like this:
'/home/user/file1.pdf' # original
: $ cat filesmv2.txt | xargs mv -t ~/tmpfiles
Source: [[https://unix.stackexchange.com/questions/115734/move-file-by-list-in-file-with-leading-whitespace][stackexchange]]
*Moving multiple files* using wildcards. This moves all files containing the sequence "nasa" in the middle of
the name. '*' indicates that there are characters or numbers before and after the word "nasa"
: mv *roco* -t ./direct
** Flashing USB with "dd"
"dd" (convert and copy a file) is useful to copy information from a disk. It can be used
to create bootable USB. An example with parabola:
: sudo dd if=/home/user/Documents/parabola-2016.11.03-dual.iso of=/dev/sdb1
The ISO image is copied to "sdb1" (USB volume)
More files with examples of use of "dd" [../bash/dd_command.org][here]] and [[~/Documentos/GNU/bash/dd_command_wiki.org][here]]
** Display files
Show content of a file. Concatenates and writes file.
Using "tac" reverses the result of "cat" command
: cat file.txt
Using head, tail and less to show parts of files
Showing the first two entries
#+begin_src bash
head -n 2 /etc/hosts
#+end_src
#+RESULTS:
| 127.0.0.1 | localhost |
| 127.0.1.1 | myuser |
Showing the last two entries
#+begin_src bash
tail -n 2 /etc/hosts
#+end_src
#+RESULTS:
| ff02::1 | ip6-allnodes |
| ff02::2 | ip6-allrouters |
#+begin_src bash
less /etc/hosts
#+end_src
#+RESULTS:
| 127.0.0.1 | localhost | | |
| 127.0.1.1 | myuser | | |
| ::1 | localhost | ip6-localhost | ip6-loopback |
| ff02::1 | ip6-allnodes | | |
| ff02::2 | ip6-allrouters | | |
** Backup a file
Using "cp"
: cp bbdb{,.bak}
: bbdb.bak
Backups with tar (tape archiver)
Back up of "home"
* c = create file
* v = verbose
* f = write to a file/device
* z = compress the file (gzip) "tar.bz"
* j = compress the file (bzip2)
: tar -czvf /tmp/home.tar.gz /home
** Simple code to encrypt
Encrypt:
: gpg -c X
: gpg X
Use the code to remove the original file:
: shred --remove X
** Searching
*** Search for files
'find' can be used with multiple arguments such as:
* -name: search for name
* -iname: search for case-insensitive name
* -type: type of file
* f = file
* d = directory
Search within all the directory and subdirectories by name
#+begin_src bash
find ./ -name "gitea*"
#+end_src
#+RESULTS:
./imagens/gitea-sm.png
Search for all the files ending in '.png'
#+begin_src bash
find ./ -name "*.png"
#+end_src
#+RESULTS:
./imagens/gitea-sm.png
./files/plot.png
Search for empty files in 'home'
#+begin_src bash
find /home -type f -empty
#+end_src
'find' can also be used with grep to match patterns:
#+begin_src bash
cd ~/Documentos/dat/intro_r/answers
find . | grep -E '.js|.png'
#+end_src
#+RESULTS:
| ./plot.png |
| ./vizjs.js |
| ./Rplot.png |
| ./bipartiteD3Script.js |
| ./demo1.js |
To find file with different name patters, 'find' can also be used as follows:
#+begin_src bash
find . -type f \( -name "*.org" -o -name "*.md" \)
#+end_src
#+RESULTS:
| ./README.org |
| ./bash_tutorial.org |
*** Matching patterns
Print lines within files that match patterns
: grep
Example: searching for all entries that have "root"
#+begin_src bash
grep 'root' /etc/passwd
#+end_src
#+RESULTS:
: root:x:0:0::/root:/bin/bash
Additional arguments can be used with ~grep~:
* r: recursive search (within directories)
* n: print line number
* i: case-insensitive search
* '*' Wildcards to search in all the directories
Search within files in the actual directory with wildcards
#+begin_src bash
grep -rni IFELSE *
#+end_src
#+RESULTS:
|answers/repl_values.R:137: mutate(Status = ifelse(Status == "almost peak" &|
|answers/categ_colum.R:40:df$lead_likely <- ifelse(grepl("UNCOVERED CURB | SERVICE", df$remarks), "non-lead", "unknown")|
|docs/vectorization.org:354:(counting number of repeated value), and =ifelse= (vectorized if...else|
'grep' can be used also to search for files in directories as shown in the anterior
example using grep with extended regular expressions ("-E") and 'find'
: find . | grep -E '.js|.png'
In this example, 'find' will be passed to 'grep' and any pattern matching
will be printed
** Download files
This code download the file and the argument ~-O~ is to rename the file
#+begin_example
wget https://filetodownload.org -O new_file
#+end_example
* Devices
- mount /what /where
- what = device name
- where = directory
- /mnt = devices that mount occasionally
- /media = devices that mount frequently
: run/$USER/media/$LABEL
* List process
Use ~ps~, ~top~ or ~htop~ to list process:
* ~ps~ reports a snapshot of the current process
: ps aux | less
* ~top~ display dynamic real-time view of linux process
: top
* ~htop~ displays an interactive process viewer
: htop
To kill a process use:
: kill [PID]
or
: pkill [PID]
PID = process identificator
* Evaluate time to load
In emacs:
: time emacs --eval '(save-buffers-kill-terminal)'
* Cool commands
: cal # calendar
: clear # clear output
: uname # display system information (-r, -p, -a)
: wc # wordcount (file name)
: date # check the date
* References:
* [[https://wiki-dev.bash-hackers.org/][The bash hackers wiki]]
* [[https://link.springer.com/content/pdf/bfm%3A978-1-4302-6829-1%2F1.pdf?error=cookies_not_supported&code=2e41714e-ca8f-4796-a077-3243c836ec90][Beginning the linux command line]]
* [[https://pandoc.org/][Pandoc manual]]
* [[https://stackoverflow.com/][stackoverflow]]
Reference in New Issue View Git Blame Copy Permalink