Practical Programming David Bouchet
Practical Work #1

The Command-Line Interface (CLI)

What Is a Command-Line Interface?

A command-line interface (CLI) is used to execute a program or a command, as you can do in a graphical-user interface (GUI) when you double-click on a file icon or when you use toolbars, buttons and menus.

However, a command-line interface is much more powerful than any graphical-user interface.

To use a command-line interface, you first need a shell, which is a command-line interpreter. It is the text-based interface between the user and the operating system.

On Linux, several shells are available. In this practical, we will use bash (Bourne Again Shell), which is the most commonly used shell on Linux.

You can access the shell through a computer terminal , a virtual console or a terminal emulator. To make it simpler, these three terms will be called either terminal or console.

A terminal provides a command prompt, which is a short line of text at the start of the command line.

Here is an example of a command prompt:

                
                    david@laptop:~$
                
            

Another example:

                
                    david@laptop:~/document/picture$
                
            

On Linux, a command prompt can be configured in different ways. It can be made up of the user login, the computer name, the current directory, the date, etc. It usually ends with the $ character for a regular user. So you have to enter your command after the $.

                
                    david@laptop:~/document/picture$ ls -lh
                
            

In the above example, ls -lh is a command. In order to simplify the examples, we are going to limit the command prompt to the sole $ character.

                
                    $ ls -lh
                
            

Finally, the result of the command will be printed at the start of the line without the $ character.

                
                    $ ls -lh
                    -rw-rw-r--  1 david david 114K dec.   1  2016 doc.7z
                    drwx------  3 david david 4,0K dec.   1  2016 document
                    -rwxr-xr-x  1 david david  239 nov.  28  2016 findall
                    -rw-rw-r--  1 david david   39 may   10  2017 key.txt
                    -rw-rw-r--  1 david david  119 mar.  30 16:45 list.txt
                    -rw-rw-r--  1 david david   79 may   10  2017 main.c
                    -rw-rw-r--  1 david david 7,8K oct.  19  2017 planning.xlsx
                    drwxrwxr-x  6 david david 4,0K nov.   8  2017 projects
                    -rw-r--r--  1 david david  12K mar.  31  2017 soft.icon
                    -rw-rw-r--  1 david david  40K may   17  2017 to_do.txt
                    -rw-rw-r--  1 david david  185 feb.   3  2017 version.txt
                
            

Do not worry if you do not understand the output of this command. We will cover the commands in the next section.

The Commands

The Global Syntax

A plain command is made up of three parts (some of them are optional).

                
                    $ command_name [OPTIONS...] [PARAMATERS...]
                
            

The parameters must be separated by the space character.

                
                    $ command_name param1 param2 param3
                
            

To include a space character into a parameter, you have to use the double quote.

                
                    $ command_name "param 1" param2 "and param 3"
                
            

A short option is made up of a single character prefixed with the - character.

                
                    $ command_name -a -b param1 param2
                
            

In the above example, the instruction has two options: -a and -b.

Several options can be merged. For instance, the example below is equivalent to the above.

                
                    $ command_name -ab param1 param2
                
            

A long option is made up of several characters prefixed with the -- characters.

                
                    $ command_name --option --second-option param1 param2
                
            

The long options cannot be merged.

Finally, let us have a look at the following command:

                
                    $ command_name -a -bcd --option --second-option param1 param2 "last param"
                
            

This command has:

You should know that there are hundreds of commands and each command can have dozens or even hundreds of different options. Therefore, it is not possible to explain every command in detail.

However, beyond the understanding of commands, you also need to grasp some of the principles in the philosophy of Linux if you want to be able to find by yourself the commands you need that will meet your requirements.

That is the purpose of this practical.

So now, open a terminal and try the commands below.

Your First Commands

Obviously, as the directory hierarchy of your computer is different from the one used in this section, you must adapt the commands to your own computer.

The pwd Command

The pwd command prints the name of the current directory (pwd stands for print working directory). Try this command and see what your current directory is.

                
                    $ pwd
                    /home/david
                
            

The cd Command

The cd command changes the current directory (cd stands for change directory).

                
                    $ pwd
                    /home/david
                    $ cd document
                    $ pwd
                    /home/david/document
                    $ cd project/c
                    $ pwd
                    /home/david/document/project/c
                
            

You can use the .. pair of characters to move back into a parent directory.

                
                    $ pwd
                    /home/david/document/project/c
                    $ cd ..
                    $ pwd
                    /home/david/document/project
                    $ cd ../../..
                    $ pwd
                    /home
                
            

Type cd without parameters to move into your home directory.

                
                    $ pwd
                    /home/david/document/project/c
                    $ cd
                    $ pwd
                    /home/david
                
            

Start the path with the / character to specify an absolute path.

                
                    $ pwd
                    /home/david/document/project/c
                    $ cd /home/david
                    $ pwd
                    /home/david
                    $ cd /
                    $ pwd
                    /
                
            

The last current directory in the above example is the root directory (/). It is the topmost directory (it has no parent).

Use the ~ character to specify your home directory. The cd ~ instruction is then equivalent to the cd instruction without parameters.

                
                    $ pwd
                    /home/david/document/project/c
                    $ cd ~/document
                    $ pwd
                    /home/david/document
                    $ cd ~
                    $ pwd
                    /home/david
                
            

A path can be absolute or relative. A path starting with the / character or the ~ character is absolute. Otherwise, it is relative to the current working directory.

You can go back to the previous working directory by using the - character.

                
                    $ pwd
                    /home/david/document/project/c
                    $ cd ~/picture
                    $ pwd
                    /home/david/picture
                    $ cd -
                    $ pwd
                    /home/david/document/project/c
                
            

The ls Command

The ls command lists the directory contents (ls is shorthand for list). Without parameters, it lists the contents of the current directory.

                
                    $ pwd
                    /home/david/document/project/c
                    $ ls
                    a.out  facto.c  facto.h  facto.o  lib  main.c  main.h  main.o
                
            

Some terminals color-code items according to their types, which makes them more easily identifiable.

You can also use the -F option, which appends indicators to entries. Be careful, respecting the case is indispensable: -F is not the same as -f.

Indicator Meaning
None Non-executable regular file
* Executable regular file
/ Directory
Others Special file

There are several kinds of special files, but the purpose of this practical is not to explain them all. So just ignore them for the time being.

                
                    $ ls -F
                    a.out*  facto.c  facto.h  facto.o  lib/  main.c  main.h  main.o
                
            

Therefore, we can see that a.out is an executable file and lib is a directory.

To obtain extra information about files, you can also try the -l option, which uses a long listing format.

                
                    $ ls -l
                    total 96
                    -rwxrwxr-x 1 david david 8664 may   17 23:23 a.out
                    -rw-rw-r-- 1 david david  101 may   17 23:23 facto.c
                    -rw-rw-r-- 1 david david   18 may   17 23:15 facto.h
                    -rw-rw-r-- 1 david david 1264 may   17 23:21 facto.o
                    drwxrwxr-x 2 david david 4096 may   17 23:10 lib
                    -rw-rw-r-- 1 david david   90 may   17 23:21 main.c
                    -rw-rw-r-- 1 david david   38 may   17 23:18 main.h
                    -rw-rw-r-- 1 david david 1592 may   17 23:20 main.o
                
            

This output requires some explanations. To begin with, ignore the first line: total 96. It indicates the total number of blocks allocated by the directory, and, at your level, this piece of information is not yet useful.

Then, from right to left, you find:

Usually, the owner of the file is the one who has created the file.

For the time being, you do not have to know much about groups. Just remember that on Linux, users can be grouped together so that they can have the same access rights to specific files and directories.

Now, let us have a look at the file type and the access rights. For instance, the -rwxrwxr-x can be split into four groups:

The leftmost character is an indicator that gives the type of the file. A file can be a regular file, a directory or a special file. On Linux, everything is a file, even a directory. As said previously, you can ignore the special files for the time being.

Indicator Meaning
- Regular file (executable or not)
d Directory
Others Special file

We can then see that this directory has:

Then, the nine remaining characters (three groups of three characters) give the access permissions of the file:

For example, the a.out file (rwxrwxr-x) has the following access rights:

Owner Group Others
rwx rwx r-x

Three types of access rights are possible:

Indicator Access Right
r Read
w Write
x Execute

For instance, the a.out file (rwxrwxr-x) can be read, written and executed by the owner and the members of the group. The other users can only read and execute it.

The facto.c file (rw-rw-r--) cannot be executed at all. It can be read and written by the owner and the members of the group, but can be read only by the other users.

You can also add parameters to the ls command. They can be filenames or directories.

                
                    $ ls
                    a.out  facto.c  facto.h  facto.o  lib  main.c  main.h  main.o
                    $ ls -l a.out
                    -rwxrwxr-x 1 david david 8664 may   17 23:23 a.out
                    $ ls -l a.out main.c
                    -rwxrwxr-x 1 david david 8664 may   17 23:23 a.out
                    -rw-rw-r-- 1 david david   90 may   17 23:21 main.c
                    $ ls -l lib
                    total 1848
                    -rw-r--r-- 1 david david 296936 may   12  2014 libqeglfs.so
                    -rw-r--r-- 1 david david 268168 may   12  2014 libqkms.so
                    -rw-r--r-- 1 david david 189384 may   12  2014 libqlinuxfb.so
                    -rw-r--r-- 1 david david 164424 may   12  2014 libqminimalegl.so
                    -rw-r--r-- 1 david david  39656 may   12  2014 libqminimal.so
                    -rw-r--r-- 1 david david 147848 may   12  2014 libqoffscreen.so
                    -rw-r--r-- 1 david david 708920 may   12  2014 libqxcb.so
                    $ ls -l a.out main.c lib
                    -rwxrwxr-x 1 david david 8664 may   17 23:23 a.out
                    -rw-rw-r-- 1 david david   90 may   17 23:21 main.c
        
                    lib:
                    total 1848
                    -rw-r--r-- 1 david david 296936 may   12  2014 libqeglfs.so
                    -rw-r--r-- 1 david david 268168 may   12  2014 libqkms.so
                    -rw-r--r-- 1 david david 189384 may   12  2014 libqlinuxfb.so
                    -rw-r--r-- 1 david david 164424 may   12  2014 libqminimalegl.so
                    -rw-r--r-- 1 david david  39656 may   12  2014 libqminimal.so
                    -rw-r--r-- 1 david david 147848 may   12  2014 libqoffscreen.so
                    -rw-r--r-- 1 david david 708920 may   12  2014 libqxcb.so
                
            

Let us see another useful option of the ls command: -h. It prints human-readable sizes (e.g. 105K 352M 1G).

                
                    $ ls -lh lib
                    total 1,9M
                    -rw-r--r-- 1 david david 290K may   12  2014 libqeglfs.so
                    -rw-r--r-- 1 david david 262K may   12  2014 libqkms.so
                    -rw-r--r-- 1 david david 185K may   12  2014 libqlinuxfb.so
                    -rw-r--r-- 1 david david 161K may   12  2014 libqminimalegl.so
                    -rw-r--r-- 1 david david  39K may   12  2014 libqminimal.so
                    -rw-r--r-- 1 david david 145K may   12  2014 libqoffscreen.so
                    -rw-r--r-- 1 david david 693K may   12  2014 libqxcb.so
                
            

The -h and -F short options have their long equivalents.

Short Long
-h --human-readable
-F --classify

For example, the two commands below are equivalent.

                
                    $ ls -l --classify --human-readable 
                    total 96K
                    -rwxrwxr-x 1 david david 8,5K may   17 23:23 a.out*
                    -rw-rw-r-- 1 david david  101 may   17 23:23 facto.c
                    -rw-rw-r-- 1 david david   18 may   17 23:15 facto.h
                    -rw-rw-r-- 1 david david 1,3K may   17 23:21 facto.o
                    drwxrwxr-x 2 david david 4,0K may   21 18:10 lib/
                    -rw-rw-r-- 1 david david   90 may   17 23:21 main.c
                    -rw-rw-r-- 1 david david   38 may   17 23:18 main.h
                    -rw-rw-r-- 1 david david 1,6K may   17 23:20 main.o
                    $ ls -lFh
                    total 96K
                    -rwxrwxr-x 1 david david 8,5K may   17 23:23 a.out*
                    -rw-rw-r-- 1 david david  101 may   17 23:23 facto.c
                    -rw-rw-r-- 1 david david   18 may   17 23:15 facto.h
                    -rw-rw-r-- 1 david david 1,3K may   17 23:21 facto.o
                    drwxrwxr-x 2 david david 4,0K may   21 18:10 lib/
                    -rw-rw-r-- 1 david david   90 may   17 23:21 main.c
                    -rw-rw-r-- 1 david david   38 may   17 23:18 main.h
                    -rw-rw-r-- 1 david david 1,6K may   17 23:20 main.o
                
            

The -a or --all option is also very useful. It lists the hidden files. On Linux, the files starting with the . character are hidden. In other words, this option does not ignore entries starting with the . character.

For instance:

                
                    $ ls -la
                    total 116
                    drwxrwxr-x 4 david david 4096 may   24 12:42 .
                    drwxrwxr-x 7 david david 4096 may   17 23:07 ..
                    -rwxrwxr-x 1 david david 8664 may   17 23:23 a.out
                    -rw-rw-r-- 1 david david  101 may   17 23:23 facto.c
                    -rw-rw-r-- 1 david david   18 may   17 23:15 facto.h
                    -rw-rw-r-- 1 david david 1264 may   17 23:21 facto.o
                    drwxrwxr-x 7 david david 4096 may   24 12:42 .git
                    -rw-rw-r-- 1 david david    0 may   24 12:42 .gitignore
                    drwxrwxr-x 2 david david 4096 may   21 18:10 lib
                    -rw-rw-r-- 1 david david   90 may   17 23:21 main.c
                    -rw-rw-r-- 1 david david   38 may   17 23:18 main.h
                    -rw-rw-r-- 1 david david 1592 may   17 23:20 main.o
                
            

Four new lines have appeared:

                
                    $ ls -la
                    total 116
                    drwxrwxr-x 4 david david 4096 may   24 12:42 .
                    drwxrwxr-x 7 david david 4096 may   17 23:07 ..
                    # ...snip...
                    drwxrwxr-x 7 david david 4096 may   24 12:42 .git
                    -rw-rw-r-- 1 david david    0 may   24 12:42 .gitignore
                    # ...snip...
                
            

To sum up:

ls
List Directory Contents
Option Description
Short Long
-F --classify Append indicator to entries.
-l Use a long listing format.
-h --human-readable With -l, print human-readable sizes.
-a --all Do not ignore entries starting with the . character.

So far, we have seen only four options of the ls command, but actually this command has many more options (around sixty).

The tree Command

The ls command is not the only one that allows you to list directory contents, but it is the most commonly used. Another really useful command is the tree command. Let us see it briefly.

The tree command allows you to list directory contents in a tree-like format. Without parameters, it lists the contents of the current directory.

For example:

                
                    $ ls -F
                    dir1/  dir2/  README
                    $ tree
                    .
                    ├── dir1
                    │   ├── file1.pdf
                    │   ├── file2.pdf
                    │   ├── file3.pdf
                    │   ├── image1.jpg
                    │   ├── image2.jpg
                    │   ├── image3.jpg
                    │   └── README
                    ├── dir2
                    │   ├── file1.txt
                    │   ├── file2.txt
                    │   ├── file3.txt
                    │   ├── page1
                    │   │   ├── index.html
                    │   │   ├── README
                    │   │   └── style.css
                    │   ├── page2
                    │   │   ├── index.html
                    │   │   ├── README
                    │   │   └── style.css
                    │   ├── picture1.png
                    │   ├── picture2.png
                    │   ├── picture3.png
                    │   └── README
                    └── README
                
            

Before learning new commands, you should know that one of the Linux principles is that the less you type, the more productive you are and the fewer mistakes you make. The shell provides different ways to stick to this principle. Let us see two of them: the autocompletion and the history.

Autocompletion

Command-line interpreters often come with autocompletion facilities. Some of them are smarter than others and they are not always enabled by default. It depends on your Linux distribution.

Usually, you can autocomplete commands, options and parameters by using the TAB key.

So while typing a command, press TAB at any time. The command will be completed automatically if there is only one. If there are more, you must press TAB a second time to have all of the possible lines displayed on the terminal.

For instance, let us examine the following example. Start by typing the following characters:

                
                    $ ls --h
                
            

Now, press the TAB key only once. Nothing happens. It means that several possibilities are available.

Press the TAB key a second time. All the possibilities are displayed.

                
                    $ ls --h
                    --help                --hide-control-chars
                    --hide=               --human-readable
                    $ ls --h
                
            

We can see that four long options start with --h. Now, append a u to your command.

                
                    $ ls --hu
                
            

Press the TAB key only once.

                
                    $ ls --human-readable
                
            

As you can see, the option has been fully completed.

What we did here for a long option can be done for command names and parameters as well. So try the autocompletion until you get the hang of it.

History

There is another way to avoid typing commands: the history of commands.

By pressing the up and down arrow keys, you can move back and forth through the history of commands. Try to execute some commands and then press the up and down arrow keys several times. As you can see, you can easily execute previous commands without typing them again.

You can also use the following shorhands in order to execute previous commands.

Here are some examples:

                
                    $ cd /home
                    $ ls -laF
                    total 48
                    drwxr-xr-x  5 root  root   4096 june  24  2016 ./
                    drwxr-xr-x 25 root  root   4096 june   4 10:24 ../
                    drwx-----x 73 david david 20480 june  20 22:23 david/
                    $ pwd
                    /home
                    $ !! # Execute the previous command: pwd
                    p
                
                
                    wd
                    /home
                    $ !-2 # Execute the command before the previous command: ls -laF
                    l
                
                
                    s -laF
                    total 48
                    drwxr-xr-x  5 root  root   4096 june  24  2016 ./
                    drwxr-xr-x 25 root  root   4096 june   4 10:24 ../
                    drwx-----x 73 david david 20480 june  20 22:23 david/
                    $ !c # Execute the first previous command that starts with 'c': cd /home
                    c
                
                
                    d /home
                
            

Command Types

Now that we have seen a few commands and some of their options, you should ask yourselves:

That is what we are going to discuss in the next section. But first, you should know that there are several types of commands. Indeed, the way to find some documentation about commands depends on their types.

So, commands can be divided into three groups:

Let us describe them.

The Executable Files

When a command is an executable file, the shell executes this command by running a program file associated with this command. It means that some files in your computer are associated with some commands. These files are located at different places in your disk drive, but all these places must be known by the command interpreter. So, these places are held in an environment variable called $PATH. To print the contents of this variable you can type the following command:

                
                    $ echo $PATH
                    /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
                
            

Do not try to understand this command for the time being. We will see the echo command later on. So, just focus on the result (note that it can differ according to your operating system).

We can see six different paths all separated by the : character. It means that when you enter a command in the command prompt, which is associated with an executable file, the shell is then looking for its associated file in the paths held in the $PATH variable. The associated file and the command have the same name.

For instance, when you enter the ls instruction:

Therefore, the associated file of the ls command is somewhere in one of these directories. We will see later on in this practical how to find the exact location of a command.

The Aliases

In fact, an alias is not a command in itself. It is used to define new commands by replacing one or several commands and their options by a single word. In other words, it is a way to abbreviate a long command, which is often used, to a short command (remember? the less you type, the more productive you are, the fewer mistakes you make). Some operating systems have predefined aliases but you can also define your own. To print the defined aliases just execute the alias command.

                
                    $ alias
                    alias egrep='egrep --color=auto'
                    alias fgrep='fgrep --color=auto'
                    alias grep='grep --color=auto'
                    alias l='ls -CF'
                    alias la='ls -A'
                    alias ll='ls -alF'
                    alias ls='ls --color=auto'
                
            

As you can see on the above example, an ls alias is defined, but as said previously, ls is already a command.

First, you have to know that by default, the ls command prints its result with no colors. To enable the colors, you have to add its --color=auto option. But it is cumbersome to type this option every time you need the ls command. So it is much more convenient to define an alias that replaces the "ls" occurrence by the "ls --color=auto" occurrence.

Aliases take precedence over all types of commands.

For example, when you type ls -lh in the command prompt, the shell starts by checking if an alias is associated with this command. In our case, it replaces ls by ls --color=auto. So the final command (the one that is executed) is: ls --color=auto -lh

You can define your own aliases by using the following syntax:

                
                    $ alias name_of_the_alias='command_to_be_executed'
                
            

For instance:

                
                    $ alias my_ls='ls -lFh'
                    $ alias
                    alias egrep='egrep --color=auto'
                    alias fgrep='fgrep --color=auto'
                    alias grep='grep --color=auto'
                    alias l='ls -CF'
                    alias la='ls -A'
                    alias ll='ls -alF'
                    alias ls='ls --color=auto'
                    alias my_ls='ls -lFh'
                    $ my_ls
                    total 96K
                    -rwxrwxr-x 1 david david 8,5K may   17 23:23 a.out*
                    -rw-rw-r-- 1 david david  101 may   17 23:23 facto.c
                    -rw-rw-r-- 1 david david   18 may   17 23:15 facto.h
                    -rw-rw-r-- 1 david david 1,3K may   17 23:21 facto.o
                    drwxrwxr-x 2 david david 4,0K may   21 18:10 lib/
                    -rw-rw-r-- 1 david david   90 may   17 23:21 main.c
                    -rw-rw-r-- 1 david david   38 may   17 23:18 main.h
                    -rw-rw-r-- 1 david david 1,6K may   17 23:20 main.o
                
            

To remove an alias definition, you can use the unalias command followed by the name of the alias.

                
                    $ unalias my_l
                
                
                    s
                    $ my_ls
                    my_ls: command not found
                
            

The Built-in Commands

A built-in command belongs to the command interpreter (i.e. the shell). These commands can be slightly different from one shell to another. When a built-in command is executed, no external program is invoked by the interpreter.

So far, we have already seen two built-in commands: cd and pwd.

But how can we know if a command is an alias, a built-in command or an executable file?

Determining a Command Type

To determine whether a command is an alias, a built-in command or an executable file, you can use the type -a instruction followed by the command name.

                
                    $ type -
                
                
                    a cd
                    cd is a shell builti
                
                
                    n
                    $ type -a l
                
                
                    s
                    ls is aliased to `ls --color=auto'
                    ls is /bin/l
                
                
                    s
                    $ type -a pw
                
                
                    d
                    pwd is a shell builtin
                    pwd is /bin/pw
                
                
                    d
                    $ type -a typ
                
                
                    e
                    type is a shell builtin
                
            

From the above example, we can deduce that:

When a command has several types, it is executed in the following order of precedence:

  1. The aliases.
  2. The built-in commands.
  3. The executable files.

The case of the ls command, which is an alias and an executable file, has already been mentioned in a previous section.

But what about the pwd command, which is a built-in command and an executable file. So, when you enter this command, the interpreter follows the following steps:

  1. It checks if pwd is an alias and in this case, it is not.
  2. It checks if pwd is a built-in command and in this case, it is. So it executes the command and prints the result.

In fact, the /bin/pwd executable file is never executed in this context. So what is it for? In some contexts, this file can be useful, but for now you can ignore it. At your level, you can assume that if a command is both a built-in command and an executable file, only the built-in command is taken into account, and the executable file is always ignored.

You have to bear in mind that all executable files that are associated with a command must be located in a directory specified in the $PATH variable.

For instance, the ls file is in the /bin directory and this directory is specified in the $PATH variable (the last one).

                
                    $ echo $PATH
                    /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
                
            

Now that you know how to determine a command type, let us see how we can find the documentation of a command.

Documentation

The Executable Files

The --help Option

Most commands that are associated with executable files provide the --help option that prints some brief documentation about the command.

For instance, let us execute the ls --help command.

                
                    $ ls --hel
                
                
                    p
                    Usage: ls [OPTION]... [FILE]...
                    List information about the FILEs (the current directory by default).
                    Sort entries alphabetically if none of -cftuvSUX nor --sort is specified.
        
                    Mandatory arguments to long options are mandatory for short options too.
                      -a, --all                  do not ignore entries starting with .
                      -A, --almost-all           do not list implied . and ..
                          --author               with -l, print the author of each fil
                
                
                    e
                    # ...snip...
                      -
                
                
                      f                         do not sort, enable -aU, disable -ls --color
                        -F, --classify             append indicator (one of */=>@|) to entries
        
                        -h, --human-readable       with -l and/or -s, print human readable size
                
                
                    s
                    # ...snip...
                      -
                
                
                      k, --kibibytes            default to 1024-byte blocks for disk usage
                        -l                         use a long listing format
                        -L, --dereference          when showing file information for a symbolic
                                                   link, show information for the file the link
                                                   references rather than for the link itsel
                
                
                    f
                    # ...snip...
                    U
                
                
                    sing color to distinguish file types is disabled both by default and
                    with --color=never.  With --color=auto, ls emits color codes only when
                    standard output is connected to a terminal.  The LS_COLORS environment
                    variable can change the settings.  Use the dircolors command to set it.
        
                    Exit status:
                     0  if OK,
                     1  if minor problems (e.g., cannot access subdirectory),
                     2  if serious trouble (e.g., cannot access command-line argument).
        
                    GNU coreutils online help: <http://www.gnu.org/software/coreutils>
                    Full documentation at: <http://www.gnu.org/software/coreutils/ls>
                    or available locally via: info '(coreutils) ls invocation'
                
            

For the sake of convenience, only a part of the command output has been displayed. Each line starting with the # ... snip ... occurrence means that the text has been cut. But this is enough to see that we can find a brief description of the command and its options. We can recognize the -a, -F, -h and -l short options we already know as well as their long versions (when they are available). Execute this command on your terminal to see the complete result.

But sometimes this option does not provide enough information, so we can consult the manual pages as well, which are commonly called the man pages.

The Man Pages

A man page is a page of manuel that contains documentation about commands. To display a man page, type the man command followed by the name of a command. For instance:

                
                    $ man l
                
                
                    s
                
            

It displays the documentation of the ls command.

A man page is always a single page.

You can navigate through this page by using the arrow keys and the space bar. To get more information about moving, searching and jumping, press h.

Press q to exit the page.

Man pages can also be found on the Internet. Many websites display them freely. For instance, Debian, which is a famous Linux distribution, puts at your disposal all its man pages. Click on the link below to display the man page of the ls command. It should be similar to the one you displayed previously on your terminal. Several languages are also available.

man ls

The following link allows you to display any command documentation.

https://manpages.debian.org/

Note that the ls man page starts with LS(1). What does this "(1)" mean? It refers to the section of the manual. The man pages have eight sections numbered from 1 to 8. The first section (section 1) is about executable programs or shell commands. Most commands we are going to use in this practical belong to the first section. For the time being, you do not have to know the other sections, but if you are interested, you can have a view of all sections by typing the man man command (or by clicking on this link: man man).

                
                    $ man m
                
                
                    an
                
            

The Info Pages

The info command is another way to find some documentation about commands. Usually the info pages contain more recent and more detailed information than the man pages.

Unlike a man page, an info page can be made up of multiple pages. You can navigate through all these pages by using links.

But unfortunately, it is not really convenient to browse through info pages using a terminal. Try and see for yourselves. For instance type the following command:

                
                    $ info l
                
                
                    s
                
            

For the time being, the man pages are enough for you. So, do not worry if you get lost while browsing through info pages.

The Built-in Commands

You cannot use man to get information about a single built-in command.

                
                    $ man c
                
                
                    d
                    No manual entry for cd
                
            

Instead, you have to use the help command.

                
                    $ help c
                
                
                    d
                    cd: cd [-L|[-P [-e]] [-@]] [dir]
                        Change the shell working directory.
                        
                        Change the current directory to DIR.  The default DIR is the value of the
                        HOME shell variable
                
                
                    .
        
                    # ...snip...
                
            

You can also get information about all the built-in commands by typing:

                
                    $ man b
                
                
                    uiltins
                
            

How to Find Specific Commands

Now, we know some commands and how to find documentation about them. However, what do you do when you want to do something, but do not have a clue about which to use? That's when the apropos command comes in handy.

For instance, let us assume that we want to find a command that displays a simple calendar of the year 2017. So, let us try the following command:

                
                    $ apropos calenda
                
                
                    r
                    cal (1)              - displays a calendar and the date of Easter
                    calendar (1)         - reminder service
                    ncal (1)             - displays a calendar and the date of Easter
                    zshcalsys (1)        - zsh calendar system
                
            

We have four results. The first one seems right. Let us try it on.

                
                    $ cal
                         June 2018        
                    di lu ma me je ve sa  
                                    1  2  
                     3  4  5  6  7  8  9  
                    10 11 12 13 14 15 16  
                    17 18 19 20 21 22 23  
                    24 25 26 27 28 29 30 
                
            

It almost works! Only the current month is displayed. So now, if we want more detail about this command, we can use the manual.

                
                    $ man c
                
                
                    al
                
            

Once in the man page, type /year and press the Enter key to highlight all the occurrences of year. Then press n to jump to the next occurrence until you find an option that can be useful for our purpose. Finally, we can see that the -y option can be used to display a calendar for a specific year. So let us try it with the year 2017.

                
                    $ cal -y 2017
                                                2017
                          January               February                March          
                    di lu ma me je ve sa  di lu ma me je ve sa  di lu ma me je ve sa  
                     1  2  3  4  5  6  7            1  2  3  4            1  2  3  4  
                     8  9 10 11 12 13 14   5  6  7  8  9 10 11   5  6  7  8  9 10 11  
                    15 16 17 18 19 20 21  12 13 14 15 16 17 18  12 13 14 15 16 17 18  
                    22 23 24 25 26 27 28  19 20 21 22 23 24 25  19 20 21 22 23 24 25  
                    29 30 31              26 27 28              26 27 28 29 30 31     
                                                                                      
        
                           April                  May                   June          
                    di lu ma me je ve sa  di lu ma me je ve sa  di lu ma me je ve sa  
                                       1      1  2  3  4  5  6               1  2  3  
                     2  3  4  5  6  7  8   7  8  9 10 11 12 13   4  5  6  7  8  9 10  
                     9 10 11 12 13 14 15  14 15 16 17 18 19 20  11 12 13 14 15 16 17  
                    16 17 18 19 20 21 22  21 22 23 24 25 26 27  18 19 20 21 22 23 24  
                    23 24 25 26 27 28 29  28 29 30 31           25 26 27 28 29 30     
                    30                                                                
        
                            July                 August              September        
                    di lu ma me je ve sa  di lu ma me je ve sa  di lu ma me je ve sa  
                                       1         1  2  3  4  5                  1  2  
                     2  3  4  5  6  7  8   6  7  8  9 10 11 12   3  4  5  6  7  8  9  
                     9 10 11 12 13 14 15  13 14 15 16 17 18 19  10 11 12 13 14 15 16  
                    16 17 18 19 20 21 22  20 21 22 23 24 25 26  17 18 19 20 21 22 23  
                    23 24 25 26 27 28 29  27 28 29 30 31        24 25 26 27 28 29 30  
                    30 31                                                             
        
                          October               November              December        
                    di lu ma me je ve sa  di lu ma me je ve sa  di lu ma me je ve sa  
                     1  2  3  4  5  6  7            1  2  3  4                  1  2  
                     8  9 10 11 12 13 14   5  6  7  8  9 10 11   3  4  5  6  7  8  9  
                    15 16 17 18 19 20 21  12 13 14 15 16 17 18  10 11 12 13 14 15 16  
                    22 23 24 25 26 27 28  19 20 21 22 23 24 25  17 18 19 20 21 22 23  
                    29 30 31              26 27 28 29 30        24 25 26 27 28 29 30  
                
            

It works!

This time, we were lucky because the calendar keyword was enough to find the command we needed. We had to choose between four possibilities only, and the first one was right.

But sometimes, dozens of possibilities, if not hundreds, are displayed. So it can be difficult to choose the right command. To minimize the number of results, you can filter them with more keywords. For instance, we wanted to find a command that displays a calendar. So, display and calendar seem good. Let us try these two keywords.

                
                    $ apropos display calendar
                
            

But this time, we have around two hundred results. What happened? By default, the apropos command displays commands that match any keyword. In other words, it displays all the commands that match the display keyword plus all the commands that match the calendar keyword.

To display only commands that match all the supplied keywords, we have to use the -a option. (Have a look at the apropos man page to find this on your own.)

So, let us try again.

                
                    $ apropos -a display calenda
                
                
                    r
                    cal (1)              - displays a calendar and the date of Easter
                    ncal (1)             - displays a calendar and the date of Easter
                
            

So now, we can choose between two results only. At first glance, they look identical, but if you check their man pages, you could find some small differences (but it is not the purpose of this practical).

Now that you know how to find the command you need, let us see some other useful commands.

Other Useful Commands

Creating Empty Files and Directories

The touch Command

The touch command allows you to create empty files.

                
                    $ ls
                    index.html
                    $ touch style.css
                    $ ls
                    index.html  style.css
                    $ touch main.js lib.js
                    $ ls -lh
                    total 112K
                    -rw-rw-r-- 1 david david 77K june  25 23:11 index.html
                    -rw-rw-r-- 1 david david   0 june  25 23:12 lib.js
                    -rw-rw-r-- 1 david david   0 june  25 23:12 main.js
                    -rw-rw-r-- 1 david david   0 june  25 23:11 style.css
                
            

Sometimes it can be convenient to create empty files. For example, you will occasionally need to create one or several files for testing a command or a program.

You can also use the touch command on existing files. It will update the access and modification times of the files to the current time.

                
                    $ ls -lh
                    total 112K
                    -rw-rw-r-- 1 david david 77K june  25 23:11 index.html
                    -rw-rw-r-- 1 david david   0 june  25 23:12 lib.js
                    -rw-rw-r-- 1 david david   0 june  25 23:12 main.js
                    -rw-rw-r-- 1 david david   0 june  25 23:11 style.css
                    $ touch index.html lib.js 
                    $ ls -lh
                    total 112K
                    -rw-rw-r-- 1 david david 77K june  25 23:36 index.html
                    -rw-rw-r-- 1 david david   0 june  25 23:36 lib.js
                    -rw-rw-r-- 1 david david   0 june  25 23:12 main.js
                    -rw-rw-r-- 1 david david   0 june  25 23:11 style.css
                
            

The mkdir Command

The mkdir command allows you to create directories if they do not already exist.

                
                    $ mkdir first_dir
                    $ ls -F
                    first_dir/
                    $ mkdir second_dir third_dir fourth_dir
                    $ ls -F
                    first_dir/  fourth_dir/  second_dir/  third_dir/
                    $ mkdir second_dir
                    mkdir: cannot create directory ‘second_dir’: File exists
                
            

Removing Files and Directories

The rm Command (Removing Files)

The rm command allows you to remove files.

                
                    $ touch file1 file2 file3 file4 file5
                    $ ls
                    file1  file2  file3  file4  file5
                    $ rm file2
                    $ ls
                    file1  file3  file4  file5
                    $ rm file1 file3 file5
                    $ ls
                    file4
                
            

WARNING! the shell does not have any trash can. That means after you delete files with the rm command, they will be permanently deleted. Therefore, you have to be very careful when using this command.

The rmdir Command (Removing Empty Directories)

The rmdir command allows you to remove empty directories only.

                
                    $ mkdir dir1 dir2 dir3 dir4 dir5
                    $ ls -F
                    dir1/  dir2/  dir3/  dir4/  dir5/
                    $ rmdir dir4
                    $ ls -F
                    dir1/  dir2/  dir3/  dir5/
                    $ rmdir dir1 dir3 dir5
                    $ ls -F
                    dir2/
                    $ touch dir2/file.txt
                    $ ls dir2
                    file.txt
                    $ rmdir d
                
                
                    ir2
                    rmdir: failed to remove 'dir2': Directory not empt
                
                
                    y
                    $ ls -F
                    dir2/
                
            

The rm -rf Command (Removing Non-Empty Directories)

In order to remove non-empty directories, you can use the rm command with the couple of -r and -f options.

                
                    $ ls -F
                    dir2/
                    $ ls dir2
                    file.txt
                    $ rm -rf dir2
                
            

The equivalent command using the long options is:

                
                    $ rm --recursive --force dir2
                
            

As said before, this command deletes files permanently. So, be careful when using it.

Downloading Files

The wget Command

Among other things, the wget command allows you to download any file on the Internet. Its syntax is as follows:

                
                    $ wget url_of_the_file_to_download
                
            

It downloads the file in the current directory.

For instance, go to your home directory. Create the pw_01 directory. Download the example.tar.bz2 compressed file in the pw_01 directory. To do so, type the following command.

                
                    $ cd
                    $ mkdir pw_01
                    $ cd pw_01
                    $ wget http://www.debug-pro.com/epita/prog/s3/pw/pw_01_cli/example.tar.bz2
                    $ ls
                    dir_structexample.tar.bz2
                
            

Try this command on your own to download some other files of your choice on the Internet.

The wget command can do much more than downloading a single file. It can download web pages and full websites. With its -c option, it can also resume downloads that had failed in the middle of the downloading process (you do not have to restart the download from scratch). We are not going to use all of these features in this practical (downloading single files will be enough for what we have to do), but if you want to know more about this command, do not hesitate to read its man page.

Decompressing and Compressing Files

The tar Command

A common way to decompress or compress files on Linux is to use the tar command with a specific lossless data-compression algorithm. Actually, the tar command was initially designed to manipulate archives, which are collections of mutiple files joined together in a single file (often called tarball). The file extension of such a file is usually .tar.

An archive is not necessarily compressed, but the tar command is also able to invoke other programs that create and read compressed archives by using a great variety of algorithms. On Linux, one of the most common used file-compression algorithm is bzip2. The file extension of such a file is usually .tar.bzip2.

In this practical, we are going to see how to list archive contents and how to decompress and compress files using the bzip2 algorithm only. If you want to know how to do other types of archive manipulation or how to use different compression algorithms you should read their man or info pages.

Decompressing bzip2 Files

Let us decompress the file you have downloaded in the previous section: example.tar.bzip2.

                
                    $ cd ~/pw_01
                    $ ls
                    example.tar.bz2
                    $ tar -jxvf example.tar.bz2 
                    example/image/
                    # ...snip...
                    example/README
                    example/AUTHORS
                    example/
                    $ ls -F
                    example/ example.tar.bz2
                
            

Let us examine the tar -jxvf command, which decompresses a bzip2-compressed tarball. There are four options:

The equivalent command using the long options is:

                
                    $ tar --bzip2 --extract --verbose --file example.tar.bz2
                
            

It is easier to remember but much more cumbersome to key in.

Once executed, we can see that a new folder has been created (i.e. example). It contains all the extracted files. If you no longer need it, you can delete the compressed archive.

                
                    $ rm example.tar.bz2
                    $ ls -F
                    example/
                
            

Compressing Directories with bzip2

Go into the example directory you have extracted in the previous section and list its contents. You should find a compress_me directory.

                
                    $ cd ~/pw_01/example
                    $ ls -F
                    AUTHORS  compress_me/  image/  test/  README  txt/
                
            

Browse into the compress_me directory in order to determine its structure (e.g. you can use the tree command). It should look like this:

compress_me Directory Hierarchy

Go back into the example directory.

Now, we are going to compress the compress_me directory and call the output compressed file compress_me.tar.bz2.

                
                    $ cd ~/pw_01/example
                    $ ls -F
                    AUTHORS  compress_me/  image/  test/  README  txt/
                    $ tar -jcvf compress_me.tar.bz2 compress_me/
                    compress_me/
                    compress_me/javascript/
                    compress_me/javascript/lib/
                    compress_me/javascript/lib/string.js
                    compress_me/javascript/lib/array.js
                    compress_me/javascript/main.js
                    compress_me/css/
                    compress_me/css/garden.css
                    compress_me/css/sky.css
                    compress_me/index.html
                    $ ls -F
                    AUTHORS  compress_me/  compress_me.tar.bz2  image/  test/  README  txt/
                
            

Let us examine the tar -jcvf command, which compresses a folder into a bzip2 tarball. There are four options:

The equivalent command using the long options is:

                
                    $ tar --bzip2 --create --verbose --file compress_me.tar.bz2 compress_me/
                
            

Comparing the decompression and compression instructions, we can notice that only the second option has changed. The -x option (--extract) has been replaced by the -c option (--create). For the compression, we also have to specify the directory we want to compress as a parameter.

Once executed, the compress_me.tar.bz2 file has been created. It contains all the contents of the compress_me directory. If you no longer need it, you can delete this directory.

                
                    $ ls -F
                    AUTHORS  compress_me/  compress_me.tar.bz2  image/  test/  README  txt/
                    $ rm -rf compress_me
                    $ ls -F
                    AUTHORS  compress_me.tar.bz2  image/  test/  README  txt/
                
            

Listing Archive Contents

Sometimes, it can be useful to list the contents of an archive in order to know what files it contains. To do so, use the -t and -f options.

                
                    tar -tf compress_me.tar.bz2 
                    compress_me/
                    compress_me/javascript/
                    compress_me/javascript/lib/
                    compress_me/javascript/lib/string.js
                    compress_me/javascript/lib/array.js
                    compress_me/javascript/main.js
                    compress_me/css/
                    compress_me/css/garden.css
                    compress_me/css/sky.css
                    compress_me/index.html
                
            

The equivalent command using the long options is:

                
                    $ tar --list --file compress_me.tar.bz2 
                
            

Printing on the Terminal

The echo Command

The echo command can be used to print some text on the terminal.

                
                    $ echo Hello World!
                    Hello World!
                    $ echo I am David
                    I am David
                
            

It can also be used to print the contents of a variable. For the time being the sole variable you know is the PATH environment variable. The name of a variable must be prefixed with the $ character.

                
                    $ echo $PATH
                    /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
                
            

The cat Command

The cat command can be used to print the contents of text files. Go to the example directory and print the contents of the AUTHORS and README files.

                
                    $ cd ~/pw_01/example
                    $ ls -F
                    AUTHORS  compress_me.tar.bz2 image/  test/  README  txt/
                    $ cat AUTHORS 
                    David Bouchet
                    $ cat README 
                    This directory contains some files and subdirectories
                    that can be used as examples f
                
                
                    or the first practical.
                
            

When the textual contents of a file is too large, the cat command is not really convenient. For example, go to the example/txt directory and print the contents of the long_text.txt file.

                
                    $ cd ~/pw_01/example/txt/
                    $ cat long_text.txt 
                    This f
                
                
                    ile contains 200 lines.
                    Line 2
                    Line 3
                    Line 4
                    Line 5
                    # ...snip...
                    Line 196
                    Line 197
                    Line 198
                    Line 199
                    This is the last line.
                
            

The text is so long that the terminal has scrolled down to the end of the file. So, if you want to see the beginning of the text, you have to scroll everything up. There are different ways to solve this problem.

The more and less Commands

In order to display the contents of the long_text.txt file in a more convenient way, you can use the more command to go through the file screenful by screenful. You can press the space key to continue and the q key to quit. So try the following command:

                
                    $ more long_text.txt 
                
            

But the more command is primitive and in some cases it is better to use the less command, which has many more features. Actually, you already know less because the man command uses less to display its pages.

So, try the following command and navigate throughout the text as if you were browsing a man page.

                
                    $ less long_text.txt 
                
            

The head and tail Commands

Sometimes, we just want to display the beginning or the end of a file. To do so, we can use the head and tail commands, which display the first lines of a file or the last ones. By default, ten lines will be displayed. You can change this number by using the -n option.

                
                    $ head long_text.txt 
                    This f
                
                
                    ile contains 200 lines.
                    Line 2
                    Line 3
                    Line 4
                    Line 5
                    Line 6
                    Line 7
                    Line 8
                    Line 9
                    Line 10
                    $ head -n 3 long_text.txt 
                    This f
                
                
                    ile contains 200 lines.
                    Line 2
                    Line 3
                    $ tail long_text.txt 
                    Line 191
                    Line 192
                    Line 193
                    Line 194
                    Line 195
                    Line 196
                    Line 197
                    Line 198
                    Line 199
                    This is the last line.
                    $ tail -n 2 long_text.txt 
                    Line 199
                    This is the last line.
                
            

Copying, Moving and Renaming

The cp Command

The cp command can be used to copy files. Here are some common examples:

You can copy one file into the same directory. You have to change the filename.

                
                    $ cd ~/pw_01/example/test/cp 
                    $ ls -F
                    dir1/  dir2/  file1  file2  file3
                    $ cp file1 new_file1
                    $ ls -F
                    dir1/  dir2/  file1  file2  file3  new_file1
                
            

You can copy one file into another directory. You can keep the filename.

                
                    $ cp file1 dir1
                    $ ls -F dir1
                    file1
                
            

Or you can change the filename.

                
                    $ cp file2 dir1/new_file2
                    $ ls -F dir1
                    file1  new_file2
                
            

You can copy several files into another directory. You have to keep the filenames.

                
                    $ cp file1 file2 file3 dir2
                    $ ls -F dir2
                    file1  file2  file3
                
            

Thanks to its -r option, the cp command can also be used to copy directories.

                
                    $ cp -r dir1 dir2
                    $ ls -F dir2
                    dir1/  file1  file2  file3
                    $ ls -F dir2/dir1
                    file1  new_file2
                
            

The mv Command

You can use the mv command to move or rename files and directories. Here are some common examples:

You can rename files or directories.

                
                    $ cd ~/pw_01/example/test/cp
                    $ ls -F
                    dir1/  dir2/  file1  file2  file3  new_file1
                    $ mv new_file1 file4
                    $ ls -F
                    dir1/  dir2/  file1  file2  file3  file4
                    $ mv dir1 dir0
                    $ ls -F
                    dir0/  dir2/  file1  file2  file3  file4
                
            

You can move files or directories into other directories.

                
                    $ ls -F dir0
                    file1  new_file2
                    $ mv file2 file3 dir0
                    $ ls -F
                    dir0/  dir2/  file1  file4
                    $ ls -F dir0
                    file1  file2  file3  new_file2
                    $ mv dir2 dir0
                    $ ls -F
                    dir0/  file1  file4
                    $ ls -F dir0
                    dir2/  file1  file2  file3  new_file2
                
            

Wildcard Characters

The wildcard characters allow you to specify several files or directories by using only one parameter.

There are three wildcard characters:

The Star Wildcard

The star wildcard can represent any single character, any strings, or no character at all. In other words, it can represent an undefined number of characters (starting from zero).

For instance, go to the ~/pw_01/example/test/wildcard directory and try the following examples.

                
                    $ cd ~/pw_01/example/test/wildcard
                    $ ls
                    image_1.jpg  photo_1.jpg  picture_1.jpg
                    image_1.png  photo_1.png  picture_1.png
                    image_2.jpg  photo_2.jpg  picture_2.jpg
                    image_2.png  photo_2.png  picture_2.png
                    image_3.jpg  photo_3.jpg  picture_3.jpg
                    image_3.png  photo_3.png  picture_3.png
                    $ ls image*
                    image_1.jpg  image_2.jpg  image_3.jpg
                    image_1.png  image_2.png  image_3.png
                    $ ls *png
                    image_1.png  photo_1.png  picture_1.png
                    image_2.png  photo_2.png  picture_2.png
                    image_3.png  photo_3.png  picture_3.png
                    $ ls *2*
                    image_2.jpg  photo_2.jpg  picture_2.jpg
                    image_2.png  photo_2.png  picture_2.png
                
            

Try different possible patterns on your own in order to get the hang of it.

The Question Mark Wildcard

The question mark wildcard represents any single character. Here are some examples.

                
                    $ ls photo_?.png
                    photo_1.png  photo_2.png  photo_3.png
                    $ ls ?????_1.jpg
                    image_1.jpg  photo_1.jpg
                
            

The Square Brackets Wildcard

The square brackets wildcard represents any characters enclosed in the brackets. These characters can be specified explicitly or by using a range of characters. Try the following examples.

                
                    $ ls image_[13].png
                    image_1.png  image_3.png
                    $ ls image_[1-3].png
                    image_1.png  image_2.png  image_3.png
                
            

Not only can a range of characters represent digits but also letters. For instance:

Combining Wildcards

We can also combine wildcards.

                
                    $ ls ?[mh]*.png
                    image_1.png  image_3.png  photo_2.png
                    image_2.png  photo_1.png  photo_3.png
                
            

The above instruction lists all the files whose second letter is either 'm' or 'h' and end with .png.

Be careful when you use wildcards with the rm command or any commands that affect files. For instance, it is not obvious what files the following instruction is supposed to delete.

                
                    $ rm ?[mh]*[13].png
                
            

Mistakes are bound to happen!

You have to keep in mind that the rm instruction deletes files permanently.

Here is a tip to check your command before executing it: use the echo command just before the full instuction. It will display the command you want to execute.

                
                    $ echo rm ?[mh]*[13].png
                    rm image_1.png image_3.png photo_1.png photo_3.png
                
            

Then, check the command, and if everything is right, remove the echo command.

Now that you know more about wildcards, we can see how to find files and directories in directory hierarchies.

Finding Files and Directories

The find Command

The find command allows you to find files in a directory hierarchy. Its global syntax is as follows:

                
                    $ find [STARTING-POINT] [EXPRESSION]
                
            

A great variety of criteria are available (e.g. search by name, type, size, date, etc.). In this practical we will talk about the search by name only, but have a look at the man page of the find command and try other criteria on your own.

Let us see some examples, but first go to the ~/pw_01/example/test/find directory.

                
                    $ cd ~/pw_01/example/test/find
                    $ tree
                    .
                    ├── dir1
                    │   ├── file1.pdf
                    │   ├── file2.pdf
                    │   ├── file3.pdf
                    │   ├── image1.jpg
                    │   ├── image2.jpg
                    │   ├── image3.jpg
                    │   └── README
                    ├── dir2
                    │   ├── file1.txt
                    │   ├── file2.txt
                    │   ├── file3.txt
                    │   ├── page1
                    │   │   ├── index.html
                    │   │   ├── README
                    │   │   └── style.css
                    │   ├── page2
                    │   │   ├── index.html
                    │   │   ├── README
                    │   │   └── style.css
                    │   ├── picture1.png
                    │   ├── picture2.png
                    │   ├── picture3.png
                    │   └── README
                    └── README
                
            

Now, try the command below.

                
                    $ find . -name README
                    ./dir2/README
                    ./dir2/page1/README
                    ./dir2/page2/README
                    ./README
                    ./dir1/README
                    $ find . -name index.html
                    ./dir2/page1/index.html
                    ./dir2/page2/index.html
                    $ find . -name "*.png"
                    ./dir2/picture1.png
                    ./dir2/picture3.png
                    ./dir2/picture2.png
                    $ find . -name "[ip]*"
                    ./dir2/picture1.png
                    ./dir2/page1
                    ./dir2/page1/index.html
                    ./dir2/page2
                    ./dir2/page2/index.html
                    ./dir2/picture3.png
                    ./dir2/picture2.png
                    ./dir1/image2.jpg
                    ./dir1/image1.jpg
                    ./dir1/image3.jpg
                
            

By looking at the above examples, I think that you can easily understand how this command works. However, you should notice that:

The grep Command

The grep command allows you to search files for text occurrences. Its global syntax is as follows:

                
                    $ grep PATTERN [FILES...]
                
            

Try the following examples:

                
                    $ cd ~/pw_01/example/test/find/dir2
                    $ cat file1.txt
                    The file1 contains the word [Hello].
                    $ cat file2.txt
                    The file2 contains the word [World].
                    $ cat file3.txt
                    The file3 contains the words [Hello] and [World].
                    The file3 contains again the words [Hello] and [World].
                    $ grep Hello file?.txt
                    file1.txt:The file1 contains the word [Hello].
                    file3.txt:The file3 contains the words [Hello] and [World].
                    file3.txt:The file3 contains again the words [Hello] and [World].
                    $ grep World file?.txt
                    file2.txt:The file2 contains the word [World].
                    file3.txt:The file3 contains the words [Hello] and [World].
                    file3.txt:The file3 contains again the words [Hello] and [World].
                
            

When an occurrence is found, the grep command outputs the filename followed by the line where the occurrence has been found.

By default, the command searches files that are in the current directory only. If you want to search files in all subdirectories, you can use the -R option. For instance, the following instruction searches files (in the current directory and all subdirectories) for the occurrence "Hello".

                
                    $ grep -R Hello *
                    file1.txt:The file1 contains the word [Hello].
                    file3.txt:The file3 contains the words [Hello] and [World].
                    file3.txt:The file3 contains again the words [Hello] and [World].
                    page1/index.html:    Hello World!
                
            

Redirecting Outputs

The Stantdard Output Streams

Linux defines one standard input stream and two standard output streams:

The standard input can be used by commands to get some input data.

The standard output is used by commands to output any messages except for error messages.

The standard error is used by commands to output any error messages.

By default, the output streams (sdtout and stderr) are both connected to the terminal.

Redirection

For example:

                
                    $ cd ~/pw_01/example/test/redirection
                    $ ls
                    README
                    $ cat README 
                    This directory is used to t
                
                
                    est redirection.
                    $ cat AUTHORS
                    cat: AUTHORS: No such f
                
                
                    ile or directory
                
            

First, we go to the ~/pw_01/example/test/redirection directory and we can see that there is only one file: README.

Then we print the contents of the README file. Actually, the conceptual mechanism is as follows: as there is no error, the output of the command (i.e. the contents of the file) is sent to the standard output; and by default the standard output sends everything to the terminal. So the contents of the file are printed on the terminal.

cat README

Finally, we try to print the contents of the AUTHORS file, but this file does not exist. So, an error message is printed on the terminal. Actually, the conceptual mechanism is as follows: as there is an error, the output of the command (i.e. the error message) is sent to the standard error; and by default the standard error sends everything to the terminal. So the error message is printed on the terminal.

cat AUTHORS

We can change this default behavior by redirecting the output streams to files or to the standard input of other commands.

The > and 2> Operators

The > operator is used to redirect the standard output to a file.

Let us see this first example:

                
                    $ ls
                    README
                    $ cat README
                    This directory is used to t
                
                
                    est redirection.
                    $ cat README > capture
                    $ ls
                    capture  README
                    $ cat capture 
                    This directory is used to t
                
                
                    est redirection.
                
            

The first instruction redirects the standard output to the capture file. So the contents of the README file are not sent to the terminal but to the capture file. That is the reason why nothing is printed on the terminal.

The capture file is then created (if this file already exists, it is replaced by the new one) and its contents are the output of the cat command (i.e. the contents of the README file).

cat README > capture

Now, let us see this second example:

                
                    $ ls
                    README
                    $ cat AUTHORS > capture
                    cat: AUTHORS: No such f
                
                
                    ile or directory
                    $ ls
                    capture  README
                    $ cat capture
                    $
                
            

The first instruction redirects the standard output to the capture file (as in the previous example). But this time, an error occurs because the AUTHORS file does not exist. So an error message is sent to the standard error, which is still connected to the terminal. Therefore, this message is printed on the terminal.

On the other hand, nothing is sent to the standard output nor to the capture file. The latter is still created but it will be empty.

cat README > capture

In the third example below, we are going to redirect the error standard to a file. The principle is the same as redirecting the standard output, we just have to use the 2> operator instead of the > operator.

                
                    $ cat AUTHORS 2> capture
                    $ ls
                    capture  README
                    $ cat capture
                    cat: AUTHORS: No such f
                
                
                    ile or directory
                    $ cat README 2> capture
                    This directory is used to t
                
                
                    est redirection.
                    $ ls
                    capture  README
                    $ cat capture 
                    $
                
            

It can be useful to use redirection. For instance, we can easily create small text file by using the echo command.

                
                    $ ls
                    capture  README
                    $ echo David Bouchet > AUTHORS
                    $ ls
                    AUTHORS  capture  README
                    $ cat AUTHORS
                    David Bouchet
                
            

But be careful! remember that the destination file is emptied if it already exists.

                
                    $ cat AUTHORS
                    David Bouchet
                    $ echo is the author > AUTHORS
                    $ cat AUTHORS
                    is the author
                
            

And sometimes we just want to get rid of error messages. In this case we can redirect the standard error to the null device. The null device is a special file that ignores everything.

For instance, the following instruction does not print all the directories we are not allowed to read (permission denied). Try this instruction with and without the redirection (and obviously, replace david by your login).

                
                    find / -name david 2> /dev/null
                
            

The >> and 2>> Operators

The >> and 2>> operators are similar to the > and 2> operators respectively.

The difference is that the >> and 2>> operators do not empty the destination file if it already exists. The new text is appended to the existing file.

                
                    $ cat AUTHORS 
                    is the author
                    $ echo David Bouchet > AUTHORS 
                    $ cat AUTHORS 
                    David Bouchet
                    $ echo is the author >> AUTHORS 
                    $ cat AUTHORS 
                    David Bouchet
                    is the author
                
            

The | Operator

The | operator, which has to be read "the pipe operator", is one of the most powerful tools of the command interpreter. It allows you to link multiple commands by connecting the standard output of a command to the standard input of another command.

Pipe Operator

So far, we have not used the standard input of commands. Indeed, we have only used options and parameters. That was enough for what we had to do. Besides, some commands never read their standard input (e.g. ls, cp, mv, rm, etc.)

However, the standard input is another way to pass data to a command.

For instance, let us asume that we want to list the contents of the ~/pw_01/example/test/pipe directory and all of its subdirectories.

                
                    $ cd ~/pw_01/example/test/pipe
                    $ ls -lR
                    .:
                    total 32
                    drwxrwxr-x 2 david david  4096 july  21 22:54 dir1
                    drwxrwxr-x 4 david david  4096 july  23 15:54 dir2
                    drwxrwxr-x 2 david david 12288 july   6 14:55 dir3
                    -rw-rw-r-- 1 david david    62 july   6 14:37 README
                    # ...snip...
                    -rw-rw-r-- 1 david david     0 july   6 14:55 file_9e.txt
                    -rw-rw-r-- 1 david david     0 july   6 14:55 file_9.jpg
                    -rw-rw-r-- 1 david david     0 july   6 14:55 file_9.png
                
            

This listing is so long that we want to print one screenful at a time. By using options, parameters and redirection, it could be done in this way:

                
                    $ ls -lR > temp_list
                    $ more temp_list 
                    $ rm temp_list 
                
            

We had to create and delete a temporary file. It works but it is cumbersome. On the other hand, by using the pipe operator, it is simpler to do it this way:

                
                    ls -lR | more
                
            

First the ls -lR is executed. Then, its output is sent to the standard input of the more command. Finally the more command prints the contents of its standard input screenful by screenful. No temporary files have been created.

Now that you know how to link commands together, let us see a more complex example and let us learn some new commands as well.

Let us assume that we want to list, in alphabetical order, all the README files of the example directory (and its subdirectories) that do not contain the "Author" occurrence.

First, let us do the opposite, that is, listing all the files that contain the "Author" occurrence. To do so, we are going to use the grep command we have already seen in a previous section.

                
                    $ cd ~/pw_01/example/test/pipe
                    $ grep -R Author
                    test/pipe/dir2/README:Author: David Bouchet
                    test/pipe/README:Author: David Bouchet
                    test/pipe/dir1/README:Author: David Bouchet
                
            

Actually, we just want the paths of the file. So we can use the -l option of the grep command.

                
                    $ grep -Rl Author
                    test/pipe/dir2/README
                    test/pipe/README
                    test/pipe/dir1/README
                
            

Now, if we replace the -l option by the -L option, the grep command will list all the files that do not contain the "Author" occurrence.

                
                    $ grep -RL Author
                    AUTHORS
                    README
                    test/redirection/README
                    test/wildcard/photo_2.png
                    # ...snip...
                    compress_me/css/sky.css
                    compress_me/index.html
                    txt/long_text.txt
                
            

The problem is that we have all the files that do not contain the "Author" occurrence, not just the README files. So, among all of these files we should print only the lines that contain the "README" occurrence. To do this, we can send this list to another grep command by using the pipe operator.

                
                    $ grep -RL Author | grep README
                    README
                    test/redirection/README
                    test/pipe/dir2/page1/README
                    test/pipe/dir2/page2/README
                    test/find/dir2/README
                    test/find/dir2/page1/README
                    test/find/dir2/page2/README
                    test/find/README
                    test/find/dir1/README
                
            

We have all the README files that do not contain the "Author" occurrence, but this list is not sorted. To sort it we can link the sort command (man sort).

                
                    $ grep -RL Author | grep README | sort
                    README
                    test/find/dir1/README
                    test/find/dir2/page1/README
                    test/find/dir2/page2/README
                    test/find/dir2/README
                    test/find/README
                    test/pipe/dir2/page1/README
                    test/pipe/dir2/page2/README
                    test/redirection/README
                
            

Now, let us assume that we do not want the filenames but only the number of README files that do not contain the "Author" occurrence. To do so, we can use the wc command (man wc) with its -l option that counts the number of lines.

                
                    $ grep -RL Author | grep README | wc -l
                    9
                
            

So, nine README files do not contain the "Author" occurrence.

Exercise

Here are some questions to practice the stock of knowledge you have just acquired. Try to answer them on your own before looking at the keys. When it comes to computer programming, there are often different ways to reach a goal or to be effective. You may use commands that are different from those in the keys.

If you do not know an answer, look first at the man pages before anything else. The most important thing is that you answer each question on your own (never mind whether you get it right or wrong) and then read the key carefully.

Go to the ~/example/test/pipe directory.

Key
                        
                            $ cd ~/example/test/pipe
                        
                    

List the contents of this directory by appending an indactor to entries. The expected result is as follows:

                
                    dir1/  dir2/  dir3/  README
                
            

Key
                        
                            $ ls -F
                        
                    

List the contents of this directory in a tree-like format. Find the command and then the option in order to list the directories only (not the files). The expected result is as follows:

                
                    .
                    ├── dir1
                    ├── dir2
                    │   ├── page1
                    │   └── page2
                    └── dir3
                
            

Key
                        
                            $ tree -d
                        
                    

Print the contents of the README file. The expected result is as follows:

                
                    This directory contains some files to t
                
                
                    est the pipe operator.
                    Author: David Bouchet
                
            

Key
                        
                        $ cat README
                        
                    

Rename the dir3 directory as test_dir.

Key
                        
                        $ mv dir3 test_dir
                        
                    

Go to the test_dir directory.

Key
                        
                        $ cd test_dir
                        
                    

List all the files that end with .tar.bz2. The expected result is as follows:

                
                    doc.tar.bz2
                
            

Key
                        
                            $ ls *.tar.bz2
                        
                    

List the contents of the doc.tar.bz2 archive. The expected result is as follows:

                
                    doc_00.pdf
                    doc_01.pdf
                    doc_02.pdf
                
            

Key
                        
                            $ tar -tf doc.tar.bz2
                        
                    

Decompress the contents of the doc.tar.bz2 archive (enable the verbose mode). The expected result is as follows:

                
                    doc_00.pdf
                    doc_01.pdf
                    doc_02.pdf
                
            

Key
                        
                            $ tar -jxvf doc.tar.bz2
                        
                    

Remove the doc.tar.bz2 file.

Key
                        
                            rm doc.tar.bz2
                        
                    

List all the files that have an underscore followed by two digits in a row. The expected result is as follows:

                
                    doc_00.pdf    file_11.json  file_13.png   file_16.jpeg  file_18.json
                    doc_01.pdf    file_11.png   file_14.jpeg  file_16.json  file_18.png
                    doc_02.pdf    file_12.jpeg  file_14.json  file_16.png   file_19.jpeg
                    file_10.jpeg  file_12.json  file_14.png   file_17.jpeg  file_19.json
                    file_10.json  file_12.png   file_15.jpeg  file_17.json  file_19.png
                    file_10.png   file_13.jpeg  file_15.json  file_17.png
                    file_11.jpeg  file_13.json  file_15.png   file_18.jpeg
                
            

Key
                        
                            $ ls *_[0-9][0-9]*
                        
                    

List all the files that have a four-letter extension. The expected result is as follows:

                
                    file_10.jpeg  file_12.jpeg  file_14.jpeg  file_16.jpeg  file_18.jpeg
                    file_10.json  file_12.json  file_14.json  file_16.json  file_18.json
                    file_11.jpeg  file_13.jpeg  file_15.jpeg  file_17.jpeg  file_19.jpeg
                    file_11.json  file_13.json  file_15.json  file_17.json  file_19.json
                
            

Key
                        
                            $ ls *.????
                        
                    

By using the wildcards, print the instruction that removes all files that have the small letter 'a' just before the extension. The expected result is as follows:

                
                    rm file_0a.png file_0a.txt file_1a.png file_1a.txt
                
            

Key
                        
                            $ echo rm *a.*
                        
                    

Now, remove all of these files.

Key
                        
                            $ rm *a.*
                        
                    

Remove all files that have the letter either 'b', 'c', 'd' or 'e' just before the extension.

Key
                        
                            $ rm *[bcde].*
                        
                    

Go back to the parent directory.

Key
                        
                            $ cd ..
                        
                    

From the current directory (~/example/test/pipe), compress the dir2/page2 directory (enable the verbose mode). The compressed file must be called page2.tar.bz2. The expected result is as follows:

                
                    dir2/page2/
                    dir2/page2/README
                    dir2/page2/style.css
                    dir2/page2/index.html
                
            

Key
                        
                            $ tar -jcvf page2.tar.bz2 dir2/page2
                        
                    

From the current directory (~/example/test/pipe), remove the dir2/page2 directory (enable the verbose mode). The expected result is as follows:

                
                    removed 'dir2/page2/README'
                    removed 'dir2/page2/style.css'
                    removed 'dir2/page2/index.html'
                    removed directory 'dir2/page2'
                
            

Key
                        
                            $ rm -rfv page2/dir2
                        
                    

From the current directory (~/example/test/pipe), copy the dir2/page1 directory to the dir1 directory.

Key
                        
                            cp -r dir2/page1 dir1
                        
                    

From the current directory (~/example/test/pipe), move the page2.tar.bz2 archive to the ~/example directory (enable the verbose mode and use relative path). The expected result is as follows:

                
                    'page2.tar.bz2' -> '../../page2.tar.bz2'
                
            

Key
                        
                            $ mv -v page2.tar.bz2 ../..
                        
                    

Create an AUTHORS file that contains only the last line of the README file.

Key
                        
                            $ tail -n 1 README > AUTHORS
                        
                    

Append the first line of the README file to the AUTHORS file.

Key
                        
                            $ head -n 1 README >> AUTHORS
                        
                    

Create the hello file that contains the line "Hello,".

Key
                        
                            $ echo Hello, > hello
                        
                    

Create the world file that contains the line "World!".

Key
                        
                            $ echo World! > world
                        
                    

Concatenate the hello and world files. The output file should be called hello_world.

Key
                        
                            $ cat hello world > hello_world
                        
                    

Append the dir2/file1.txt and the dir2/file2.txt files to the hello_world file.

Key
                        
                            $ cat dir2/file[12].txt >> hello_world
                        
                    

From the current directory (~/example/test/pipe) and all of its subdirectories, print the number of PNG files, that is, the number of filenames that end with ".png". The expected result is as follows:

                
                    23
                
            

Key
                        
                            $ find . -name "*.png" | wc -l               
                        
                    

From the current directory (~/example/test/pipe) and all of its subdirectories, list all the JPEG files, that is, the filenames that end with ".jpg" or ".jpeg", and sort them in alphabetical order. Use the -o option of the find command. The expected result is as follows:

                
                    ./dir1/image1.jpg
                    ./dir1/image2.jpg
                    ./dir1/image3.jpg
                    ./test_dir/file_0.jpg
                    ./test_dir/file_10.jpeg
                    ./test_dir/file_11.jpeg
                    ./test_dir/file_12.jpeg
                    ./test_dir/file_13.jpeg
                    ./test_dir/file_14.jpeg
                    ./test_dir/file_15.jpeg
                    ./test_dir/file_16.jpeg
                    ./test_dir/file_17.jpeg
                    ./test_dir/file_18.jpeg
                    ./test_dir/file_19.jpeg
                    ./test_dir/file_1.jpg
                    ./test_dir/file_2.jpg
                    ./test_dir/file_3.jpg
                    ./test_dir/file_4.jpg
                    ./test_dir/file_5.jpg
                    ./test_dir/file_6.jpg
                    ./test_dir/file_7.jpg
                    ./test_dir/file_8.jpg
                    ./test_dir/file_9.jpg
                
            

Key
                        
                            $ find . -name "*.jpg" -o -name "*.jpeg" | sort
                        
                    

From the current directory (~/example/test/pipe) and all of its subdirectories, list all the files that contain the "firstName" occurrence. You have to print the paths of the files and the lines that contain this occurrence. The expected result is as follows:

                
                    test_dir/file_10.json:  "firstName": "John",
                    test_dir/file_12.json:  "firstName": "Jack",
                    test_dir/file_11.json:  "firstName": "Jim",
                    test_dir/file_13.json:  "firstName": "Marc",
                    test_dir/file_14.json:  "firstName": "John",
                
            

Key
                        
                            $ grep -R firstName *
                        
                    

From the current directory (~/example/test/pipe) and all of its subdirectories, list all the files that contain the "firstName" occurrence. You have to print the paths of the files only and they must be sorted in alphabetical order. The expected result is as follows:

                
                    test_dir/file_10.json
                    test_dir/file_11.json
                    test_dir/file_12.json
                    test_dir/file_13.json
                    test_dir/file_14.json
                
            

Key
                        
                            $ grep -Rl firstName * | sort
                        
                    

Finally, we want to print the long listing format of the files listed in the previous question. To do so, you have to send the output of the previous question to the ls -l command, but the problem is that the ls command does not read its standard input. So the following instruction will not work:

                
                    [any command] | ls -l
                
            

Therefore, you have to use the xargs command, which turns each line of its standard input into parameters for another command. Read the man page of this command (man xargs) and answer the question. Note also that the sort command is useless here because the ls command sorts its result by default. The expected result is as follows:

                
                    -rw-rw-r-- 1 david david 219 août   9 21:39 test_dir/file_10.json
                    -rw-rw-r-- 1 david david 217 août   9 21:39 test_dir/file_11.json
                    -rw-rw-r-- 1 david david 220 août   9 21:38 test_dir/file_12.json
                    -rw-rw-r-- 1 david david 220 août   9 21:39 test_dir/file_13.json
                    -rw-rw-r-- 1 david david 220 août   9 21:38 test_dir/file_14.json
                
            

Key
                        
                            $ grep -Rl firstName * | xargs ls -l
                        
                    

Submission

For this practical you are just going to create an AUTHORS file in your repository.

Due Date

By Friday 27 September 2019 23:42

Directory Hierarchy

Your git repository must contain the following files and directories:

The AUTHORS file must contain the following information.

AUTHORS
                
                    First Name
                    Family Name
                    Login
                    Email Address
                
            

The last character of your AUTHORS file must be a newline character.

For instance:

AUTHORS
                
                    $ cat AUTHORS 
                    John
                    Smith
                    john.smith
                    john.smith@epita.fr
                    $ # Command prompt ready for the next command...
                
            

To do so, you can type the following instructions (replace john.smith by you own login).

                
                    $ git clone git@git.cri.epita.net:p/2023-s3-tp/tp01-john.smith
                    $ cd tp01-john.smith/
                    $ mkdir pw_01_cli
                    $ # Create the "pw_01_cli/AUTHORS" file.
                    $ git add pw_01_cli/AUTHORS
                    $ git commit -s -m "First commit"
                    $ git push
                
            

For further information, see https://doc.cri.epita.fr/git/.