Difference between revisions of "Linux1"

From SourceWiki
Jump to navigation Jump to search
 
(31 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
[[category:Pragmatic Programming]]
 
[[category:Pragmatic Programming]]
 
'''A not so short introduction to the Linux command line'''
 
'''A not so short introduction to the Linux command line'''
 
 
=Introduction=
 
=Introduction=
Linux (sometimes called GNU/Linux) is an operating system, i.e. the middle man between you and the computer hardware. Therefore Linux is a full replacement for Microsoft Windows or Mac OS X. Linux comes with a range of graphical user interfaces where you can use the mouse to interact with the computer as you are used to. However, in most setup within the University (and particularly for the HPC facilities), you will not have access to this graphical user interface. Instead, you will be using Linux at the command line. Therefore this practical introduces some of the basic commands that you must know in order to carry out your research in a Linux based environment.
+
Linux (sometimes called GNU/Linux) is an operating system, which you can think of as the middle man between you and the computer hardware. Linux is a full replacement for Microsoft Windows or MacOS X. Linux comes with a range of graphical user interfaces where you can use the mouse to interact with the computer as you would under Windows or OS X. However, it is common to use the Linux '''command line''', for example when you are using the University's HPC facilities. Therefore this practical introduces some of the basic tools that you will find useful on the Linux command line.
  
The user interface on a computer is technically called a [http://en.wikipedia.org/wiki/Shell_(computing) shell]. When using Linux at the command line, most often the [http://en.wikipedia.org/wiki/Bash <tt>bash</tt> shell] is used. There are a range of shells (csh, zsh ...) for Linux with slightly different syntaxes. The <tt>bash</tt> shell is often the default and we will use <tt>bash</tt> syntax in this practical.
+
The Linux command line interface is often referred to as a [http://en.wikipedia.org/wiki/Shell_(computing) shell]. The [http://en.wikipedia.org/wiki/Bash <tt>bash</tt> shell] is common. There are a range of shells (csh, zsh ...) for Linux which use slightly different syntax. Since the <tt>bash</tt> shell is often the default we will use <tt>bash</tt> it for this practical.
  
 
=Logging in=
 
=Logging in=
Login is performed via a bit of protocol called '''secure shell''' or <tt>ssh</tt>. There is a client for Microsoft Windows which should already be installed on your computer. If not, contact a computer officer. To login select the machine that you want to log into, for instance <tt>dylan.ggy.bris.ac.uk</tt> or <tt>bluecrystal.bris.ac.uk</tt> and then enter your username and password. the system will log you in and you will directly be taken to your home directory.
+
Login is performed using the '''secure shell''' or <tt>ssh</tt>. There is a client for Microsoft Windows which should already be installed on your computer. (If not, contact a computer officer.To login, type the '''full name''' of the machine that you want to log into, for instance <tt>dylan.ggy.bris.ac.uk</tt> or <tt>bluecrystal.bris.ac.uk</tt> and then enter your username and password. The system will log you in and you will be taken to your home directory.
  
Although you used your password to login, it is important to note that there are other login options available. For instance, you can configure ssh to log you in automatically using authentication keys instead of using a password. This is very interesting when you want scripts to run automatically and perform operations on the network. This is really beyond the scope of this practical.
+
On this occasion, you used your password to login.  There are other login options available, however. For instance, you can configure ssh to log you in automatically using authentication key-pairs, rather then typing a password. This appraoch can be very useful for (a) the lazy person!; and (b) if you would like to be able to run scripts automatically on a remote machine and perform operations on the network. This is really beyond the scope of this practical.
  
 
= Getting the content for this practical =  
 
= Getting the content for this practical =  
Now that you are logged in a Linux system, it is time to get the practical content. The necessary files for this practical are hosted in a version control system. To obtain them, just type the following command:
+
Now that you are logged in a Linux system, it is time to get the practical content. The files for this practical are hosted in a version control system. To get them, just cut & paste the following onto your command line:
 
<pre>
 
<pre>
$ svn export http://source.ggy.bris.ac.uk/subversion-open/linux1/trunk linux1
+
$ svn export https://svn.ggy.bris.ac.uk/subversion-open/linux1/trunk linux1
 
</pre>
 
</pre>
  
The dollar sign "$" is called the '''shell prompt'''. You do not need to type it in. It is merely there to show that the shell is waiting for a command. When a line starts without a dollar, it means that it is an output from the shell. This is a very common convention that will be used throughout this practical.
+
Note that we are using the dollar sign "$" here to represent your '''shell prompt'''. You do not need to type it in. It is merely there to show that the shell is waiting for a command. We will represent outputs from the shell as lines without a "$".
  
This will fetch all necessary files and put them in a folder called <tt>linux1/</tt>. Ignore the cryptic syntax so far, an introduction to version control using subversion (svn) will be given later on.
+
The svn command will fetch all necessary files and put them in a folder called <tt>linux1/</tt>. You can ignore the cryptic syntax.  We will look at version control using subversion (svn) in a later practical.
  
 
=Navigation=
 
=Navigation=
Line 36: Line 35:
 
</pre>
 
</pre>
  
Obviously, the full path will depend on where your home directory is located. The absolute path starts at <tt>/</tt> called the '''root''' and then lists the branches of the directory tree leading to the current directory. Note that the directories are separated by slashes "<tt>/</tt>" and not backslashes as used by DOS in Microsoft Windows.
+
Obviously, the full path will depend on where your home directory is located. The absolute path starts at <tt>/</tt> called the '''root''' and then lists the branches of the directory tree leading to the current directory. Note that the directories are separated by slashes "<tt>/</tt>" and not backslashes as used by Microsoft DOS.
  
 
The <tt>cd</tt> command also accepts absolute paths. For instance:
 
The <tt>cd</tt> command also accepts absolute paths. For instance:
Line 48: Line 47:
 
</pre>
 
</pre>
  
To help navigation, every directory on a Linux system contains two shortcuts:
+
As an aid to navigation, every directory on a Linux system contains two shortcuts:
 
* <tt>.</tt> (yes, one single dot!) links to itself
 
* <tt>.</tt> (yes, one single dot!) links to itself
 
* <tt>..</tt> (two consecutive dots) links to its parent
 
* <tt>..</tt> (two consecutive dots) links to its parent
  
The <tt>..</tt> shortcut especially is important as it allows you to go up one directory without using a full path:
+
The <tt>..</tt> shortcut is especially useful as it allows you to go up one directory without using a full path:
 
<pre>
 
<pre>
 
$ pwd
 
$ pwd
Line 74: Line 73:
 
</pre>
 
</pre>
  
Furthermore, you can use <tt>cd</tt> with the argument "<tt>-</tt>" to simply go back where you were before, a it like the back button on a web browser. It only worksone level though so using it twice sends you back to where you started. For instance:
+
Furthermore, you can use <tt>cd</tt> with the argument "<tt>-</tt>" to simply go back where you were before, a bit like the back button on a web browser. It only remembers one 'level', however, so using it twice sends you back to where you started. For instance:
  
 
<pre>
 
<pre>
Line 144: Line 143:
 
Obviously you can ignore these files most of the time. This is why they are hidden.
 
Obviously you can ignore these files most of the time. This is why they are hidden.
  
When you want more detail about the files and directories, use the "-l" option to use the <b>l</b>ong listing format. Let's go ack n the <tt>linux1</tt> directory and try it:
+
When you want more detail about the files and directories, use the "-l" option to use the <b>l</b>ong listing format. Let's go back to the <tt>linux1</tt> directory and try it:
  
 
<pre>
 
<pre>
Line 156: Line 155:
 
</pre>
 
</pre>
  
The detailed output for each item starts with a big block which contains the file or directory permissions, the number of directories inside the directory, the file/directory owner, the file/directory group, the file size, the last modified date and time and eventually the file/directory name. The output might slightly vary between systems. Permissions, ownership and groups are explained later on. Note that the number of directories is one for files and at least 2 for directories (because dot and dot dot are always present).
+
The detailed output for each item starts with a big block which contains the file or directory '''permissions''', the number of links to the entity, the file/directory owner, the file/directory group, the file size, the last modified date and time and eventually the file/directory name. The output might slightly vary between systems. Permissions, ownership and groups are explained later on. Note that the number of links is one for files and at least 2 for directories (because dot and dot dot are always present).
  
 
The file size is given in bytes. To make it easier to read, it is possible to use the -h option (for <b>h</b>uman readable):
 
The file size is given in bytes. To make it easier to read, it is possible to use the -h option (for <b>h</b>uman readable):
Line 168: Line 167:
 
</pre>
 
</pre>
  
Only the directory size has been change and you can see the K has been used for KiloBytes (KB). There is a more useful example in the folder2 directory.
+
Only the directory size is changed and you can see '''K''' has been used for KiloBytes (KB). There is a more useful example in the folder2 directory.
  
 
<pre>
 
<pre>
Line 184: Line 183:
 
Note that 104,448 bytes is "rounded" to 102KB. This is not an error, it is simply that 1KB is 1,024 Bytes and not 1,000...
 
Note that 104,448 bytes is "rounded" to 102KB. This is not an error, it is simply that 1KB is 1,024 Bytes and not 1,000...
  
When several arguments are given to <tt>ls</tt>, they can be lumped together, therefore "<tt>ls -l -h</tt>" is identical to <tt>ls -lh</tt>".
+
When several arguments are given to <tt>ls</tt>, they can be grouped together, therefore "<tt>ls -l -h</tt>" is identical to <tt>ls -lh</tt>".
  
There is a very useful combination of options for <tt>ls</tt>. To find the last file modified in a directory, use "<tt>ls -rtl</tt>". The "t" option use chronological order instead of alphabetical, "l" for long output and "r" for reverse, you end up with the youngest file or directory at the bottom of the list.
+
There is a very useful combination of options for <tt>ls</tt>. To find the last file modified in a directory, use "<tt>ls -ltr</tt>". The "t" option use chronological order instead of alphabetical, "l" for long output and "r" for reverse, you end up with the youngest file or directory at the bottom of the list.
  
 
<pre>
 
<pre>
$ ls -rtl
+
$ ls -ltr
 
total 32
 
total 32
 
-rw-r--r-- 1 jp jp  73 2008-02-04 12:46 file1
 
-rw-r--r-- 1 jp jp  73 2008-02-04 12:46 file1
Line 197: Line 196:
 
</pre>
 
</pre>
  
In the example above, the directory <tt>folder2</tt> contains the latest files.
+
In the above example, the directory <tt>folder2</tt> contains the latest files.
  
 
= Getting help =  
 
= Getting help =  
As you have seen with the <tt>ls</tt> example, some commands have many functions and it is easy to lose track. Luckily it is usually quite easy to get help for most commands. Two helps systems are available with the commands <tt>man</tt> and <tt>info</tt>. Just use either, followed by the name of the command you want help for. For instance to find help about the <tt>ls</tt> command using the "<tt>man</tt>" system, type::
+
As you have seen with the <tt>ls</tt> example, some commands have many functions and it is easy to lose track. Luckily it is usually quite easy to get help for most commands. Two ways to get help are available.  You can use <tt>man</tt> (short for '''man'''ual) and <tt>info</tt>. To use either, just add on the name of the command your interested in (or bemused by!). For example, to find help about the <tt>ls</tt> command using the "<tt>man</tt>" pages, type::
 
<pre>
 
<pre>
 
man ls
 
man ls
Line 215: Line 214:
 
Use the arrows to navigate up and down line by line, the space bar to go down a whole screen height, and enter q to quit. You can use enter on a any link identified by an asterisk to be taken directly to that content. You can also enter "<tt>/word</tt>" to look for the string "word".
 
Use the arrows to navigate up and down line by line, the space bar to go down a whole screen height, and enter q to quit. You can use enter on a any link identified by an asterisk to be taken directly to that content. You can also enter "<tt>/word</tt>" to look for the string "word".
  
The <tt>info</tt> system is newer than <tt>man</tt> and probably its successor but <tt>man</tt> is still very popular. Now, to get more information about them, you could try "<tt>info man</tt>" for instance...
+
The <tt>info</tt> system is newer than the <tt>man</tt> pages and probably its successor, but <tt>man</tt> remains very popular. Now, to get more information about them, you could try "<tt>info man</tt>" for instance...
  
 
= Unix permissions =
 
= Unix permissions =
  
We saw earlier on that the long output of <tt>ls</tt> contains a block about file permissions. To understand this block, you need to know that users on a Linux system are organised in groups. Each file or directory has an owner who is a user and also belongs to a group. For each file and directory on a Linux system, it is possible to give permissions or access rights to the owner of a file, to the group to which the file belongs to ... and also to the other users of the system (i.e. those who don't own the file or belong to the group).
+
We saw earlier on that the long output of <tt>ls</tt> contains a block about file permissions. To understand this block, you need to know that users on a Linux system are organised into '''groups'''. Each file or directory has an owner, who is a user, and that user also belongs to a group. For each file and directory on a Linux system, it is possible to give permissions or access rights to the owner of a file, to the group to which the file belongs to ... and also to the other users of the system, i.e. those who don't own the file or belong to the group.
  
The permissions that can be given are called rad access (r), write access (w) and execute access (x). The permissions are usually listed together so a file with "rwx" for a user means that the user has all right on that file. "r-x" means the user can read and execute but not write/modify. "r--" means read only access. Note that in order to be able to enter a directory, a user need both read and execute access ("r-x").
+
The permissions include read access (r), write access (w) and execute access (x). The permissions are usually listed together so a file with "rwx" for a user means that the user has all rights on that file. "r-x" means the user can read and execute but not write/modify. "r--" means read-only access. Note that in order to be able to enter a directory (e.g. using cd), a user needs both read and execute access ("r-x").
  
 
The output of <tt>ls -l</tt> is now easier to understand, for instance:
 
The output of <tt>ls -l</tt> is now easier to understand, for instance:
Line 234: Line 233:
 
</pre>
 
</pre>
  
All files in this directory belong to the user <tt>jp</tt> and also the group <tt>jp</tt>. The user <tt>jp</tt> has read and write permissions on <tt>file1</tt> and <tt>file2</tt>, users belonging to the group <tt>jp</tt> and other users only have read access. For the two directories <tt>folder1</tt> and <tt>folder2</tt>, only the user <tt>jp</tt> can modify them. However, all users can enter them. Note that being able to modify a directory includes the right to add or remove files to the directory.
+
All files in this directory belong to the user <tt>jp</tt> and also the group <tt>jp</tt>. The user <tt>jp</tt> has read and write permissions for <tt>file1</tt> and <tt>file2</tt>, users belonging to the group <tt>jp</tt> and other users have only read access. For the two directories <tt>folder1</tt> and <tt>folder2</tt>, only the user <tt>jp</tt> can modify them. However, all users can enter those directories. Note that being able to modify a directory includes the right to add or remove files.
  
Often, the permissions are simplified (!) using a binary counting system where "--x" is 1, "-w-" is 2 and "r--" is 4. Therefore, "rwx" is 7 and "r-x" is 5. Using this system for the user, group and other users, the permissions of a file or directory can be sumarised as a 3 digit number. For instance, 755 can be used instead of "rwxr-xr-x" or 644 instead of "rw-r--r--". These two are the typical permissions for a directory and file respectively.  
+
Often, the permissions are simplified(!) using a binary counting system where "--x" is 1, "-w-" is 2 and "r--" is 4. Therefore, "rwx" is 7 and "r-x" is 5. Using this system for the user, group and other users, the permissions of a file or directory can be sumarised as a 3 digit number. For example, 755 can be used instead of "rwxr-xr-x" or 644 instead of "rw-r--r--". These two are the typical permissions for a directory and file respectively.  
  
Note that on each line of the <tt>ls -l</tt> output, the first character marks the type of the entry, it is a dash "-" for a regular file, a "d" for a directory and it could be a "l" for a link. We will introduce links later.
+
Note that on each line of the <tt>ls -l</tt> output, the first character marks the '''type''' of the entity:  A dash "-" for a regular file, a "d" for a directory and you would see an "l" for a link. We will introduce links later.
  
The permissions on a file or directory can be changes with the command <tt>chmod</tt>.
+
The permissions on a file or directory can be changed using the command <tt>chmod</tt> (short for '''ch'''ange '''mod'''alities).
  
 
<pre>
 
<pre>
Line 265: Line 264:
 
</pre>
 
</pre>
  
Similarly, use the letter u for the user/owner, g for the group and o for the others. be careful, o is for the others, '''not''' the owner. <tt>chmod</tt> can also be used with a 3 digit permission summary too.
+
Use the letter '''u''' for the user/owner, '''g''' for the group and '''o''' for the others. Be careful, o is for the others, '''not''' the owner. <tt>chmod</tt> can also be used with a 3 digit permission summary too.
  
 
=Creating/deleting/modifying=
 
=Creating/deleting/modifying=
Line 271: Line 270:
 
Now that we know about permissions, it is time to actually create a some content.
 
Now that we know about permissions, it is time to actually create a some content.
  
To create a directory, use the command <tt>mkdir</tt>:
+
To create a directory, use the command <tt>mkdir</tt> (short for '''m'''a'''k'''e '''dir'''ectory):
  
 
<pre>
 
<pre>
Line 315: Line 314:
  
 
==Deleting files or directories==
 
==Deleting files or directories==
To delete a file use the command <tt>rm</tt> (<b>r</b>emove). For instance:
+
To delete a file use the command <tt>rm</tt> (<b>r</b>e<b>m</b>ove). For instance:
  
 
<pre>
 
<pre>
Line 324: Line 323:
 
</pre>
 
</pre>
  
Attention, there is no trash can when you use <tt>rm</tt>. When you delete something, it's gone for good! The option "-i" can be used to force <ttrm</tt> to ask for aconfirmation before actually deleting anything:
+
'''Attention''', there is no trash can when you use <tt>rm</tt>. When you delete something, it's gone for good! The option "-i" can be used to force <ttrm</tt> to ask for confirmation prior to actually deleting anything:
  
 
<pre>
 
<pre>
Line 349: Line 348:
 
</pre>
 
</pre>
  
Note that if your system default to the "-i" option, deleting a directory is time conduming as a confirmation will be asked for each file/directory. It is therefore advisable to use option "-f" for <b>f</b>orce.
+
Note that if your system default to the "-i" option, deleting a directory is time conduming as a confirmation will be asked for each file/directory. It is therefore advisable to use option "-f" (for <b>f</b>orce).
  
 
==Editing files==
 
==Editing files==
To edit a file, you can simply use some of the files editors provided by the Linux machine. It varies with the system but often, the following editors are present: <tt>vim</tt>, <tt>emacs</tt>, <tt>nano</tt> and <tt>gedit</tt>. The first three will open in your secure shell window. <tt>vim</tt> and <tt>emacs</tt> are a bit daunting at first. To exit <tt>vim</tt> type "Escape" and then enter ":q". To exit <tt>emacs</tt> type control-x and then ctrl-c. <tt>nano</tt> is easier to use to start with.  
+
Linux can provide quite an array of different editors for modifying files. Precisely which tools are installed varies from system to system, but often the following editors are present: <tt>vim</tt>, <tt>emacs</tt>, <tt>nano</tt> and <tt>gedit</tt>. The first three will open in your secure shell window. <tt>vim</tt> and <tt>emacs</tt> can be a bit daunting at first. To exit <tt>vim</tt> type "Escape" and then enter ":q". To exit <tt>emacs</tt> type control-x and then ctrl-c. <tt>nano</tt> is easier to use to start with.  
  
<tt>gedit</tt> is actually a graphical text editor, i.e. it opens in its own window and you get a set of buttons/menus to save, copy, cut, paste etc... However, because you are not physically connected to the display of the an extra operation is required to allow that Linux machine to open a window on the screen of your PC. This is done with a piece of software called Exceed. You need to start Exceed from the start menu and then select "Edit->Settings->Profile" and tick the box "Tunnel X11 connections" under "Settings->Tunneling". The configuration only has to be done only once. However, Exceed has to be running for this to work.
+
<tt>gedit</tt> is actually a graphical text editor, i.e. it opens in its own window and you get a set of buttons/menus to save, copy, cut, paste etc... However, because you are not physically connected to the display of the Linux machine (i.e. your screen is not attached to directly the server you are logged into), an extra operation is required to allow that Linux machine to open a window on the screen of your PC. This is done with a piece of software called Exceed. You need to start Exceed from the start menu.  You must also change some settings on your <tt>SSH</tt> connection.  Select "Edit->Settings->Profile Settings" and tick the box "Tunnel X11 connections" under "Connections->Tunneling", and save your settings. This configuration only has to be done only once. However, Exceed has to be running for this to work (and you may need to restart your <tt>SSH</tt> connection).
  
On important thing to remember is that when you are connected to one of the local Linux machines, your files are also accessible directly on your computer. So you can edit them in MS Windows, using your favourite text editor and you don't really need to use any of the Linux file editors.
+
(Note that your files on dylan are also available (via something called Samba) through your PC, so you can edit them in MS Windows, using your favourite text editor and you don't really need to use any of the Linux file editors.)
  
 
==Copying directories and files==
 
==Copying directories and files==
Line 369: Line 368:
  
 
=Displaying content=
 
=Displaying content=
Sometimes, you want to look a the content of a file but don't need to edit it. It is possible to display file content on Linux by a range of methods. The command <tt>cat</tt> simply displays the content of a file to the screen. It is quick but not appropriate for large files. <tt>head</tt> and <tt>tail</tt> can be used to look respectively at the top or the end of a file. Both commands usually default to the first or last 10 lines of the file althpugh this can be modified with an optional argument. These commands are very convenient when you know that what you are looking for is at the beginning or at the end of your file.
+
Sometimes, you want to look a the content of a file but don't need to edit it. It is possible to display file content on Linux by a range of methods. The command <tt>cat</tt> (short for con'''cat'''enate) simply displays the content of a file to the screen, if it is given a single file. It is quick but not very convenient for large files! <tt>head</tt> and <tt>tail</tt> can be used to look respectively at the top or bottom of a file. Both commands usually default to 10 lines of the file, althpugh this can be modified with an optional argument (-n). These commands are very convenient if you would like to look at a file header or the very last lines written to a file (both quite common situations).
 +
 
 
<pre>
 
<pre>
 
$ head file2
 
$ head file2
Line 394: Line 394:
 
</pre>
 
</pre>
  
When you want access to the whole file, the commands <tt>more</tt> or <tt>less</tt> can be used. They both work in a similar way. As for <tt>info</tt>, <tt>less</tt> is a bit more modern and you can use the p and down arrows to navigate the file. With <tt>more</tt>, use the space bar to go down ne page. Try them both and see what is in the middle of <tt>file2</tt> for instance.
+
When you want to view the whole file, the commands <tt>more</tt> or <tt>less</tt> can be used. They both work in a similar way. Like <tt>info</tt>, <tt>less</tt> is more modern and you can use the '''p''' and down arrows to navigate the file. When using <tt>more</tt>, use the space bar to go down a page. Try them both and see what is in the middle of <tt>file2</tt> for instance.
  
Attention, displaying content can only be done for text files. '''It will not work for binary files!'''. For instance, there is a word document in the <tt>folder2</tt> directory. Try to print the top of the file with <tt>head</tt> for instance and look at the results... It is actually dangerous to use <tt>cat</tt>, <tt>head</tt> or <tt>tail</tt> with binary files because the random garbage content could actually execute actions on the shell. So avoid looking inside binary files with these simple viewers.
+
Note that displaying content can only be done for text files. '''It will not work for binary files!'''. For instance, there is a word document in the <tt>folder2</tt> directory. Try to view the top of the file with <tt>head</tt> for instance... It is actually dangerous to use <tt>cat</tt>, <tt>head</tt> or <tt>tail</tt> with binary files because the seemingly random characters churned out to the screen could actually execute actions on the shell. So avoid looking inside binary files with these simple viewers.
  
 
=Comparing content=
 
=Comparing content=
Line 416: Line 416:
 
> I added this into file5.
 
> I added this into file5.
 
</pre>
 
</pre>
 +
 +
Note that there are also graphical tools for comparing files.  You can try <tt>kompare</tt>, <tt>tkdiff</tt> or <tt>meld</tt>, if they are installed on your system.
  
 
We can also use <tt>diff</tt> to compare directories. At the moment <tt>folder1</tt> and <tt>folder3</tt> should be the same. Let's check that:
 
We can also use <tt>diff</tt> to compare directories. At the moment <tt>folder1</tt> and <tt>folder3</tt> should be the same. Let's check that:
Line 434: Line 436:
  
 
=Wild cards and bash completion=
 
=Wild cards and bash completion=
Before going further, it is time to introduce some f the time savers that make Linux such a powerful development environment. So far you have typed commands and fle names in full but Linux is clever enough to help you along the way either by completing the text for you in a similar way to predictive texting on a mobile phone, or by performing substitutions.
+
Before going further, it is time to introduce some time-savers that make Linux a powerful development environment. So far you have typed commands and fle names in full but Linux is clever enough to help you out, either by completing the text for you in a way you could compare to predictive texting on a mobile phone, or by performing substitutions.
  
Completion is done with the tab key. When your press <TAB>, the shell tries to understand what are the possible choices and limits it for you. For instance try to type:  
+
Completion is done with the tab key. (Note that we are assuming you are using the <tt>bash</tt> shell here.)  When your press <TAB>, the shell tries to understand what are the possible choices and limits it for you. For instance try to type:  
 
<pre>
 
<pre>
 
ls fi<TAB> # (don't press enter!)
 
ls fi<TAB> # (don't press enter!)
Line 449: Line 451:
 
</pre>
 
</pre>
  
Another time saver is the wildcard written as an asterisk or star "<tt>*</tt>". It basically matches any number of consecutive characters. For instance if you want to update the time stamp on both <tt>file1</tt> and <tt>file2</tt>, you could type either:
+
Another time saver is the asterisk wildcard, "<tt>*</tt>". It matches any number of consecutive characters. For example, if you want to update the time stamp on both <tt>file1</tt> and <tt>file2</tt>, you could type either:
 
<pre>
 
<pre>
 
$ touch file*
 
$ touch file*
Line 460: Line 462:
 
</pre>
 
</pre>
  
This is very useful way of minimising typing. Another important property: if there is only one directory in a directory, you can simply give the star argument to the <tt>cd</tt> command to go directly into it:
+
This is very useful way of minimising the amount of typing that you have to do. Another important property: if there is only one directory in a directory, you can simply give the star argument to the <tt>cd</tt> command to go directly into it:
  
 
<pre>
 
<pre>
Line 470: Line 472:
  
 
=Searching for things=
 
=Searching for things=
Linux is a powerful environment to sift through many files and directories and find content. It is possible to search for files or directories (i.e. when you know (part of) their names or to search inside them for specific content.
+
Linux provides a powerful environment to sift through many files and directories and find content. It is possible to search for files or directories, when you know (part of) their names, or to search inside them for specific content.
 
==Locating files and directories==
 
==Locating files and directories==
There are two main ways of locating files and directories. the <tt>locate</tt> command can be used to search for whole files of folders containing a particular character string. Try for instance to locate "nano":
+
There are two main ways of locating files and directories. the <tt>locate</tt> command can be used to search for whole files of folders containing a particular character string. Try for instance to locate "nano":
 
<pre>
 
<pre>
 
locate nano
 
locate nano
Line 483: Line 485:
 
</pre>
 
</pre>
  
You have the list of all files containing "nano" on your system. Unfortunately, the locate database is not updated very often and it probably would not find new files or folders. Try to locate the <tt>folder3</tt> for instance. The database is usually updated daily so <tt>locate</tt> would find <tt>folder3</t> if you looked tomorrow probably.
+
You have the list of all files containing "nano" on your system. Unfortunately, the database used by <tt>locate</tt> is not updated very often and it probably would not find new files or folders. Try to locate the <tt>folder3</tt> for instance. The database is usually updated daily so <tt>locate</tt> would find your <tt>folder3</tt> if you looked tomorrow probably.
  
The <tt>find</tt> command can be used to recursively search a given path for a pattern. For instance, to search recursively, starting from the current directory and looking for the pattern "folder_b", use:
+
The <tt>find</tt> command can be used to recursively search a given path for a pattern. For instance, to search recursively, starting from the current directory and looking for the pattern "folder_b", use:
 
<pre>
 
<pre>
 
$ find . -name 'folder_b'
 
$ find . -name 'folder_b'
Line 522: Line 524:
 
</pre>
 
</pre>
  
To search recursively through folder, use the "-r" option. Attention you need to make sure that the list of things to search through includes folders if you want them to be searched recursively!
+
To search recursively through folder, use the "-r" option. Note that you need to make sure that the list of things to search through includes folders if you want them to be searched recursively!
 
<pre>
 
<pre>
 
bash-3.2$ grep -ri pre f*
 
bash-3.2$ grep -ri pre f*
Line 545: Line 547:
 
</pre>
 
</pre>
  
Symbolic links are just "shortcuts" or different addresses/paths to the same content. You can edit <tt>file3</tt> and see the changes in <tt>file1</tt> automatically. the only exception is that you can delete the link (<tt>file3</tt>) and the target will stay.
+
Symbolic links are just "shortcuts" or different addresses/paths to the same content. You can edit <tt>file3</tt> and see the changes in <tt>file1</tt> automatically. The only exception is that you can delete the link (<tt>file3</tt>) and the target will stay.
  
 
To create a symbolic link, the command <tt>ln</tt> is used with the option "-s":
 
To create a symbolic link, the command <tt>ln</tt> is used with the option "-s":
Line 552: Line 554:
 
$ ln -s /gsa13/ggjpr/linux1/file2 folder2/newlink
 
$ ln -s /gsa13/ggjpr/linux1/file2 folder2/newlink
 
</pre>
 
</pre>
 +
 +
= Other useful commands.. =
 +
 +
Include:
 +
 +
* '''<tt>mv</tt>''': for '''m'''o'''v'''ing/renaming files or directories
  
 
= To go further =  
 
= To go further =  
The [[category:Pragmatic_programming]] course continues with an introduction to the Fortran programming language: [[Fortran1]].
+
The [[:category:Pragmatic Programming | Pragmatic Programming]] course continues with an introduction to the Fortran programming language: [[Fortran1]].
  
 
Now that you know an small set of Linux commands, you can also learn how to put them together by reading [[Linux2]].
 
Now that you know an small set of Linux commands, you can also learn how to put them together by reading [[Linux2]].

Latest revision as of 15:58, 5 July 2010

A not so short introduction to the Linux command line

Introduction

Linux (sometimes called GNU/Linux) is an operating system, which you can think of as the middle man between you and the computer hardware. Linux is a full replacement for Microsoft Windows or MacOS X. Linux comes with a range of graphical user interfaces where you can use the mouse to interact with the computer as you would under Windows or OS X. However, it is common to use the Linux command line, for example when you are using the University's HPC facilities. Therefore this practical introduces some of the basic tools that you will find useful on the Linux command line.

The Linux command line interface is often referred to as a shell. The bash shell is common. There are a range of shells (csh, zsh ...) for Linux which use slightly different syntax. Since the bash shell is often the default we will use bash it for this practical.

Logging in

Login is performed using the secure shell or ssh. There is a client for Microsoft Windows which should already be installed on your computer. (If not, contact a computer officer.) To login, type the full name of the machine that you want to log into, for instance dylan.ggy.bris.ac.uk or bluecrystal.bris.ac.uk and then enter your username and password. The system will log you in and you will be taken to your home directory.

On this occasion, you used your password to login. There are other login options available, however. For instance, you can configure ssh to log you in automatically using authentication key-pairs, rather then typing a password. This appraoch can be very useful for (a) the lazy person!; and (b) if you would like to be able to run scripts automatically on a remote machine and perform operations on the network. This is really beyond the scope of this practical.

Getting the content for this practical

Now that you are logged in a Linux system, it is time to get the practical content. The files for this practical are hosted in a version control system. To get them, just cut & paste the following onto your command line:

$ svn export https://svn.ggy.bris.ac.uk/subversion-open/linux1/trunk linux1

Note that we are using the dollar sign "$" here to represent your shell prompt. You do not need to type it in. It is merely there to show that the shell is waiting for a command. We will represent outputs from the shell as lines without a "$".

The svn command will fetch all necessary files and put them in a folder called linux1/. You can ignore the cryptic syntax. We will look at version control using subversion (svn) in a later practical.

Navigation

To change directories (sometimes called folders), the command cd is used. It stands for ... "change directory" (!) To navigate to the directory containing the files for this practical, simply type:

$ cd linux1

By doing this operation, you technically used a relative path. Indeed linux1 is not the full directory location but merely its relative path from where you were before. To find out the absolute path of your current location, use the command pwd (print working directory):

$ pwd
/gsa13/ggjpr/linux1

Obviously, the full path will depend on where your home directory is located. The absolute path starts at / called the root and then lists the branches of the directory tree leading to the current directory. Note that the directories are separated by slashes "/" and not backslashes as used by Microsoft DOS.

The cd command also accepts absolute paths. For instance:

$ cd /gsa3/ggjpr
$ pwd
/gsa3/ggjpr
$ cd /gsa3/ggjpr/linux1
$ pwd
/gsa13/ggjpr/linux1

As an aid to navigation, every directory on a Linux system contains two shortcuts:

  • . (yes, one single dot!) links to itself
  • .. (two consecutive dots) links to its parent

The .. shortcut is especially useful as it allows you to go up one directory without using a full path:

$ pwd
/gsa13/ggjpr/linux1
$ cd ..
$ pwd
/gsa3/ggjpr
$ cd ..
$ pwd
/gsa3

It is important to know that if you don't give an argument to cd, it will take you back to your home directory.

$ pwd
/gsa3
$ cd
$ pwd
/gsa3/ggjpr

Furthermore, you can use cd with the argument "-" to simply go back where you were before, a bit like the back button on a web browser. It only remembers one 'level', however, so using it twice sends you back to where you started. For instance:

$ pwd
/gsa13/ggjpr/linux1
$ cd
$ pwd
/gsa13/ggjpr
$ cd -
$ pwd
/gsa13/ggjpr/linux1
$ cd -
$ pwd
/gsa13/ggjpr

Listing a directory content

The content of the directory can be listed with the command ls (for list). For example, let's go back in the linux1 directory and list its content:

$ cd linux1
$ ls
file1  file2  folder1  folder2

So our linux1 directory contains two files and two directories. Depending on the secure shell software and the Linux machine, you could also have a colourised output to differentiate easily between files and directories. By default, an alphabetical order is used when listing directory content but sometimes, the directories are listed together before the files.

The ls command also accepts some options to modify its behaviour. The options are listed after a dash "-". For instance, the "-r" option reverse the listing order:

$ ls -r 
folder2 folder1 file2  file1

Files and folders can be hidden on Linux. To show hidden files, use ls with the "-a" option (for all):

$ ls -a
.  ..  file1  file2  folder1  folder2  .hiddenfile  .hiddenfolder

You can see that:

  • the dot and dot dot shortcuts are actualy hidden folders
  • hidden files and folders also start with a dot (".")

If you go in your home directory, there should be a few hidden files and folders already. They are usually used to store configuration information. Here is a typical home directory on a (heavily used) Linux system:

$ cd
$ ls -a
.              .gconfd            .mcoprc                      Templates
..             .gimp-2.4          .metacity                    .themes
.adobe         .gnome             .mozilla                     .thumbnails
.bash_history  .gnome2            Music                        .thunderbird
.bash_logout   .gnome2_private    .nautilus                    .tomboy
.bash_profile  .gstreamer-0.10    .openoffice.org2.0           .tomboy.log
.bashrc        .gtk-bookmarks     Public                       .Trash
bin            .gtkrc-1.2-gnome2  .pulse                       .unison
.config        hydrostab          .pulse-cookie                unison.log
Desktop        .ICEauthority      .qt                          Videos
.dmrc          .icons             .recently-used.xbel          .viminfo
Documents      linux1             .Skype                       .vimrc
Download       .kde               skype-2.0.0.13-fc5.i586.rpm  .wapi
.esd_auth      .libgda            .ssh                         .xine
.evolution     linux.tgz          .subversion                  .xsession-errors
.fontconfig    .local             systel90
fortran1       .macromedia        tecplot360-2008
.gconf         .mcop               

Obviously you can ignore these files most of the time. This is why they are hidden.

When you want more detail about the files and directories, use the "-l" option to use the long listing format. Let's go back to the linux1 directory and try it:

$ cd linux1
$ ls -l
total 32
-rw-r--r-- 1 jp jp   73 2008-02-01 10:43 file1
-rw-r--r-- 1 jp jp  284 2008-02-01 10:43 file2
drwxr-xr-x 2 jp jp 4096 2008-02-01 10:43 folder1
drwxr-xr-x 2 jp jp 4096 2008-02-01 21:30 folder2

The detailed output for each item starts with a big block which contains the file or directory permissions, the number of links to the entity, the file/directory owner, the file/directory group, the file size, the last modified date and time and eventually the file/directory name. The output might slightly vary between systems. Permissions, ownership and groups are explained later on. Note that the number of links is one for files and at least 2 for directories (because dot and dot dot are always present).

The file size is given in bytes. To make it easier to read, it is possible to use the -h option (for human readable):

$ ls -l -h
total 32
-rw-r--r-- 1 jp jp   73 2008-02-04 12:46 file1
-rw-r--r-- 1 jp jp  284 2008-02-04 12:50 file2
drwxr-xr-x 2 jp jp 4.0K 2008-02-04 23:40 folder1
drwxr-xr-x 2 jp jp 4.0K 2008-02-04 23:41 folder2

Only the directory size is changed and you can see K has been used for KiloBytes (KB). There is a more useful example in the folder2 directory.

$ cd folder2
$ ls -l
total 120
-rw-r--r-- 1 jp jp 104448 2008-02-04 23:41 file4.doc
-rwxr-xr-x 1 jp jp     48 2008-02-04 23:46 runme
$ ls -l -h
total 120K
-rw-r--r-- 1 jp jp 102K 2008-02-04 23:41 file4.doc
-rwxr-xr-x 1 jp jp   48 2008-02-04 23:46 runme

Note that 104,448 bytes is "rounded" to 102KB. This is not an error, it is simply that 1KB is 1,024 Bytes and not 1,000...

When several arguments are given to ls, they can be grouped together, therefore "ls -l -h" is identical to ls -lh".

There is a very useful combination of options for ls. To find the last file modified in a directory, use "ls -ltr". The "t" option use chronological order instead of alphabetical, "l" for long output and "r" for reverse, you end up with the youngest file or directory at the bottom of the list.

$ ls -ltr
total 32
-rw-r--r-- 1 jp jp   73 2008-02-04 12:46 file1
-rw-r--r-- 1 jp jp  284 2008-02-04 12:50 file2
drwxr-xr-x 2 jp jp 4096 2008-02-04 23:40 folder1
drwxr-xr-x 2 jp jp 4096 2008-02-04 23:46 folder2

In the above example, the directory folder2 contains the latest files.

Getting help

As you have seen with the ls example, some commands have many functions and it is easy to lose track. Luckily it is usually quite easy to get help for most commands. Two ways to get help are available. You can use man (short for manual) and info. To use either, just add on the name of the command your interested in (or bemused by!). For example, to find help about the ls command using the "man" pages, type::

man ls

Use the arrows to navigate up and down line by line, the space bar to go down a whole screen height, and enter q to quit. You can also enter "/word" to look for the string "word".

Similarly you can use the "info" help system:

info ls

Use the arrows to navigate up and down line by line, the space bar to go down a whole screen height, and enter q to quit. You can use enter on a any link identified by an asterisk to be taken directly to that content. You can also enter "/word" to look for the string "word".

The info system is newer than the man pages and probably its successor, but man remains very popular. Now, to get more information about them, you could try "info man" for instance...

Unix permissions

We saw earlier on that the long output of ls contains a block about file permissions. To understand this block, you need to know that users on a Linux system are organised into groups. Each file or directory has an owner, who is a user, and that user also belongs to a group. For each file and directory on a Linux system, it is possible to give permissions or access rights to the owner of a file, to the group to which the file belongs to ... and also to the other users of the system, i.e. those who don't own the file or belong to the group.

The permissions include read access (r), write access (w) and execute access (x). The permissions are usually listed together so a file with "rwx" for a user means that the user has all rights on that file. "r-x" means the user can read and execute but not write/modify. "r--" means read-only access. Note that in order to be able to enter a directory (e.g. using cd), a user needs both read and execute access ("r-x").

The output of ls -l is now easier to understand, for instance:

$ ls -l
total 32
-rw-r--r-- 1 jp jp   73 2008-02-04 12:46 file1
-rw-r--r-- 1 jp jp  284 2008-02-04 12:50 file2
drwxr-xr-x 2 jp jp 4096 2008-02-04 23:40 folder1
drwxr-xr-x 2 jp jp 4096 2008-02-04 23:46 folder2

All files in this directory belong to the user jp and also the group jp. The user jp has read and write permissions for file1 and file2, users belonging to the group jp and other users have only read access. For the two directories folder1 and folder2, only the user jp can modify them. However, all users can enter those directories. Note that being able to modify a directory includes the right to add or remove files.

Often, the permissions are simplified(!) using a binary counting system where "--x" is 1, "-w-" is 2 and "r--" is 4. Therefore, "rwx" is 7 and "r-x" is 5. Using this system for the user, group and other users, the permissions of a file or directory can be sumarised as a 3 digit number. For example, 755 can be used instead of "rwxr-xr-x" or 644 instead of "rw-r--r--". These two are the typical permissions for a directory and file respectively.

Note that on each line of the ls -l output, the first character marks the type of the entity: A dash "-" for a regular file, a "d" for a directory and you would see an "l" for a link. We will introduce links later.

The permissions on a file or directory can be changed using the command chmod (short for change modalities).

$ chmod -r file1
$ ls -ltotal 48
--w------- 1 jp jp   73 2008-02-04 12:46 file1
-rw-r--r-- 1 jp jp  284 2008-02-04 12:50 file2
drwxr-xr-x 2 jp jp 4096 2008-02-04 23:40 folder1
drwxr-xr-x 2 jp jp 4096 2008-02-04 23:46 folder2
drwxr-xr-x 3 jp jp 4096 2008-02-05 01:36 folder_a
drwxr-xr-x 2 jp jp 4096 2008-02-05 01:34 new-folder

To add read and execute access to the group but not the owner or the others, use

$ chmod g+rx file1
$ ls -l
--w-r-x--- 1 jp jp   73 2008-02-04 12:46 file1
-rw-r--r-- 1 jp jp  284 2008-02-04 12:50 file2
drwxr-xr-x 2 jp jp 4096 2008-02-04 23:40 folder1
drwxr-xr-x 2 jp jp 4096 2008-02-04 23:46 folder2
drwxr-xr-x 3 jp jp 4096 2008-02-05 01:36 folder_a
drwxr-xr-x 2 jp jp 4096 2008-02-05 01:34 new-folder

Use the letter u for the user/owner, g for the group and o for the others. Be careful, o is for the others, not the owner. chmod can also be used with a 3 digit permission summary too.

Creating/deleting/modifying

Creating directories and files

Now that we know about permissions, it is time to actually create a some content.

To create a directory, use the command mkdir (short for make directory):

$ mkdir new-folder
$ ls
file1  file2  folder1  folder2  new-folder

You can also create nested directory using the option "-p" (for create parents):

$ mkdir -p folder_a/folder_b/folder_c
$ ls
file1  file2  folder1  folder2  folder_a  new-folder
$ cd folder_a
$ ls
folder_b
$ cd folder_b
$ ls
folder_c

To create a new empty file, use the command touch

$ touch new_file
$ ls -rtl
total 4
-rw-r--r-- 1 jp jp 0 2008-02-05 01:38 new-file

New directories are usually created with the permissions 755 (rwxr-xr-x) and new files with 644 (rw-r--r--).

The touch command can actually be used to modify the time stamp of the file, i.e. it changes its latest modification date without changing its content.

2$ ls -rtl
total 4
-rw-r--r-- 1 jp jp 0 2008-02-05 01:38 new-file
bash-3.2$ touch new-file
bash-3.2$ ls -rtl
total 4
-rw-r--r-- 1 jp jp 0 2008-02-05 01:42 new-file

Deleting files or directories

To delete a file use the command rm (remove). For instance:

$ ls
new-file
$ rm new-file
$ ls

Attention, there is no trash can when you use rm. When you delete something, it's gone for good! The option "-i" can be used to force <ttrm to ask for confirmation prior to actually deleting anything:

$ touch new-file
$ rm: remove regular empty file `new-file'? y
$ ls

On some systems, rm defaults to the "-i" option.

To delete a directory, use the option "-r".

$ cd ..
$ rm -r folder_c

"-r" is for recursive. When the directory is not empty, you cannot simply delete it.

$ cd ../..
$ rm -r folder_a
$ ls
file1  file2  folder1  folder2  new-folder

Note that if your system default to the "-i" option, deleting a directory is time conduming as a confirmation will be asked for each file/directory. It is therefore advisable to use option "-f" (for force).

Editing files

Linux can provide quite an array of different editors for modifying files. Precisely which tools are installed varies from system to system, but often the following editors are present: vim, emacs, nano and gedit. The first three will open in your secure shell window. vim and emacs can be a bit daunting at first. To exit vim type "Escape" and then enter ":q". To exit emacs type control-x and then ctrl-c. nano is easier to use to start with.

gedit is actually a graphical text editor, i.e. it opens in its own window and you get a set of buttons/menus to save, copy, cut, paste etc... However, because you are not physically connected to the display of the Linux machine (i.e. your screen is not attached to directly the server you are logged into), an extra operation is required to allow that Linux machine to open a window on the screen of your PC. This is done with a piece of software called Exceed. You need to start Exceed from the start menu. You must also change some settings on your SSH connection. Select "Edit->Settings->Profile Settings" and tick the box "Tunnel X11 connections" under "Connections->Tunneling", and save your settings. This configuration only has to be done only once. However, Exceed has to be running for this to work (and you may need to restart your SSH connection).

(Note that your files on dylan are also available (via something called Samba) through your PC, so you can edit them in MS Windows, using your favourite text editor and you don't really need to use any of the Linux file editors.)

Copying directories and files

To copy a directory and file, use the command cp with the option "-r" for directories:

$ cp file1 file5
$ cp -r folder1 folder3
$ ls
XXX

Displaying content

Sometimes, you want to look a the content of a file but don't need to edit it. It is possible to display file content on Linux by a range of methods. The command cat (short for concatenate) simply displays the content of a file to the screen, if it is given a single file. It is quick but not very convenient for large files! head and tail can be used to look respectively at the top or bottom of a file. Both commands usually default to 10 lines of the file, althpugh this can be modified with an optional argument (-n). These commands are very convenient if you would like to look at a file header or the very last lines written to a file (both quite common situations).

$ head file2
This is file 2.
This is another text file.
It contains some text at the top.






$ tail file 2









and more text at the end, including the word Computing".

When you want to view the whole file, the commands more or less can be used. They both work in a similar way. Like info, less is more modern and you can use the p and down arrows to navigate the file. When using more, use the space bar to go down a page. Try them both and see what is in the middle of file2 for instance.

Note that displaying content can only be done for text files. It will not work for binary files!. For instance, there is a word document in the folder2 directory. Try to view the top of the file with head for instance... It is actually dangerous to use cat, head or tail with binary files because the seemingly random characters churned out to the screen could actually execute actions on the shell. So avoid looking inside binary files with these simple viewers.

Comparing content

It is also possible to compare the content of two files or directories. For instance, the files file1 and file5 are identical so far. We could check it with the command diff.

$ diff file1 file5
$ # => no output, therefore files are identical

Now modify the file file5 with a text editor and try again:

$ ... modify file5 somehow ...
$ 2$ diff file1 file5
1c1,2
< This file 1. It contains a bit of text including the word "PREHISTORIC".
---
> This file 5. It contains a bit of text including the word "PREHISTORIC".
> I added this into file5.

Note that there are also graphical tools for comparing files. You can try kompare, tkdiff or meld, if they are installed on your system.

We can also use diff to compare directories. At the moment folder1 and folder3 should be the same. Let's check that:

$ diff folder1 folder3
$ 

Then add things to folder3 and try again:

$ touch folder3/another-file
$ mkdir folder3/new-folder
$ diff folder1 folder3
Only in folder3: another-file
Only in folder3: new-folder

Wild cards and bash completion

Before going further, it is time to introduce some time-savers that make Linux a powerful development environment. So far you have typed commands and fle names in full but Linux is clever enough to help you out, either by completing the text for you in a way you could compare to predictive texting on a mobile phone, or by performing substitutions.

Completion is done with the tab key. (Note that we are assuming you are using the bash shell here.) When your press <TAB>, the shell tries to understand what are the possible choices and limits it for you. For instance try to type:

ls fi<TAB> # (don't press enter!)

The shell knowns that the only content starting with "fi" have also the next letters in common and simply adds them for you. And you only have to add the last digit yourself.

If you hit <TAB> twice, it will actually show you a list of possible choices to help you go further.

$ ls fi<TAB> <TAB> # => then becomes "ls file"
file1 file2

Another time saver is the asterisk wildcard, "*". It matches any number of consecutive characters. For example, if you want to update the time stamp on both file1 and file2, you could type either:

$ touch file*

or even shorter

$ touch fi*

This is very useful way of minimising the amount of typing that you have to do. Another important property: if there is only one directory in a directory, you can simply give the star argument to the cd command to go directly into it:

$ mkdir -p folder_a/folder_b/folder_c/folder_d/folder_e
$ cd folder_a/*/*/*/*
$ pwd
/gsa13/ggjpr/linux1/folder_a/folder_b/folder_c/folder_d/folder_e

Searching for things

Linux provides a powerful environment to sift through many files and directories and find content. It is possible to search for files or directories, when you know (part of) their names, or to search inside them for specific content.

Locating files and directories

There are two main ways of locating files and directories. the locate command can be used to search for whole files of folders containing a particular character string. Try for instance to locate "nano":

locate nano
/usr/bin/nano
/usr/bin/rnano
/usr/share/nano
/usr/share/doc/nano-2.0.6
/usr/share/doc/nano-2.0.6/AUTHORS
... much more content...

You have the list of all files containing "nano" on your system. Unfortunately, the database used by locate is not updated very often and it probably would not find new files or folders. Try to locate the folder3 for instance. The database is usually updated daily so locate would find your folder3 if you looked tomorrow probably.

The find command can be used to recursively search a given path for a pattern. For instance, to search recursively, starting from the current directory and looking for the pattern "folder_b", use:

$ find . -name 'folder_b'
./folder_a/folder_b

Wildcards can be used too, for instance:

$ find . -name 'fol*_*'
./folder_a
./folder_a/folder_b

Searching inside files and directories

The grep command can be used to look inside files. grep stands for Generalised Regular Expression Parser.The syntax is the following:

grep "pattern" list of files and directories to search

For instance, to look for the patter "PRE" inside the files and directories starting with "file", use:

$ grep PRE file*
file1:This file 1. It contains a bit of text including the word "PREHISTORIC".

Note that grep is case sensitive by default and the "-i" option can be used to ignore the case.

$ grep pre file*
$ # => no output, i.e. nothing found
$ grep -i pre file*
file1:This file 1. It contains a bit of text including the word "PREHISTORIC".

To search recursively through folder, use the "-r" option. Note that you need to make sure that the list of things to search through includes folders if you want them to be searched recursively!

bash-3.2$ grep -ri pre f*
file1:This file 1. It contains a bit of text including the word "PREHISTORIC".
file5:This file 5. It contains a bit of text including the word "PREHISTORIC".
folder1/file3:This file 1. It contains a bit of text including the word "PREHISTORIC".
folder3/file3:This file 1. It contains a bit of text including the word "PREHISTORIC".

A new type of file: symbolic links

You have noticed that folder1/file3 contains the searched pattern above. Let's look in more detail at this file:

$ ls -l folder1/file3
lrwxrwxrwx 1 jp jp 8 2008-02-04 23:40 folder1/file3 -> ../file1

file3 is called a symbolic link and it links to file1. So it has the same content:

$ diff file1 folder1/file3
$ # => so no difference between the two files

Symbolic links are just "shortcuts" or different addresses/paths to the same content. You can edit file3 and see the changes in file1 automatically. The only exception is that you can delete the link (file3) and the target will stay.

To create a symbolic link, the command ln is used with the option "-s":

$ ln -s /gsa13/ggjpr/linux1/file2 folder2/newlink

Other useful commands..

Include:

  • mv: for moving/renaming files or directories

To go further

The Pragmatic Programming course continues with an introduction to the Fortran programming language: Fortran1.

Now that you know an small set of Linux commands, you can also learn how to put them together by reading Linux2.