Linux2

Leveraging the power of the Linux command line

Introduction

Roll call: Jonny, Lauren, Emma, Guy, Tim, Rita, SarahS, Jenny

This practical follows Linux1 which introduced the fundamentals of the Linux command line. During this practical, we will learn how to combine some commands together to create scripts that perform more complex actions.

Getting the content for this practical

The necessary files for this practical are hosted in a version control system. To obtain them, just type the following command:

$ svn export http://source.ggy.bris.ac.uk/subversion-open/linux2/trunk linux2

This will fetch all necessary files and put them in a folder called linux2/. Ignore the cryptic syntax so far, an introduction to version control using subversion (svn) will be given later on.

Output redirection

In the Linux1 practical, we have discovered a few Linux commands. Some of these commands use input from the keyboard (standard input) and output data to the screen (standard output). It is possible to (a) redirect input and output and (b) link commands together to perform complex actions. The files for this section are in the example1 directory

Redirecting standard input and output

The diff command by default outputs to the screen:

$ diff file1 file2

It is easy to redirect its output to a file so that the output can be saved for later. This is done by using the sign">" and the name of a file:

$ diff file1 file2 > diff12.txt
$ diff file2 file3 > diff23.txt

Now that we have the difference between the two files, we could concatenate them into one big file:

$ cat diff12.txt diff23.txt > diff.txt

However, it is also possible to append the output of one command to a file, not the second call below, it uses a double ">":

$ diff file1 file2 > diff.txt
$ diff file2 file3 >> diff.txt

The result is exactly the same. Just remember that a single ">" will overwrite the content of a file, a double will concatenate

It is also possible to redirect the input although it not used as often as most commands . For instance consider the function sort which can be used to sort things. usuall, it takes and input from

sort < file4 

You can also combine the two:

sort < file4 > file4-sorted.txt

Pipelines

pipes between simple commands

ls -l | more

diff file1 file2 | tail

sort file4 | uniq
<pre>

<pre>
grep -i scene file1 | wc -l

Automating things

"batch files"

$ cd example2

convert is a small utility from the program Imagemagick which allows the manipulation of images at the command line. For instance, to resize an image to 2000 pixels max and rename it, you could use:

$ convert image-large.jpg -resize 2000 image-2000.jpg

Now let's say

#!/bin/bash

echo "Create thumbnails."
convert image-large.jpg -resize 2000 image-2000.jpg
convert image-large.jpg -resize 1000 image-1000.jpg
convert image-large.jpg -resize 500 image-500.jpg
convert image-large.jpg -resize 100 image-100.jpg
convert image-large.jpg -resize 10 image-10.jpg

echo "Move thumbnails."
mkdir thumbnails
mv image-*0.jpg thumbnails/

echo "Compress thumbnails."
zip -r thumbnails thumbnails

echo "Clean up."
rm -rf thumbnails

echo "All done."

Launching, monitoring and controlling jobs

Background, bg, jobs, fg, nohup, top, kill

Shell Scripting

Arithmetic

Putting everything together in one script.

Environment Variables

SHELL PWD PATH (LD_LIBRARY_PATH)

Text Processing

sed, awk.

Managing Data?

du, df, file. Use of symbolic links. tar, zip, gzip, bzip2