Python1
Python for Scientists
Introduction
With thanks to Simon Metson and Mike Wallace for much of the following material.
Getting Started on BlueCrystal Phase-2
After you have logged in, type the following at the command line:
module add languages/python-2.7.2.0 python
This should start up an interactive python session:
Python 2.7.2 (default, Aug 25 2011, 10:51:03) [GCC 4.3.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>>
where we can type commands at the >>> prompt.
Python as a Calculator
To get started, let's just try a few commands out. If you type:
>>> print "Hello!"
you'll get:
Hello!
If you try:
>>> print 5 + 9
you'll get:
14
So far so simple! Here is a copy of a session containing a few more commands where we've set the values of some variables and also defined and run our own function:
>>> five = 5
>>> neuf = 9
>>> print five + neuf
14
>>> def say_hello():
... print "Hello, world!"
... # hit return here
>>> say_hello()
Hello, world!
You can exit an interactive session at any time by typing Ctrl-D.
Getting Help
One of the good things about Python is that it has lots of useful online documentation. (There are good books on the language too.) For example, take a look at: http://docs.python.org/. You can also type help() and the interpreter prompt:
>>> help()
Welcome to Python 2.7! This is the online help utility.
If this is your first time using Python, you should definitely check out
the tutorial on the Internet at http://docs.python.org/tutorial/.
Enter the name of any module, keyword, or topic to get help on writing
Python programs and using Python modules. To quit this help utility and
return to the interpreter, just type "quit".
...
help> keywords
Here is a list of the Python keywords. Enter any keyword to get more help.
and elif if print
...
help> if
The ``if`` statement
********************
The ``if`` statement is used for conditional execution:
if_stmt ::= "if" expression ":" suite
( "elif" expression ":" suite )*
["else" ":" suite]
It selects exactly one of the suites by evaluating the expressions one
by one until one is found to be true...
...
help> quit
You are now leaving help and returning to the Python interpreter.
...
>>>
Making a Script
An interactive session can be fun and useful for trying things out. However--to save our fingers--we will typically want to execute a series of commands as a script, created using your favourite text editor. Here are the contents of an example script:
#!/bin/env python
print "Hello, from a python script!"
Ensure that your script is executable:
chmod u+x myscript.py
and now you can run it:
[ggdagw@bigblue4 ~]$ ./myscript.py Hello, from a python script!
Python and Whitespace
Love it of hate it, Python incorporates whitespace in it's syntax. (It's either that or demarcate blocks with some other syntax, such as ending a line with a semi-colon as it is in C. Pick your poison.) Spacing is therefore key in creating a valid python script. For example:
message = "happy days!"
if len(message) > 10:
print "longer.."
else:
print "shorter.."
will work, but:
message = "happy days!"
if len(message) > 10:
print "longer.."
else:
print "shorter.."
will not:
File "./myscript.py", line 7
print "shorter.."
^
IndentationError: expected an indented block
It is therefore a great advantage, when writing to python script, to use a text editor which has a dedicated python mode--such as emacs--and will actively help you to keep your spacing correct. See, http://wiki.python.org/moin/PythonEditors, for an extensive list.
Nuts and Bolts
Types
Python has intrinsic types including, integers, floats, booleans and complex numbers. It is dynamically typed (meaning that you don't have to have a block of variable declarations at the top of your script), but it is not weakly typed, for example:
>>> my_complex = 2 + 0.5j
>>> my_complex
(2+0.5j)
>>> my_complex.real
2.0
>>> my_complex.imag
0.5
>>> name = 'fred'
>>> lucky = 7
>>> name + lucky
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: cannot concatenate 'str' and 'int' objects
Strings
The eagle-eyed will have spotted in a previous examples that we could ask the length a character string--straight off the bat. No need to write a counting routine ourselves:
print len(message)
We also take slices of our character string. In my case
print message[:5]
Since a string is an object (in the object oriented programming sense of the word, but more of that another time...) we can call a number of methods that operate on a string. A selected sample include:
| s.find(sub) | Finds the first occurrence of the given substring |
| s.islower() | Checks whether all characters are lowercase |
| s.upper() | Returns s converted to uppercase |
| s.strip() | Removes leading and trailing whitespace |
| s.replace(old,new) | Replaces substring old with new |
| s.split([sep]) | Splits s uses (optional) sep as a delimiter. Returns a list |
Lists and Tuples
An example of a list is:
shopping = ['bread', 'marmalade', 'milk', 'tea']
and we can inquire about the length of that using the same function as before:
len(shopping)
We can also take slices of a list, as we did with a string:
shopping[0:2]
and even reset a portion of the list that way:
shopping[0:2] = ['bagels', 'jam']
Since a list is also an object, we have more handy methods, including:
| s.append(x) | Appends an new element x to the end of s |
| s.count(x) | Returns the number of elements in s |
| s.reverse(x) | Reverses items of s in place |
| s.sort([compfunc]) | Sorts items of s in place. compfunc is an optional comparison function |
Tuples are very similar to lists and support many of the same operations (indexing, slicing, concatenation etc.) but differ in that they are not mutable after creation:
>>> mytuple = ('fred', 'ginger', 7, 2.5)
>>> mylist = ['fred', 'ginger', 7, 2.5]
>>> mylist[2] = 8
>>> print mylist
['fred', 'ginger', 8, 2.5]
>>> print mytuple[2]
7
>>> mytuple[2] = 8
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignmentList comprehension:
>>> numbers = [12, 3, 90, 40, 52, 11, 10]
>>> small_numbers_doubled = [number * 2 for number in numbers if number < 20]
>>> small_numbers_doubled
[24, 6, 22, 20]
Dictionaries
A dictionary is an associative array or hash table, containing key-value pairs:
mydict = {'thomas':'blue', 'james':'red', 'henry':'green'}
>>> print mydict['james']
red
We can write much more user-friendly and intuitive code using dictionaries, rather than arbitrary indexes into a list.
Some example dictionary methods are:
| m.keys() | Returns a list of the keys in m |
| m.items() | Returns a list of the (key,value) pairs in m |
| m[k] = x | Sets m[k] to x |
| m.update(b) | Adds objects from dictionary b to m |
Control Structures
Of course, we'll need conditionals and loops etc. to go beyond the simplest of scripts. Here is an if-then-else, python style:
if sky == ‘blue’:
birds_sing()
elif sky == ‘black’:
birds_sleep()
else:
pass #do nothing
and a classic for loop:
for ii in range(1,10):
print ii
We'll also see a while loop shoehorned into the next example.
For our control statements, we can use comparison operators such as, ==, !=, >, <, <=, >=, and logical operators, such as, and, or,not
File Input and Output
Here's some code for printing the contents of a text file:
fp = open("foo.txt","r")
line = fp.readline()
while line:
line = line.strip()
print line
line = fp.readline()
fp.close()
We could open a file for writing with:
fp = open("foo.txt","w")
Object Oriented Programming in Python
Here is an example of using a class in python:
#!/usr/bin/env python
class Radio:
"A simple radio"
def __init__(self,freq=0.0,name=""):
"Constructor method"
self.__frequency=freq
self.name=name
def tune(self,freq):
self.__frequency=freq
def tuned_to(self):
print self.name, "tuned to:", self.__frequency
if __name__ == "__main__":
# declare two radio instances
car = Radio(name="car")
kitchen = Radio(91.5,"kitchen")
# call some methods
car.tuned_to()
kitchen.tuned_to()
car.tune(89.3)
car.tuned_to()
# Docstrings--double quotes at the top of the class:
print car.__doc__
# NB members not private by default:
print car.name
# BUT leading double underscores will trigger
# name mangling and hence the member will be hidden
print car.__frequency
Running the script gives us:
car tuned to: 0.0
kitchen tuned to: 91.5
car tuned to: 89.3
A simple radio
car
Traceback (most recent call last):
File "./foo.py", line 27, in <module>
print car.__frequency
AttributeError: Radio instance has no attribute '__frequency'
Using Packages
Python packages are great because they provide us with a whole lot of extra functionality--above and beyond the core language--that we didn't have to write and debug ourselves.
Let's walk through a simple example using a package. At an interactive prompt type:
from random import randint
This will give us access to the randint(x,y) function, which returns a randomly chosen integer from the given range [x,y]:
>>> randint(0,10)
4
>>> randint(0,10)
1
>>> randint(0,10)
3
>>> randint(0,10)
0
OK, so far so good. One thing to note is that the above import statement has drawn the name randint into our current namespace. What if we had already defined a function named randint. That could cause problems. In order to protect ourselves from this kind of problem, there are several import variants.
By default, functions will be added to a namespace with the same name as the package. In order to call the functions we will, in this case, have to prefix them with there namespace:
>>> import random
>>> random.randint(0,10)
6
Should we desire, we can apply a little more control and specify the namespace for the import ourselves:
>>> import random as rnd
>>> rnd.randint(0,10)
3
Another--more 'devil-may-care'--approach is to do away with the separate namespace and pull everything from a given package into the current namespace:
>>> from random import *
>>> randint(0,10)
9
>>> random()
0.3172268098313996
(The random() function returns a randomly selected floating point number in the range [0, 1)--that is, between 0 and 1, including 0.0 but always smaller than 1.0.)
Numpy
OK, let's move onto looking at python's numerical processing capabilities. We will start by looking at the numpy package:
from numpy import *
Now that we have access to the functions from numpy, let's create an array. Note that a numpy array is an object of a different type to an intrinsic array in Python. A simple approach is to use the array function. For example we might enter:
a = array([[1.0,0.0,0.0],[0.0,1.0,0.0],[0.0,0.0,1.0]])
b = array([[1,2,3],[4,5,6],[7,8,9]])
>>> a
array([[ 1., 0., 0.],
[ 0., 1., 0.],
[ 0., 0., 1.]])
>>> b
array([[1, 2, 3],
[4, 5, 6],
[7, 8, 9]])
>>> transpose(b)
array([[1, 4, 7],
[2, 5, 8],
[3, 6, 9]])
Given an array, we may inquire about it's shape:
print a.shape
and we are told that it is a 2-dimensional array (i.e. an array of rank 2) and that the length of both dimensions is 3:
(3, 3)
We can also apply operators to array objects. For example:
a = a * 9
array([[ 9., 0., 0.],
[ 0., 9., 0.],
[ 0., 0., 9.]])
Note, however, that most operations on numpy arrays are done element-wise, which is different to a linear algebra operation that you may have been expecting. We will return to linear algebra operations when we look at the scipy package.
Should we so desire, we could re-shape the array. One way to do this is to to set it's shape attribute directly:
>>> a.shape = (1,9)
>>> a
array([[ 9., 0., 0., 0., 9., 0., 0., 0., 9.]])
As with the list example, it can be useful to read or change the value of an element (or sub array) individually. Let's turn the array back to it's rank-2 form and try it out:
>>> a.shape = (3,3)
>>> a[1,1] = 777.0
>>> print a
[[ 9. 0. 0.]
[ 0. 777. 0.]
[ 0. 0. 9.]]
>>> a[1:,1:] = [[777.0, 777.0],[777.0, 777.0]]
>>> print a
[[ 9. 0. 0.]
[ 0. 777. 777.]
[ 0. 777. 777.]]
This is all pretty handy so far, but specifying the value of each element explicitly could become a chore. Happily some helper functions exist to give you a head start with some building blocks. For example, your can use:
>>> b = zeros((3,3)
>>> print b
>>> b = ones((3,2))
>>> print b
>>> b = identity(2)
>>> print b
>>> big = resize(b, (6,6))
>>> print big
The use of resize in the last example illustrates a useful replicating feature.
A list of all the functions and operations contained within numpy is: http://scipy.org/Numpy_Example_List.
Pylab and Matplotlib
The above examples are quite natty, but we have deliberately kept the array sizes small so that we can print the element values easily. In practice, you may find that your array sizes are much larger and printing the values to the screen is impractical. Fear not! Python has many packages which help you plot your data, so that you can explore it.
Using the pylab plotting interface we can create:
import pylab
from numpy import arange, pi, cos, sin
t = arange(0.0, 3.0, 0.01)
c = cos(2 * pi * t)
s = sin(2 * pi * t)
pylab.ylabel('some numbers')
pylab.xlabel('some more numbers')
pylab.plot(t, c, 'r', lw=2)
pylab.plot(t, s, 'b', lw=2)
pylab.plot(t, c-s, 'gs', lw=2)
pylab.ylim(-1.5, 1.5)
pylab.title('sin and cos functions')
pylab.savefig('curves', dpi=300)
Where curves.png looks like:
You can open .png images from the linux command line (inc. bluecrystal) using, e.g.: display -resize 1000 curves.png
We can also use Matplotlib directly for more control:
>>> x = arange(-5,10)
>>> y = arange(-4,11)
>>> z1 = sqrt(add.outer(x**2,y**2))
>>> Z = sin(z1)/z1
>>> import matplotlib.pyplot as plt
>>> from pylab import meshgrid
>>> X, Y = meshgrid(x,y)
>>> plt.figure()
>>> plt.contour(X,Y,Z)
>>> plt.show()
and you should get a window similar to:
Perhaps the best way next step for matplotlib is to look at the gallery: http://matplotlib.org/gallery.html. Just click on a figure and you will get the code used to generate it--a really great resource!
Input and Output
The foregoing is all very interesting, but life would be rather dull if you had to re-enter all your data by hand whenever you set to work with Python and numpy. Therefore we need a means to save data to a file and load it again. Happily, we can do this rather easily using a couple of routines from the pylab package:
>>> from numpy import *
>>> from pylab import load
>>> from pylab import save
>>> data = zeros((3,3))
>>> save('myfile.txt', data)
>>> read_data = load("myfile.txt")
warning, the load() function of numpy will be shadowed in the above example. One way to protect yourself against this is to make use of namespaces: Modify your import command to import pylab and then use pylab.load(..).
Scipy
- http://www.scipy.org/
- ..and good examples on http://scipy-lectures.github.com/intro/scipy.html
- Many useful features:
- Integration & Differentiation
- Optimisation (curve fitting, etc)
- Fourier transforms
- Signal processing
- Statistical algorithms
- Much, much more...
- If you know Python you can use SciPy
An example: Differentiation
>>> # derivative of x^2 at x=3
...
>>> from scipy import derivative
>>> derivative(lambda x: x**2, 3)
6.0
>>> # also works with arrays
...
>>> from numpy import array
>>> my_array = array([1,2,3])
>>> derivative(lambda x: x**2,my_array)
array([ 2., 4., 6.])
Google for many more examples pertaining to your favourite numerical procedure!
Packages
Now, we've touched on a couple, but there are thousands of python packages available. Before you start writing your own function for X, check that someone hasn't contributed code for that already at http://pypi.python.org/pypi.