Python Notes – 10 : Threading

Welcome to our tenth note in the Python learning process. In this note we will talk about threading, threads communication and synchronization.

Threads basics

A running program is called a "process". Each process has memory, list of open files, stack, program counter, etc…. Normally, a process executes statements in a single sequence of control-flow.

The following commands create an entirely new process: fork(),system(), popen(), etc… This child process runs independently of the parent. Has own set of resources. There is minimal sharing of information between parent and child.

On the other side, a thread is kind of like a process (it’s a sequence of control-flow). Except that it exists entirely inside a process and shares resources. A single process may have multiple threads of execution. This is extremely useful when an application wants to perform many concurrent tasks on shared data.

Problems with Threads

• Scheduling : To execute a threaded program, must rapidly switch between threads. This can be done by the user process (user-level threads) or Can be done by the kernel (kernel-level threads).
• Resource Sharing : Since threads share memory and other resources, you must be very careful. Operation performed in one thread could cause problems in another.
• Synchronization : Threads often need to coordinate actions because they can get "race conditions" (outcome dependent on order of thread execution). You will often need to use locking primitives (mutual exclusion locks, semaphores, etc...)

Python Threads

Python supports threads on the following platforms : Solaris, Windows, systems that support the POSIX threads library (pthreads).

Thread scheduling is tightly controlled by a global interpreter lock and scheduler. Only a single thread is allowed to be executing in the Python interpreter at once. Thread switching only occurs between the execution of individual byte-codes. Long-running calculations in C/C++ can block execution of all other threads. However, most I/O operations do not block.

Python threads are somewhat more restrictive than in C. Effectiveness may be limited on multiple CPUs (due to interpreter lock). Threads can interact strangely with other Python modules (especially signal handling). Not all extension modules are thread-safe.

The thread module

The thread module provides low-level access to threads, thread creation, and Simple mutex locks.

Creating a new thread

Thread.start_new_thread(func,[args [,kwargs]]) Executes a function in a new thread. Syntax like:

import thread

import time

def print_time(delay):

    while 1:

    time.sleep(delay)

    print time.ctime(time.time())

thread.start_new_thread(print_time,(5,))      # Start the thread

# Go do something else

statements

…….

The function print_time will execute in a separate thread and will continue printing the time every 5 seconds. Python will continue executing our statements also.

Thread termination

Thread silently exits when the function returns. Thread can explicitly exit by calling thread.exit() or sys.exit(). Also uncaught exception causes thread termination (and prints error message). Other threads continue to run even if one had an error.

Simple locks

allocate_lock(). Creates a lock object, initially unlocked. Only one thread can acquire the lock at once. Threads block indefinitely until lock becomes available.

import thread

lk = thread.allocate_lock()

def foo():

    lk.acquire()         # Acquire the lock

    …                     #critical section

    lk.release()         # Release the lock

You might use this if two or more threads were allowed to update a shared data structure.

The main thread

When Python starts, it runs as a single thread of execution. This is called the "main thread." which is on its own and it’s not a big deal. However, if you launch other threads it has some special properties.

Termination of the main thread

If the main thread exits and other threads are active, the behavior is system dependent. Usually, this immediately terminates the execution of all other threads without cleanup. Cleanup actions of the main thread may be limited as well.

Signal handling

Signals can only be caught and handled by the main thread of execution. Otherwise you will get an error (in the signal module). The keyboard-interrupt can be caught by any thread (non-deterministically).

The threading module

The threading module is a high-level threads module that implements threads as classes (similar to Java). It provides an assortment of synchronization and locking primitives. It is built using the low-level thread module.

Creating a new thread (as a class)

When defining threads as classes all you need to supply is the following:

1. A constructor that calls threading.Thread.__init__(self)
2. A run() method that performs the actual work of the thread.

A few additional methods are also available

• t.join([timeout])    # Wait for thread t to terminate
• t.getName()          # Get the name of the thread
• t.setName(name)   # Set the name of the thread
• t.isAlive()             # Return 1 if thread is alive.
• t.isDaemon()         # Return daemonic flag
• t.setDaemon(val)   # Set daemonic flag

Example: Inherit from the "Thread" class, provide required methods, and utilize the available methods.

import threading, time

class PrintTime(threading.Thread):

    def __init__(self,interval):

        threading.Thread.__init__(self)         # Required

        self.interval = interval

    def run(self):

        while 1:

        time.sleep(self.interval) 

        print time.ctime(time.time())

t = PrintTime(5)                                   # Create a thread object

t.start()                                             # Start it

Daemon threads

Normally, interpreter exits only when all threads have terminated. However, a thread can be flagged as a daemon thread (runs in background). Interpreter really only exits when all non-daemonic threads exit. You can use this to launch threads that run forever, but which can be safely killed.

Threads synchronization

The threading module provides the following synchronization primitives:

• Mutual exclusion locks
• Reentrant locks
• Conditional variables
• Semaphores
• Events

When would you need these threads synchronization mechanisms ?

• When threads are updating shared data structures.
• When threads need to coordinate their actions in some manner (events).

The Lock object

Provides a simple mutual exclusion lock. Only one thread is allowed to acquire the lock at once. Most useful for coordinating access to shared data.

import threading

data = [ ]                            # Some data

lck = threading.Lock()            # Create a lock

def put_obj(obj):

    lck.acquire()

    data.append(obj)

    lck.release()

def get_obj():

    lck.acquire()

    r = data.pop()

    lck.release()

    return r

The RLock object

A mutual-exclusion lock that allows repeated acquisition by the same thread. Allows nested acquire(), release() operations in the thread that owns the lock. Only the outermost release() operation actually releases the lock.

import threading

data = [ ]                  # Some data

lck = threading.Lock()  # Create a lock

def put_obj(obj):

    lck.acquire()

    data.append(obj)

    ...

    put_obj(otherobj)   # Some kind of recursion

    ...

    lck.release()

def get_obj():

    lck.acquire()

    r = data.pop()

    lck.release()

    return r

The Condition object

Creates a condition variable. Synchronization primitive typically used when a thread is interested in an event or state change. Could help in the producer-consumer classic problem.

data = []                      # Create data queue and a condition variable

cv = threading.Condition()

# Consumer thread

def consume_item():

    cv.acquire()              # Acquire the lock

    while not len(data):

        cv.wait()             # Wait for data to show up

    r = data.pop()

    cv.release()             # Release the lock

    return r

# Producer thread

def produce_item(obj):

    cv.acquire()            # Acquire the lock

    data.append(obj)

    cv.notify()              # Notify a consumer

    cv.release()           # Release the lock

Semaphores

A locking primitive based on a counter. Each acquire() method decrements the counter. Each release() method increments the counter. If the counter reaches zero, future acquire() methods block. Common use: limiting the number of threads allowed to execute code

sem = threading.Semaphore(5)      # No more than 5 threads allowed

def fetch_file(host,filename):

    sem.acquire()                         # Decrements count or blocks if zero

    ...

    sem.release()                         # Increment count

Events

A communication primitive for coordinating threads. One thread signals an "event" while other threads wait for it to happen.

e = Event()                       # Create an event object

def signal_event():             # Signal the event

     e.set()

def wait_for_event():         # Wait for event

    e.wait()

def clear_event():             # Clear event

    e.clear()

Event is similar to a condition variable, but all threads waiting for event are awakened.

Locks and Blocking

By default, all locking primitives block until lock is acquired. In general, this is uninterruptible. Fortunately, most primitives provide a non-blocking option

if not lck.acquire(0):

     # lock couldn’t be acquired!

This works for Lock, RLock, and Semaphore objects. On the other hand condition variables and events provide a timeout option

cv = Condition()

...

cv.wait(60.0)                  # Wait 60 seconds for notification

On timeout, the function simply returns. Up to caller to detect errors.

The Queue Module

Provides a multi-producer, multi-consumer FIFO queue object. It can be used to safely exchange data between multiple threads.

• q = Queue(maxsize)    # Create a queue.
• q.qsize()                   # Return current size.
• q.empty()                 # Test if empty.
• q.full()                     # Test if full.
• q.put(item)               # Put an item on the queue.
• q.get()                    # Get item from queue

The Queue object also supports non-blocking put/get.

• q.put_nowait(item).
• q.get_nowait()

These raise the Queue.Full or Queue.Empty exceptions if an error occurs. Return values for qsize(), empty(), and full() are approximate.

Things to consider when using threads

• Global interpreter lock makes it difficult to fully utilize multiple CPUs.
• You don’t get the degree of parallelism you might expect.
• Not all modules are thread-friendly. Example: gethostbyname() blocks all threads if nameserver down.

In this note we will talked about threading, threads communication and synchronization. In the upcoming notes, we will talk about more advanced topics in Python programming.

Python Notes – 9 : Serialization

Welcome to our ninth note in our Python learning process. We talked previously about files and how to handle it but we talked about writing and reading only the primitive data types as integers and strings. We also talked about objects and classes. Now, what if we want to write a compound data type or a complex object to a file. This note will talk about writing objects to files, which is called object serialization.

pickle

The pickle module is a Python built-in module that object serialization and de-serialization. To store a data structure, use the dump method and then close the file in the usual way:

>>> pickle.dump(12.3, f)

>>> pickle.dump([1,2,3], f)

>>> f.close()

Then we can open the file for reading and load the data structures we dumped:

>>> f = open("test.pck","r")

>>> x = pickle.load(f)

>>> x

12.3

>>> type(x)

<type 'float'>

>>> y = pickle.load(f)

>>> y

[1, 2, 3]

>>> type(y)

<type 'list'>

Each time we invoke load, we get a single value from the file, complete with its original type.

What can be serialized and de-serialized

The following types can be serialized and de-serialized using pickle:

• None, True, and False
• integers, long integers, floating point numbers, complex numbers
• normal and Unicode strings
• tuples, lists, sets, and dictionaries containing only picklable objects
• functions defined at the top level of a module
• built-in functions defined at the top level of a module
• classes that are defined at the top level of a module
• instances of such classes whose__dict__ or __setstate__() is picklable

Things to consider when using pickle

• Attempts to pickle unpicklable objects will raise the picklingError exception; when this happens, an unspecified number of bytes may have already been written to the underlying file.
• Trying to pickle a highly recursive data structure may exceed the maximum recursion depth, a RuntimeError will be raised in this case. You can carefully raise this limit with sys.setrecursionlimit().

cPickle

The cPickle is an optimized version of pickle written in C, so it can be up to 1000 faster than pickle.

marshal

The marshal module can also be used for serialization. Marshal is similar to pickle, but is intended only for simple objects. Can’t handle recursion or class instances. On the plus side, it’s pretty fast if you just want to save simple objects to a file. Data is stored in a binary architecture independent format.To serialize:

import marshal

marshal.dump(obj,file)                  # Write obj to file

To unserialize:

obj = marshal.load(file)

shelve

The shelve module provides a persistent dictionary. It is works like a dictionary, but data is stored on disk.

Keys must be strings. Data can be any object serializable with pickle.

import shelve

d = shelve.open("data") # Open a ’shelf’

d[’foo’] = 42 # Save data

x = d[’bar’] # Retrieve data

Shelf operations

d[key] = obj              # Store an object

obj = d[key]              # Retrieve an object

del d[key]                 # Delete an object

d.has_key(key)          # Test for existence of key

d.keys()                   # Return a list of all keys

d.close()                  # Close the shelf

In this note will talked about writing objects to files, which is called object serialization. Object serialization is very useful in persisting your application logic to resume its execution later, transfer of execution to remote machine, and many other applications scenarios.

Python Notes – 8 : Object-Oriented Basics

Welcome to our eighth note in our Python learning process. This note will talk about object oriented features in Python.

Classes and Objects

A class definition looks like this:

class Point:

    pass

Class definitions can appear anywhere in a program, but they are usually near the beginning (after the import statements). By creating the Point class, we created a new type, also called Point. The members of this type are called instances of the type or objects. Creating a new instance is called instantiation. To instantiate a Point object, we call a function named Point:

>>> blank = Point()

The variable blank is assigned a reference to a new Point object. A function like Point that creates new objects is called a constructor. If you tried to get the type of blank, you got instance:

>>> type(blank)

<type 'instance'>

If you tried to print blank:

>>> print blank

<__main__.point instance at 0x01922AF8>

The result indicates that blank is an instance of the Point class and it was defined in __main__ . 0x01922AF8 is the unique identifier for this object, written in hexadecimal (base 16).

Attributes

We can add new data to an instance using dot notation:

>>> blank.x = 3.0

>>> blank.y = 4.0

These new data items called attributes.

>>> print blank.y

4.0

>>> x = blank.x

>>> print x

3.0

Sameness

To find out if two references refer to the same object, use the == operator. For example:

>>> p1 = Point()

>>> p1.x = 3

>>> p1.y = 4

>>> p2 = Point()

>>> p2.x = 3

>>> p2.y = 4

>>> p1 == p2

0

Even though p1 and p2 contain the same coordinates, they are not the same object. If we assign p1 to p2, then the two variables are aliases of the same object:

>>> p2 = p1

>>> p1 == p2

1

This type of equality is called shallow equality because it compares only the references, not the contents of the objects. To compare the contents of the objects - deep equality - we can write our own function to do that, like that:

def samePoint(p1, p2) :

    return (p1.x == p2.x) and (p1.y == p2.y)

Now if we create two different objects that contain the same data, we can use samePoint to find out if they represent the same point.

>>> p1 = Point()

>>> p1.x = 3

>>> p1.y = 4

>>> p2 = Point()

>>> p2.x = 3

>>> p2.y = 4

>>> samePoint(p1, p2)

1

Copying

Aliasing can make a program difficult to read because changes made in one place might have unexpected effects in another place. Copying an object is often an alternative to aliasing. The copy module contains a function called copy that can duplicate any object:

>>> import copy

>>> p1 = Point()

>>> p1.x = 3

>>> p1.y = 4

>>> p2 = copy.copy(p1)

>>> p1 == p2

0

>>> samePoint(p1, p2)

1

Copy works fine for objects that doesn't contain any embedded objects. If the object contains references to other objects, Copy will copy the embedded references to the destination. This ends up that the both copies reference the same internal objects.

You can use deepcopy which copies not only the object but also any embedded objects.

>>> b2 = copy.deepcopy(b1)

Now b1 and b2 are completely separate objects.

The initialization method

The initialization method is a special method that is invoked when an object is created. The name of this method is __init__.

class point:

    def __init__(self, x = 0, y = 0):

        Self.x = x

        Slef.y = y

When we invoke the point constructor, the arguments we provide are passed along to init:

>>> first = point(5,7)

>>> first.x

5

>>> first.y

7

Because the parameters are optional, we can omit them:

>>> second = point()

>>> second.x

0

>>> second.y

0

We can also provide a subset of the parameters by naming them explicitly:

>>> third = point(y=10)

>>> third.x

0

>>> third.y

10

The __str__ method

The __str__ method of any class is called by the Python in any operation that requires the class instance to be converted to string. Operations like that are print. Syntax like that:

class xyz:

    def __str__(self):

        return "Our class xyz"

>>> a = xyz()

>>> a

<__main__.xyz instance at 0x02627300>

>>> print y

Our class xyz

Instances as parameters

You can pass an instance as a parameter in the usual way. For example:

def printPoint(p):

    print '(' + str(p.x) + ', ' + str(p.y) + ')'

Instances as return values

Functions can return instances. For example:

def sumPoints(A,B)

    Z = Point ()

    Z.x = A.x + B.x

    Z.y = A.y + B.y

    return Z

Operator overloading

Operator overloading means changing the definition and behavior of the built-in operators when they are applied to user-defined types. For example, to override the addition operator + , we provide a method named __add__ in our point class :

class Point:

    def __add__(self, other):

        return Point(self.x + other.x, self.y + other.y)

the first parameter is the object on which the method is invoked. The second parameter is conveniently named other to distinguish it from self. Now, when we apply the + operator to Point objects, Python invokes add :

>>> p1 = Point(3, 4)

>>> p2 = Point(5, 7)

>>> p3 = p1 + p2

>>> print p3

(8, 11)

The expression p1 + p2 is equivalent to p1. add (p2), but obviously more elegant. You can change the behavior of many operators through overloading their respective functions, which are available at http://www.python.org/doc/2.2/ref/numeric-types.html

Inheritance

Inheritance is the ability to define a new class that is a modified version of an existing class. The new class inherits all of the methods of the existing class. The new class may be called child class or subclass. The syntax is like:

class class1(object):

    K = 7

    def __init__(self, color='green'):

        Self.color = color

    def Hello1(self):

        Print "Hello from class1"

    def printColor(self):

        print "preferred ", self.color

class class2(class1):

    def Hello2(self):

    print "Hello from class2"

    print self.k

Here class2 is the child of class1.

>>> c1 = class1('blue')

>>> c2 = class2('red')

>>> c1.Hello1()

Hello from class1

>>> c2.Hello2()

Hello from class2

7

Child class can access parent class methods

>>> c2.Hello1()

Hello from class1

The parent constructor called automatically for Childs, as following:

>>> c1.printColor()

preferred blue

>>> c2.printColor()

preferred red

You can check for class methods, attributes using hasattr method:

if hasattr(class1, "Hello2"):

    print c1.Hello2()

else:

    print "Class1 does not contain method Hello2()"

Class1 does not contain method Hello2()

To check the inheritance relation between two class :

if issubclass(class2, class1)

     Print "Class2 is a subclass of Class1”

In this note we tried to cover as much as we can of the Python object oriented features. We give it a more advanced note in the future.

Python Notes – 7 : Files & directories

Welcome to our seventh note in our Python learning process. This note will talk specifically about files, directories, and exceptions.

Files

Opening a file creates a file object. Syntax is like that:

>>> f = open("test.dat","w")

>>> print f

<open file 'test.dat', mode 'w' at fe820>

The first parameter to open is the file name, the second parameter is the mode. Modes are: w for write, r for read.

To write data in the file we invoke the write method on the file object:

>>> f.write("Now is the time")

>>> f.write("to close the file")

After we done we can close the file like that:

>>> f.close()

The read method reads data from the file. With no arguments, it reads the entire contents of the file:

>>> text = f.read()

>>> print text

Now is the time to close the file

read can also take an argument that indicates how many characters to read.

If not enough characters are left in the file, read returns the remaining characters. When we get to the end of the file, read returns the empty string:

>>> print f.read(5)

Now i

>>> print f.read(1000006)

s the timeto close the file

>>> print f.read()

>>>

The write method write data to the file. It takes strings only.

>>> x = 52

>>> f.write (str(x))

Directories

When you create a new file by opening it and writing, the new file goes in the current directory (wherever you were when you ran the program). Similarly, when you open a file for reading, Python looks for it in the current directory. If you want to open a file somewhere else, you have to specify the path to the file, which is the name of the directory (or folder) where the file is located:

>>> f = open("/usr/share/dict/words","r")

>>> print f.readline()

Whatever exist

glob module

Returns filenames in a directory that match a pattern.

import glob

a = glob.glob("*.html")

b = glob.glob("image[0-5]*.gif")

Pattern matching is performed using rules of Unix shell. Tilde (~) and variable expansion is not performed.

fnmatch module

Matches filenames according to rules of Unix shell.

import fnmatch

if fnmatch(filename,"*.html"):

...

Case-sensitivity depends on the operating system.

Other File-Related Modules

• fcntl : Provides access to the fcntl() system call and file-locking operations

import fcntl, FCNTL

fcntl.flock(f.fileno(),FCNTL.LOCK_EX) # Lock a file

• tempfile : Creates temporary files

• gzip : Creates file objects with compression/decompression. Compatible with the GNU gzip program.

import gzip

f = gzip.open("foo","wb")

f.write(data)

Exceptions

Whenever a runtime error occurs, it creates an exception. Usually, the program stops and Python prints an error message.

>>> print 55/0

ZeroDivisionError: integer division or modulo

We can handle the exception using the try and except statements.

filename = raw_input('Enter a file name: ')

try:

f = open (filename, "r")

except:

print 'There is no file named', filename

The try statement executes the statements in the first block. If no exceptions occur, it ignores the except statement. If any exception occurs, it executes the statements in the except branch and then continues.

If your program detects an error condition, you can make it raise an exception.

def inputNumber () :

x = input ('Pick a number: ')

if x == 17 :

raise 'BadNumberError', '17 is a bad number'

return x

The raise statement takes two arguments: the exception type and specific

information about the error. And here how the previous example appears:

>>> inputNumber ()

Pick a number: 17

BadNumberError: 17 is a bad number

This note talked about files, directories, and exceptions. The next note will go into the object oriented features of Python.

Python Notes – 6 : Tuples & Dictionaries

Welcome to our sixth note in our Python learning process. In this note we will talk about tuples and dictionaries.

Tuples

Tuples are similar to lists but its elements can't be modified. Syntactically, a tuple is a comma-separated list of values:

>>> tuple = 'a', 'b', 'c', 'd', 'e'

Although it is not necessary, it is conventional to enclose tuples in parentheses:

>>> tuple = ('a', 'b', 'c', 'd', 'e')

To create a tuple with a single element, we have to include the ¯nal comma:

>>> t1 = ('a',)

>>> type(t1)

<type 'tuple'>

Without the comma, Python treats ('a') as a string in parentheses:

>>> t2 = ('a')

>>> type(t2)

<type 'string'>

Tuple assignment

Tuples assignment could be used to perform multiple assignments in one statement. For example, to swap a and b we can make it like that:

>>> a, b = b, a

The left side is a tuple of variables; the right side is a tuple of values. Each value is assigned to its respective variable.

Dictionaries

Dictionaries are like lists and tuples in that all of them are compound types.

Dictionaries are different than lists and tuples in that it can use any immutable type as an index not only integers like tuples and lists. Dictionaries are not ordered.

To create an empty dictionary and then add elements to it:

>>> eng2sp = {}

>>> eng2sp['one'] = 'uno'

>>> eng2sp['two'] = 'dos'

>>> print eng2sp

{'one': 'uno', 'two': 'dos'}

You can also create the dictionary by providing a list of key-value pairs:

>>> eng2sp = {'one': 'uno', 'two': 'dos', 'three': 'tres'}

>>> print eng2sp

{'one': 'uno', 'three': 'tres', 'two': 'dos'}

Dictionary Operations

del statement removes a key-value pair from a dictionary.

>>> inventory = {'apples': 430, 'bananas': 312, 'oranges': 525, 'pears': 217}

>>> print inventory

{'oranges': 525, 'apples': 430, 'pears': 217, 'bananas': 312}

>>> del inventory['pears']

>>> print inventory

{'oranges': 525, 'apples': 430, 'bananas': 312}

The len function also works on dictionaries; it returns the number of key-value pairs:

>>> len(inventory)

4

Dictionary methods

The keys method takes a dictionary and returns a list of the keys.

>>> eng2sp.keys()

['one', 'three', 'two']

The values method is similar; it returns a list of the values in the dictionary:

>>> eng2sp.values()

['uno', 'tres', 'dos']

The items method returns both, in the form of a list of tuples|one for each key-value pair:

>>> eng2sp.items()

[('one','uno'), ('three', 'tres'), ('two', 'dos')]

The method has_key takes a key and returns true (1) if the key appears in the dictionary:

>>> eng2sp.has_key('one')

1

>>> eng2sp.has_key('deux')

0

Aliasing and copying

Dictionaries are objects that can be aliased like that:

>>> opposites = {'up': 'down', 'right': 'wrong', 'true': 'false'}

>>> alias = opposites

Or copied to a new fresh copy like that:

>>> copy = opposites.copy()

This note completed the compound data types talk we started before with lists. Lists, tuples, and dictionaries are the compound data types that Python provides. You can use them as is or you can use them to build a more complex data types and containers.

Python Notes – 5 : Objects & Values

Welcome to our fifth note in our Python learning process. In this note we will talk about one of the core concepts of the Python language semantics .

Objects and Values

An object is something a variable can refer to. Every object has a unique identifier, which we can obtain with the id function.

>>> a = "banana"

>>> b = "banana"

>>> id(a)

135044008

>>> id(b)

135044008

Interestingly, lists behave differently. When we create two lists, we get two

objects:

>>> a = [1, 2, 3]

>>> b = [1, 2, 3]

>>> id(a)

135045528

>>> id(b)

135041704

Since both a and b refer to the same object, changes made to the object through one of them appear when you access the object through the other. This is called aliasing.

>>> a = [1, 2, 3]

>>> b = [1, 2, 3]

>>> b[0] = 5

>>> print a

[5, 2, 3]

This brief note is just to differentiate between values and objects. Objects not only the instances we create of classes, some language types treated as objects and others treated as values.

We will discuss object oriented in a separate note within days.

Python Notes – 4 : Lists

Welcome to our third note in our Python learning process. In this note we will talk mainly about lists, its functions and how to use it.

Lists - Creation

A list is an ordered set of values, where each value is identified by an index. The values that make up a list are called its elements. You could create it like:

>>> X = [12, 56, 87]

>>> print X

[12, 56, 87]

>>> Y = ["one", "three", "five"]

>>> print Y

["one", "three", "five"]

The list elements don't have to be the same type.

["hello", 2.0, 5, [10, 20]]

A list within another list is said to be nested.

Another way of creating special list of consecutive integers:

>>> range(1,5)

[1, 2, 3, 4]

If there is no start of the range, it will create starting from the 0

>>> range(10)

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

If there is a third argument, it specifies the space between successive values,

which is called the step size

>>> range(1, 10, 2)

[1, 3, 5, 7, 9]

Lists - Accessing elements

You access list elements by index. Indices starts with zero. Indices could be any integer expression.

>>> numbers = range(1,10)

>>> print numbers[0]

1

>>> print numbers[5-3]

3

Lists - Length

You can get the list length using len() method. Syntax like that:

>>> len(numbers)

9

Although a list can contain another list, the nested list still counts as a single

element. The length of this list is four:

['spam!', 1, ['Brie', 'Roquefort', 'Pol le Veq'], [1, 2, 3]]

Lists - operators

in is a boolean operator that tests membership in a sequence.

>>> horsemen = ['war', 'famine', 'pestilence', 'death']

>>> 'pestilence' in horsemen

1

>>> 'debauchery' in horsemen

0

Lists - for loops

The syntax is like:

for VARIABLE in LIST:

BODY

for horseman in horsemen:

print horseman

This is equivalent to:

i = 0

while i < len(LIST):

VARIABLE = LIST[i]

BODY

i = i + 1

Any list expression can be used in a for loop:

for number in range(20):

if number % 2 == 0:

print number

List - operations

The + operator concatenates lists:

>>> a = [1, 2, 3]

>>> b = [4, 5, 6]

>>> c = a + b

>>> print c

[1, 2, 3, 4, 5, 6]

Similarly, the * operator repeats a list a given number of times:

>>> [0] * 4

[0, 0, 0, 0]

>>> [1, 2, 3] * 3

[1, 2, 3, 1, 2, 3, 1, 2, 3]

The head function takes a list as a parameter and returns the first element.

>>> a = [1, 2, 3]

>>> head(a)

1

List - slices

>>> list = ['a', 'b', 'c', 'd', 'e', 'f']

>>> list[1:3]

['b', 'c']

>>> list[:4]

['a', 'b', 'c', 'd']

>>> list[3:]

['d', 'e', 'f']

>>> list[:]

['a', 'b', 'c', 'd', 'e', 'f']

List - deletion

del removes an element from a list:

>>> a = ['one', 'two', 'three']

>>> del a[1]

>>> a

['one', 'three']

You can use a slice as an index for del:

>>> list = ['a', 'b', 'c', 'd', 'e', 'f']

>>> del list[1:5]

>>> print list

['a', 'f']

• Strings are lists and can be treated the same way you treat lists.

Nested lists

A nested list is a list that appears as an element in another list. In this list, the

three-eth element is a nested list:

>>> list = ["hello", 2.0, 5, [10, 20]]

To extract an element from the nested list, we can proceed in two steps:

>>> elt = list[3]

>>> elt[0]

10

Or we can combine them:

>>> list[3][1]

20

In this note we talked mainly lists which are very powerful feature in Python. We will continue our learning in the upcoming notes.

Python Notes – 3 : Control Statements

Welcome to our third note in our Python learning process. In this note we will talk about conditional statements, iteration and keyboard input.

Conditional Statements

The normal flow of statements execution is sequential where the interpreter executes the statements in the same sequence that it appears in the script. If your program logic requires a different flow than the that, you should use conditional statements. The syntax looks like:

if x%2 == 0:

print x, "is even"

else:

print x, "is odd"

The statement indentation is the only statement scope identifier.

Conditional statements can be chained to whatever level you want. The syntax looks like:

if x < y:

print x, "is less than", y

elif x > y:

print x, "is greater than", y

else:

print x, "and", y, "are equal"

elif is an abbreviation of "else if."

Conditional statements can be nested to whatever level you want. Take care of indentation. The syntax looks like:

if x == y:

print x, "and", y, "are equal"

else:

if x < y:

print x, "is less than", y

else:

print x, "is greater than", y

Iteration

Iteration is the way to execute some statements multiple times. The syntax is like that:

While CONDITION:

STATEMENTS IN SCOPE

STATEMENTS IN SCOPE

STATEMENTS OUT OF WHILE SCOPE

The while will execute the in scope statements until the CONDITION evaluates to false. Example:

X = 10

While X != 0:

Print X

X = X - 1

print "Finish"

This will print numbers from 10 to 1 then "Finish".

Keyboard input

Python provides built-in functions that get input from the keyboard. The simplest is called raw input. When this function is called, the program stops and waits for the user to type something. When the user presses Return or the Enter key, the program resumes and raw input returns what the user typed as

a string:

>>> input = raw_input ()

What are you waiting for?

>>> print input

What are you waiting for?

In this note we talked about conditional statements, iteration and keyboard input. These are another way of organizing your program logic to satisfy accomplish your program goal. We will continue our learning in the upcoming notes.

Python Notes – 2 : Variables, Statements, Expressions, Operators, and Functions

Welcome to our second note in our Python learning process. In this note we will talk about variables, statements, expressions, operators, comments, and functions. These are the very basic building blocks of you programs whatever its final size

Variables

The assignment statement creates new variables and gives them values

>>> message = "What's up, Doc?"

>>> n = 17

>>> pi = 3.14159

Python is a dynamically typed language. Which means that variables' types don't have to be defined before the variables use. The python interpreter figure out what type a variable is when you first assign it a value.

Variables naming

• Variable name can contain letters, numbers, and underscore.
• Variable name must begin with letter.
• Variable names are case-sensitive ( x different than X ).
• Variable names can't be one of the Python language reserved words, which are :

and	continue	else	for	import	not	raise
assert	def	except	from	in	or	return
break	def	exec	global	is	pass	try
class	elif	finally	if	lambda	print	while

Statements

A statement is an instruction that the Python interpreter can execute. A Python script is a sequence of statements. The results appear one at a time as the statements execute.

For example, the script

>>> print 1

>>> 1

>>> x = 2

>>> print x

>>> 2

Expressions

An expression is a combination of values, variables, and operators. If you type an expression on the command line, the interpreter evaluates it and displays the result:

>>> 1 + 1

2

Boolean Expressions

A boolean expression is an expression that is either true or false. Boolean expressions are expressions that uses logical operators. There are three logical operators: and, or, not. The operands of logical operands should be boolean expressions. In Python there is no boolean data type instead 0, '', [], (), {}, and None are false in a boolean context; everything else is true. The absence of boolean data types results in the following behavior of logical operators:

• All the behaviors depends on the order of expressions evaluation, which is from left to right.
• and : returns the first false value, if all values are true, returns the last value.

>>> 'a' and 'b'

'b'

>>> ' ' and 'b'

' '

>>> 'a' and 'b' and 'c'

'c'

• or : returns the first true value, if all values are false, returns the last value

>>> 'a' or 'b'

'a'

>>> ' ' or 'b'

'b'

>>> ' ' or [] or {}

{}

• not : evaluates the boolean expression that follows it and inverts it whole value.

>>> not (1 and 0)

True

>>> not (1 or 0)

False

Operators and operands

Operators are special symbols that represent computations like addition and multiplication. The values the operator uses are called operands.

The symbols +, -, and /, and the use of parenthesis for grouping, mean in

Python what they mean in mathematics. The asterisk (*) is the symbol for multiplication, and ** is the symbol for exponentiation.

Order of operations

When more than one operator appears in an expression, the order of evaluation depends on the rules of precedence. The following is operators from highest precedence to lower precedence:

• Parentheses
• Exponentiation
• Multiplication / Division
• Addition / Subtraction

Operators with the same precedence are evaluated from left to right.

Comments

Comments start with #.

Built-in Functions

Here are some of the built-in functions available in Python:

• Type conversion functions which converts values from one type to another. Like:
  • int() :

>>> int("32")

32

>>> int("Hello")

ValueError: invalid literal for int(): Hello

• float():

>>> float(32)

32.0

>>> float("3.14159")

3.14159

• str():

>>> str(32)

'32'

>>> str(3.14149)

'3.14149'

• Math functions which provides most of the familiar mathematical functions. These functions are implemented in a module called math, we will talk about modules soon. But for now, if you want to use these functions, you have to import the math module. This is very straight foreword, using following statement
  • >>> import math
  • After the import statement, you can simply call the math functions.

>>> decibel = math.log10 (17.0)

>>> angle = 1.5

>>> height = math.sin(angle)

>>> degrees = 45

>>> angle = degrees * 2 * math.pi / 360.0

>>> math.sin(angle)

0.707106781187

>>> math.sqrt(2) / 2.0

0.707106781187

There are many other built-in functions, we mentioned these two types just as examples.

Creating your own functions

A function is a named sequence of statements that performs a desired operation. This operation is specified in a function definition.

The syntax for a function definition is:

def FUNCTION_NAME( LIST OF PARAMETERS ):

STATEMENTS

• def is a keyword.
• FUNCTION_NAME could be anything except Python keywords.
• LIST OF PARAMETERS are the ordered list parameters the function needs to operate. The empty parentheses indicate no parameters needed.
• The function body doesn't have starting or ending characters (like "{" and "}" in C like languages ). Indentation is the only scope identifier for statements in Python. The first statement that doesn't follow the indentation, will be considered out of the function and will mark the function termination.

# just to show the function scope and structure

def sample_func():

    Print "we are in the function body scope"

    Print "we still in the function scope"

Print "we are out of the function scope"

# another function with parameters

def sum(x, y):

    print x + y

• To call a function just use the function name followed by the list of parameters between parentheses

>>> sample_func()

>>> sum(1,2)

3

• Functions can call other functions in their body.
• Functions can return values to its caller

def sum(x,y):

return x +y

def print_sum(x,y):

print sum(x,y)

print_sum(2,3)

5

• When you create a local variable inside a function, it only exists inside the function, and you cannot use it outside. The following will generate a run-time error when trying to access Z variable.

def sum(x,y):

Z = x +y

print Z # will generate a runtime error

Returning multiple values

def divide(a,b):

q = a/b

r = a - q*b

return q,r

>>> x,y = divide(42,5) # x = 8, y = 2

>>> x

8

>>> y

2

In this note we talked about variables, statements, expressions, operators, comments, and functions. These are the very basic building blocks of you programs whatever its final size. We will continue our learning in the upcoming notes.

Python Notes – 1 : Setup

I write this notes about Python during my learning process of it. I'm not an expert in Python. I'm just writing this notes while I learn Python aiming to help others who are learning it too. Never hesitate to comment on anything you want.

Origins of Python

• Developed by Guido van Rossum.
• Derived from ABC, a teaching language that was developed in 1980s.

Why Pyhton ?

• Python code is pretty simple, compact and easy to learn.
• Code simplicity let you focus on the core program functionality.
• Suitable for programming languages introductory courses.
• Provides a balance between the practical and the conceptual. You start writing after the first tutorial; and if you want to go further in advanced topics you will find a large library of modules that can be used to do all sorts of tasks ranging from web-programming to graphics.
• Python borrows features from both functional programming languages and object-oriented programming languages, which enables it to serve as an excellent foundation for introducing important computer science concepts.
• Python allow you to see higher level of success and a lower level of frustration :)

How to setup Python ?

Python is free downloadable from many source, and it have also many IDEs and editors. Aiming to start coding quickly without wasting time in IDE exploration, setup, configuration problems; we will use just the ActivePython. Let's see how to do that:

1. Download ActivePython, it is free.
2. After you install ActivePython you will got that the following change in your start menu. We will not use Python Intercative Shell nor PythonWin Editor in our first steps, we will use them in upcoming notes.

     

1. In our first note we will use Python from the Windows Command window. To do that we need to add the Python installation path to the Environment Variables of the operating system.
2. If you are using Windows XP or Windows 2000 do the following (if you are using Windows Vista go for step 5):
   1. Right-click My Computer and select Properties from the displayed context menu. You will got the System Properties dialog

     

1. Then click the Advanced Tab

     

1. Then click the Environment Variables button near the left bottom. You will got the Environment variables dialog

     

1. Select the variable path from the second list called System Variables. Then click Edit button, you will got the Edit System Variable dialog box

     

1. Append the Python installation path to the string in Variable value textbox. The default installation path for ActivePython in our case is C:\Python25\; the semicolon ';' is not part of the path but is mandatory to be added at the end. This semicolon acts as the delimiter that enables the operating system to differentiate between this list of paths.

1. If you using Windows Vista, do the following (very similar steps):
   1. Right-click My Computer and select Properties from the displayed context menu. You will got the System Properties dialog

     

1. Then the link Advanced system settings on the left menu, you will got the System Properties dialog box

     

1. Then click the Advanced tab

     

1. Click the Environment Variables button near the right bottom, you will got Environment Variables dialog box

     

1. Select the variable path from the second list called System Variables. Then click Edit button, you will got the Edit System Variable dialog box

     

1. Append the Python installation path to the string in Variable value textbox. The default installation path for ActivePython in our case is C:\Python25\; the semicolon ';' is not part of the path but is mandatory to be added at the end. This semicolon acts as the delimiter that enables the operating system to differentiate between this list of paths.

Now, we have Python installed and configured on our machines.

Let's begin in Python

Python is a high-level programming language. It is interpreted language, which means that its programs are executed by an interpreter rather than a compiler. There are two ways to use the interpreter:

• Command-line mode where you type your python programs at the command-line and the interpreter prints the result.
  • Open the windows command window and type python, to invoke the python interpreter.
  • Type print "Hello world !" and press enter. Guess what, you have wrote your first program in Python :)

      

• Script mode where you write your program in a file and use the interpreter to execute the contents of the file. Such a file called script. By convention python files have extension .py.
  • To try that mode, create a file script1.py in your Python installation folder ( default is : C:\Python25 ).
  • In this file write print "Hello world !" , save and close.
  • Open the windows command window and type python, to invoke the python interpreter.
  • Type import script1 and press enter. You just ran your first Python program.

      

We will use the script mode in most of our notes.

This is just the beginning of our notes about Python. Now we have Python installed and configured on our machines. We will continue our learning in the upcoming notes.