History

Python is a portable, interpreted, object-oriented programming language. Its development started in 1990 at CWI in Amsterdam, and continues at CNRI in Reston, Va. The language has an elegant (but not over-simplified) syntax; a small number of powerful high-level data types are built in. Python can be extended in a systematic fashion by adding new modules implemented in a compiled language such as C or C++. Such extension modules can define new functions and variables as well as new object types.

The Python programming language provides an increasing amount of support for XML technologies. This document attempts to introduce some basic XML processing concepts to readers who have not yet started to use Python with XML, and it takes the form of a tutorial. It is assumed that the reader knows the basic terminology of XML and is comfortable with "XML as text".

The Python implementation is portable: it runs on many brands of UNIX, on Windows, DOS, OS/2,Mac, Amiga... If your favorite system isn't listed here, it may still be supported, if there's a C compiler for it. Ask around on comp.lang.python -- or just try compiling Python yourself.

Python with other Languages

Python is often compared to other interpreted languages such as Java, JavaScript, Perl, Tcl, or Smalltalk. Comparisons to C++, Common Lisp and Scheme can also be enlightening. In this section I will briefly compare Python to each of these languages. These comparisons concentrate on language issues only. In practice, the choice of a programming language is often dictated by other real-world constraints such as cost, availability, training, and prior investment, or even emotional attachment. Since these aspects are highly variable, it seems a waste of time to consider them much for this comparison.

Java

Python programs are generally expected to run slower than Java programs, but they also take much less time to develop. Python programs are typically 3-5 times shorter than equivalent Java programs. This difference can be attributed to Python's built-in high-level data types and its dynamic typing. For example, a Python programmer wastes no time declaring the types of arguments or variables, and Python's powerful polymorphic list and dictionary types, for which rich syntactic support is built straight into the language, find a use in almost every Python program. Because of the run-time typing, Python's run time must work harder than Java's. For example, when evaluating the expression a+b, it must first inspect the objects a and b to find out their type, which is not known at compile time. It then invokes the appropriate addition operation, which may be an overloaded user-defined method. Java, on the other hand, can perform an efficient integer or floating point addition, but requires variable declarations for a and b, and does not allow overloading of the + operator for instances of user-defined classes.

For these reasons, Python is much better suited as a "glue" language, while Java is better characterized as a low-level implementation language. In fact, the two together make an excellent combination. Components can be developed in Java and combined to form applications in Python; Python can also be used to prototype components until their design can be "hardened" in a Java implementation. To support this type of development, a Python implementation written in Java is under development, which allows calling Python code from Java and vice versa. In this implementation, Python source code is translated to Java bytecode (with help from a run-time library to support Python's dynamic semantics).

Javascript

Python's "object-based" subset is roughly equivalent to JavaScript. Like JavaScript (and unlike Java), Python supports a programming style that uses simple functions and variables without engaging in class definitions. However, for JavaScript, that's all there is. Python, on the other hand, supports writing much larger programs and better code reuse through a true object-oriented programming style, where classes and inheritance play an important role.

Perl

Python and Perl come from a similar background (Unix scripting, which both have long outgrown), and sport many similar features, but have a different philosophy. Perl emphasizes support for common application-oriented tasks, e.g. by having built-in regular expressions, file scanning and report generating features. Python emphasizes support for common programming methodologies such as data structure design and object-oriented programming, and encourages programmers to write readable (and thus maintainable) code by providing an elegant but not overly cryptic notation. As a consequence, Python comes close to Perl but rarely beats it in its original application domain; however Python has an applicability well beyond Perl's niche.

Tcl

Like Python, Tcl is usable as an application extension language, as well as a stand-alone programming language. However, Tcl, which traditionally stores all data as strings, is weak on data structures, and executes typical code much slower than Python. Tcl also lacks features needed for writing large programs, such as modular namespaces. Thus, while a "typical" large application using Tcl usually contains Tcl extensions written in C or C++ that are specific to that application, an equivalent Python application can often be written in "pure Python". Of course, pure Python development is much quicker than having to write and debug a C or C++ component. It has been said that Tcl's one redeeming quality is the Tk toolkit. Python has adopted an interface to Tk as its standard GUI component library. Tcl 8.0 addresses the speed issuse by providing a bytecode compiler with limited data type support, and adds namespaces. However, it is still a much more cumbersome programming language.

Smalltalk

Perhaps the biggest difference between Python and Smalltalk is Python's more "mainstream" syntax, which gives it a leg up on programmer training. Like Smalltalk, Python has dynamic typing and binding, and everything in Python is an object. However, Python distinguishes built-in object types from user-defined classes, and currently doesn't allow inheritance from built-in types. Smalltalk's standard library of collection data types is more refined, while Python's library has more facilities for dealing with Internet and WWW realities such as email, HTML and FTP. Python has a different philosophy regarding the development environment and distribution of code. Where Smalltalk traditionally has a monolithic "system image" which comprises both the environment and the user's program, Python stores both standard modules and user modules in individual files which can easily be rearranged or distributed outside the system. One consequence is that there is more than one option for attaching a Graphical User Interface (GUI) to a Python program, since the GUI is not built into the system.

C++
Almost everything said for Java also applies for C++, just more so: where Python code is typically 3-5 times shorter than equivalent Java code, it is often 5-10 times shorter than equivalent C++ code! Anecdotal evidence suggests that one Python programmer can finish in two months what two C++ programmers can't complete in a year. Python shines as a glue language, used to combine components written in C++.

Common Lisp and Scheme
These languages are close to Python in their dynamic semantics, but so different in their approach to syntax that a comparison becomes almost a religious argument: is Lisp's lack of syntax an advantage or a disadvantage? It should be noted that Python has introspective capabilities similar to those of Lisp, and Python programs can construct and execute program fragments on the fly. Usually, real-world properties are decisive: Common Lisp is big (in every sense), and the Scheme world is fragmented between many incompatible versions, where Python has a single, free, compact implementation. Moshe Zadka wrote a far better comparison with Scheme: Python vs. Scheme.

Graphical User Interface for Python

Tk

There's a neat object-oriented interface to the Tcl/Tk widget set, called Tkinter. It is part of the standard Python distribution and well-supported -- all you need to do is build and install Tcl/Tk and enable the _tkinter module and the TKPATH definition in Modules/Setup when building Python. This is probably the easiest to install and use, and the most complete widget set. It is also very likely that in the future the standard Python GUI API will be based on or at least look very much like the Tkinter interface.For more info about Tk, including pointers to the source, see the Tcl/Tk home page at http://www.scriptics.com/. Tcl/Tk is now fully portable to the Mac and Windows platforms (NT and 95 only); you need Python 1.4beta3 or later and Tk 4.1patch1 or later.
.

wxWindows:

There's an interface to wxWindows called wxPython. wxWindows is a portable GUI class library written in C++. It supports GTK, Motif, MS-Windows and Mac as targets. Ports to other platforms are being contemplated or have already had some work done on them. wxWindows preserves the look and feel of the underlying graphics toolkit, and there is quite a rich widget set and collection of GDI classes. See the wxWindows page at http://www.wxwindows.org// for more details. wxPython is a python extension module that wraps many of the wxWindows C++ classes, and is quickly gaining popularity amongst Python developers. You can get wxPython as part of the source or CVS distribution of wxWindows, or directly from its home page at http://alldunn.com/wxPython/.

COM has become a crucial part of the software infrastructure in the Windows 95 and Windows NT environments. But there are still plenty of uncertainties about the future. Will Microsoft succeed, for example, in making COM a viable multiplatform technology? Fitting Windows NT servers into existing enterprises will all but require that DCOM and other distributed services be available on non-Microsoft platforms.

The process has taken longer than expected, and while many organizations have made promises in this area, not much code is actually shipping. Meanwhile, both CORBA-based products and Java's RMI are successfully running in multi-OS environments today. The more time passes before multiplatform DCOM becomes a reality, the larger CORBA and RMI's lead will become.

Using Python as an Integration Language

Python is in use at many places as an integration language, used to glue together ("steer") existing components. The strategy here is to create Python extension modules (written in C/C++) that make the functionality of large components written in C/C++ available to the Python programmer. The extension ("glue") modules are required because Python cannot call C/C++ functions directly; the glue extensions handle conversion between Python data types and C/C++ data types and error checking, translation error return values into Python exception.

Using Python, better applications can be developed because different kinds of programmers can work together on a project. For example, when building a scientific application, C/C++ programmers can implement efficient numerical algorithms, while scientists on the same project can write Python programs that test and use those algorithms. The scientist doesn't have to learn a low-level programming language, and the C/C++ programmer doesn't need to understand the science involved.

Without Python, large amounts of C/C++ code often have to be written just to provide a flexible enough input mechanism so that scientists can feed the program its data, in all the variantions that are required for reasons of experimental setup (for instance). With Python, Python can be used to wrote a much more flexible input mechanism in a much shorter time, or Python itself can be the ultimate flexible input mechanism. As an extreme example, Lawrence Livermore National Laboratories is using Python to eventually replace a scripting language (BASIS) that was developed in house for the same purpose; BASIS started out as a simple input mechanism for Fortran programs, and gradually acquired many features of scripting languages (variables, conditionals, loops, procedures and so on) with increasing awkwardness.

Python Stability

Python is an advanced scripting language that is being used successfully to glue together large software components. It spans multiple platforms, middleware products, and application domains. Python is an object-oriented language with high-level data structures, dynamic typing, and dynamic binding. Python has been around since 1991, and has a very active user community

Like Tcl, Python is easily extensible with C/C++/Java code, and easily embeddable in applications. Python even uses Tk, the Tcl GUI toolkit, for a de-facto standard portable GUI toolkit. Unlike Tcl, however, Python supports object-oriented programming. Python programmers can create classes, use multiple inheritance, define methods, overload operators etc.

With the introduction of retrospective "bugfix" releases the stability of the language implementations can be, and is being, improved independently of the new features offered by more recent major or minor releases. Bugfix releases, indicated by a third component of the version number, only fix known problems and do not gratuitously introduce new and possibly incompatible features or modified library functionality.

JPython, Python and Java Integration

A new Python implementation written in 100% Pure Java, dubbed JPython, is currently under development; alpha releases are available for evaluation. JPython offers seamless scripting for Java. It is a full implementation of the Python language and standard library, adding direct access to the universe of Java classes. Java code can also use Python classes -- this is important for callbacks, for instance.

The main thrust for JPython is that it does for Java what Python already does for C and C++: to present programmers with more options in the trade-off between development time and execution time, by providing a more dynamic, more expressive alternative. JPython's integration with Java is superior to Python's integration with C/C++: due to Java's Reflection API, JPython can use arbitrary Java classes without the help of a wrapper generator such as SWIG. (C/C++ code must first be made available to Java through the Java native code interface; once it is callable from Java it is callable from JPython).

Python 3000?! Yes, that's the nickname for the next generation of the Python interpreter. The name may be considered a pun on Windows 2000, or a reference to Mystery Science Theatre 3000, a suitably Pythonesque tv show with a cult following. When will Python 3000 be released? Not for a loooooong time -- although you won't quite have to wait until the year 3000.

Originally, Python 3000 was intended to be a complete rewrite and redesign of the language. It would allow me to make incompatible changes in order to fix problems with the language design that weren't solvable in a backwards compatible way. The current plan, however, is that the necessary changes will be introduced gradually into the current Python 2.x line of development,with a clear transition path that includes a period of backwards compatibility support. .

So, don't expect the announcement of the release of Python 3000 any time soon. Instead, one day you may find that you are _already_ using Python 3000 -- only it won't be called that, but rather something like Python 2.8.7. And most of what you've learned in this book will still apply! Still, in the mean time references to Python 3000 will abound; just know that this is intentionally vaporware in the purest sense of the word. Rather than worry about Python 3000, continue to use and learn more about the Python version that you do have.

In the past year, Python has made great strides. We released Python 2.0, a big step forward, with new standard library features such as Unicode and XML support, and several new syntactic constructs, including augmented assignment: you can now write x += 1 instead of x = x+1. A few people wondered what the big deal was (answer: instead of x, imagine dict[key] or list[index]), but overall this was a big hit with those users who were already used to augmented assignment in other languages.

Using the Python Interpreter

Invoking the Interpreter

The Python interpreter is usually installed as /usr/local/bin/python on those machines where it is available; putting /usr/local/bin in your Unix shell's search path makes it possible to start it by typing the command python to the shell. Since the choice of the directory where the interpreter lives is an installation option, other places are possible; check with your local Python guru or system administrator. (E.g., /usr/local/python is a popular alternative location.)

Typing an end-of-file character (Control-D on Unix, Control-Z on DOS or Windows) at the primary prompt causes the interpreter to exit with a zero exit status. If that doesn't work, you can exit the interpreter by typing the following commands: "import sys; sys.exit()".

The interpreter's line-editing features usually aren't very sophisticated. On Unix, whoever installed the interpreter may have enabled support for the GNU readline library, which adds more elaborate interactive editing and history features. Perhaps the quickest check to see whether command line editing is supported is typing Control-P to the first Python prompt you get. If it beeps, you have command line editing; see Appendix A for an introduction to the keys. If nothing appears to happen, or if P is echoed, command line editing isn't available; you'll only be able to use backspace to remove characters from the current line.

The interpreter operates somewhat like the Unix shell: when called with standard input connected to a tty device, it reads and executes commands interactively; when called with a file name argument or with a file as standard input, it reads and executes a script from that file.

A third way of starting the interpreter is "python -c command [arg] ...", which executes the statement(s) in command, analogous to the shell's -c option. Since Python statements often contain spaces or other characters that are special to the shell, it is best to quote command in its entirety with double quotes.


.

Argument Passing

When known to the interpreter, the script name and additional arguments thereafter are passed to the script in the variable sys.argv, which is a list of strings. Its length is at least one; when no script and no arguments are given, sys.argv[0] is an empty string. When the script name is given as '-' (meaning standard input), sys.argv[0] is set to '-'. When -c command is used, sys.argv[0] is set to '-c'. Options found after -c command are not consumed by the Python interpreter's option processing but left in sys.argv for the command to handle.
.

Interactive Mode

When commands are read from a tty, the interpreter is said to be in interactive mode. In this mode it prompts for the next command with the primary prompt, usually three greater-than signs (">>> "); for continuation lines it prompts with the secondary prompt, by default three dots ("... "). The interpreter prints a welcome message stating its version number and a copyright notice before printing the first prompt:

python Python 1.5.2b2 (#1, Feb 28 1999, 00:02:06) [GCC 2.8.1] on sunos5 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam

Error Handling

When an error occurs, the interpreter prints an error message and a stack trace. In interactive mode, it then returns to the primary prompt; when input came from a file, it exits with a nonzero exit status after printing the stack trace. (Exceptions handled by an except clause in a try statement are not errors in this context.) Some errors are unconditionally fatal and cause an exit with a nonzero exit; this applies to internal inconsistencies and some cases of running out of memory. All error messages are written to the standard error stream; normal output from the executed commands is written to standard output. Typing the interrupt character (usually Control-C or DEL) to the primary or secondary prompt cancels the input and returns to the primary prompt.2.1Typing an interrupt while a command is executing raises the KeyboardInterrupt exception, which may be handled by a try statement

.

Python's Strengths

Syntactically, Python code looks like executable pseudo code. Program development using Python is 5-10 times faster than using C/C++, and 3-5 times faster than using Java. In many cases, a prototype of an application can be written in Python without writing any C/C++/Java code. Often, the prototype is sufficiently functional and performs well enough to be delivered as the final product, saving considerable development time. Other times, the prototype can be translated in part or in whole to C++ or Java -- Python's object-oriented nature makes the translation a straightforward process.

The best approach is often to write only the performance-critical parts of the application in C++ or Java, and use Python for all higher-level control and customization. There are several anecdotes about applications that started out as pure C++ cpde to which Python was added as an extension language, where in each new version the percentage of the application written in Python increased, while also increasing the overall performance, functionality and reliability of the application. (E.g. Case Study: Python in a Commercial Environment, by Greg Stein, Microsoft, in Proceedings of the 6th International Python Conference, and the Alice VR project at UvA and CMU.)

Python has a strong presence on the web. It is suitable for CGI programming (on all platforms: Unix, Windows and Mac); there are interfaces to all major commercial databases. Python has a library that interfaces to the main Internet and web protocols, and has HTML parsing and generation toolkits. Python was a major implementation language for Infoseek when they were smaller. At least one company (Digital Creations) is selling a suite of server side tools using Python. And finally, Python has been used to implement a web browser (Grail).

Python is also well represented in the distributed systems world. It is one of the main languages supported by Xerox PARC's ILU (Inter-Language Unification; a CORBA compatible distributed object system), and many distributed applications have been built in Python using ILU. Python is also used by the Hector project at the University of Queensland, Australia.

.