Scripting in Computational Science by H. P. Langtangen.


1	Introduction	1
1.1	Scripting: HigherLevel Programming for the 21st Century	1
1.1.1	A Productive Pair: C and the Unix Shell	1
1.1.2	System Programming Versus Scripting	2
1.1.3	GUI Programming	6
1.1.4	When to Choose a Scripting Language	7
1.1.5	Why Scripting is Useful in Computational Science	9
1.1.6	Script or Program?	9
1.1.7	Examples on Scripting Applications	10
1.2	Preparations for Studying the Book	11
1.2.1	Tools and Example Codes	11
1.2.2	Setting Environment Variables	12

2	Basic Scripting	14
2.1	The First Python Encounter	15
2.1.1	A Scientific Hello World Script	15
2.1.2	Reading and Writing Data Files	19
2.1.3	Automating Simulation and Visualization	26
2.1.4	Running Series of Simulations	36
2.1.5	File Format Conversion	43
2.2	A List of Common Tasks	46
2.2.1	File Reading and Writing	47
2.2.2	Output Formatting	48
2.2.3	Running an Application	50
2.2.4	List Operations	51
2.2.5	Tuples	54
2.2.6	Dictionary Operations	54
2.2.7	Writing and Reading Python Data Structures	56
2.2.8	Splitting and Joining Text	57
2.2.9	Text Processing	58
2.2.10	String Operations	60
2.2.11	Numerical Expressions	61
2.2.12	Environment Variables	62
2.2.13	Functions	63
2.2.14	File Globbing	67
2.2.15	Testing File Types	67
2.2.16	Copying and Renaming Files	68
2.2.17	Creating and Moving to Directories	69
2.2.18	Removing Files and Directories	70
2.2.19	Splitting a Pathname Into Directory, Filename, Etc	70
2.2.20	PlatformDependent Operations	71
2.2.21	Traversing Directory Trees	71
2.2.22	Downloading Internet Files	74
2.2.23	CPUTime Measurements	74
2.2.24	Binary Input/Output	76
2.2.25	Parsing CommandLine Arguments	78
2.2.26	The Python Profiler	81
2.2.27	Programming with Classes	82
2.2.28	Pickling Objects Into Files	84
2.2.29	Shelving Objects	85
2.2.30	Exception Handling	86
2.2.31	Determining a Variable's Type	87
2.2.32	Exercises	89
2.3	Some Useful Python Utilities	104
2.3.1	Building and Using Modules	104
2.3.2	IDLE: an Integrated Development Environment	107
2.3.3	Python Tools for Documenting Source Code	109
2.3.4	A List of Some Useful Python Modules	111
2.3.5	About Headers in Python Scripts	111
2.4	Numerical Computing in Python	113
2.4.1	A Quick NumPy Primer	114
2.4.2	File I/O with NumPy Arrays	117
2.4.3	Random Numbers	118
2.4.4	Linear Algebra	121
2.4.5	Reading and Writing Tables With NumPy Arrays	121
2.4.6	The ScientificPython Package	124
2.4.7	The Gnuplot Module	124
2.4.8	Example: Fitting a Straight Line to Data Points	126
2.4.9	The Python--Matlab Interface	127
2.5	Regular Expressions and Text Processing	129
2.5.1	Motivation	129
2.5.2	Special Characters	132
2.5.3	Combining Regular Expressions	134
2.5.4	Groups	137
2.5.5	Extracting Multiple Matches	139
2.5.6	Splitting Text	141
2.5.7	PatternMatching Modifiers	142
2.5.8	Substitution and Backreferences	144
2.5.9	A General Substitution Script	148
2.5.10	Debugging Regular Expressions	149
2.5.11	NonGreedy Versus Greedy Regular Expressions	150
2.5.12	The Nature of Regular Expressions	154
2.5.13	Exercises	154

3	Building Graphical User Interfaces	169
3.1	Introduction to GUI Programming in Python/Tkinter	169
3.1.1	Introductory Remarks	170
3.1.2	GUI Versions of the Scientific Hello World Script	172
3.1.3	The Hello World GUI Implemented as a Class	182
3.1.4	Learning About the Pack Command	185
3.1.5	An Introduction to the Grid Geometry Manager	187
3.1.6	The Simulation and Visualization Script with a GUI	189
3.1.7	Adapting Widgets to the User's Resize Actions	195
3.2	A List of Common Widget Operations	197
3.2.1	Frame	202
3.2.2	Label	202
3.2.3	Button	203
3.2.4	Entry	203
3.2.5	Slider	204
3.2.6	Checkbutton	204
3.2.7	Radiobutton	205
3.2.8	Menu and Menubutton	205
3.2.9	Balloon Help	206
3.2.10	Option Menu	207
3.2.11	Listbox	207
3.2.12	Message Box	209
3.2.13	UserDefined Dialogs	210
3.2.14	ColorPicker Dialogs	212
3.2.15	File Selection Dialogs	213
3.2.16	Toplevel	214
3.2.17	Combo Box	215
3.2.18	Other Types of Widgets	215
3.3	More GUI Programming	220
3.3.1	Customizing Fonts and Colors	220
3.3.2	A Simple Graphical Function Evaluator	222
3.3.3	A Text Widget with Tailored Keyboard Bindings	226
3.3.4	A Fancy List Widget	228
3.3.5	Making Reuseable PmwCompatible Widgets	232
3.4	Animated Graphics with Canvas Widgets	232
3.4.1	The First Canvas Encounter	232
3.4.2	Coordinate Systems	233
3.4.3	The Mathematical Model Class	237
3.4.4	The Planet Class	238
3.4.5	Drawing and Moving Planets	240
3.4.6	Dragging Planets to New Positions	241
3.4.7	Using Pmw's Scrolled Canvas Widget	245
3.4.8	Interactive Grid Generation with a Canvas Widget	248
3.5	Web Interfaces; CGI Programming	248
3.5.1	Web Versions of the Scientific Hello World Program	249
3.5.2	Debugging CGI Scripts	253
3.5.3	HTMLgen for Generating HTML Code	254
3.5.4	A General Shell Script Wrapper for CGI Scripts	256
3.5.5	Making a Web Interface to a Script	258
3.5.6	Getting a CGI Script to Work	262
3.5.7	Security Issues	265
3.5.8	Troubleshooting	266

4	A Collection of Examples	269
4.1	Experimenting With Compiler Flags	269
4.2	A Database for NumPy Arrays	280
4.2.1	The Structure of the Database	280
4.2.2	Implementation Based on Pickling	283
4.2.3	Implementation Based on Formatted ASCII Storage	284
4.2.4	Implementation Based on Shelving	284
4.2.5	Comparing the Various Techniques	286
4.3	Estimating Convergence Rates	286
4.4	Converting Data File Formats	287
4.4.1	Converting Simple ASCII Formats	287
4.4.2	Converting Binary Files	290
4.5	Automating Remote Login and File Transfer	291
4.5.1	Using Secure Shell Commands in Scripts	291
4.5.2	Copying Documents from Internet Locations	294
4.5.3	Transferring Directory Trees in Email	295
4.6	Running Sets of Computer Experiments	295
4.6.1	Handling Multiple Values of Input Parameters
4.7	Graphics and Image Manipulation	295
4.7.1	Making MPEG Movies From PostScript Files	295
4.8	Exchanging File Collections via Email	300
4.8.1	Specifying and Packing File Collections	302
4.9	Automating Web Questionnaires	306
4.9.1	Specification of Questions and Answers	307
4.9.2	Parsing the Input File and Generating a Web Form	309
4.9.3	Processing the Form in a CGI Script.	313
4.9.4	Statistics of the Questionnaire	315
4.9.5	Making the Webform Language Specific	316
4.9.6	Links to Individual Answers	317

5	Scripting and HighPerformance Computing	319
5.1	Timing Array Operations	320
5.1.1	Timing Results in Python	320
5.1.2	Timing Results in Other Scripting Languages	322
5.1.3	Timing Results in C++/C and Fortran 77	324
5.1.4	Using NumPy Arrays for Speeding up Python Code	324
5.2	Case Study: Stochastic Beam Simulation	325
5.2.1	The Computational Problem: Stochastic Simulation	325
5.2.2	A Pure Python Implementation	326
5.2.3	Adding Histogram and Graphics	329
5.2.4	Doing Graphics with Threads	331
5.2.5	A Pure C/C++ Implementation	332
5.2.6	A NumPy Implementation	334
5.2.7	Finding Bottlenecks in the Python Script; Profiling	337
5.3	About Wrapper Code	338
5.3.1	Example on Calling C From Python	338
5.3.2	Automatic Generation of Wrapper Code	339
5.3.3	Alternatives to using F2Py, Pyfort, and SWIG	340
5.4	Combining Python and Fortran 77	341
5.4.1	Wrapping Fortran 77 Code with F2py	342
5.4.2	Wrapping Fortran 77 Code with Pyfort	345
5.5	Computing with NumPy Arrays in Fortran	347
5.5.1	Wrapping the Functions with F2py	349
5.5.2	Wrapping the Functions with Pyfort	351
5.5.3	Handling of In/Out Arguments	352
5.5.4	Migrating the Stochastic Beam Example to F77	358
5.6	Combining Python and C Code Using SWIG	360
5.6.1	Defining the Interface and Generating Wrapper Code	360
5.6.2	Compiling and Linking	362
5.7	Combining Python and C++ Code Using SWIG	367
5.7.1	Defining the Interface and Generating Wrapper Code	367
5.7.2	Compiling and Linking	369
5.7.3	The Combined Python--C/C++ Application	369
5.8	Computing with NumPy Arrays in C and C++	371
5.8.1	Working with the C Part of a NumPy Array Directly	371
5.8.2	Using SWIG to Simplify NumPy Operations in C	374
5.9	A Simple Computational Steering Example	375
5.9.1	Writing a TimeLoop for Repeated Simulations	376
5.9.2	Creating a Python Interface	377
5.9.3	The Steering Python Script	379
5.9.4	Computational Steering via a GUI	382

6	Creating Efficient Working Environments	383
6.1	Methods and Software for Wave Simulation	383
6.1.1	Modeling of Wave Phenomena	384
6.1.2	The Computational Algorithm	388
6.1.3	The NeedtoKnow Information	390
6.1.4	Visualization	392
6.2	Building a Collection of Simple Scripts	396
6.2.1	Enhanced Browsing of Mapfiles	397
6.2.2	Searching for Curvenames in a Mapfile	398
6.2.3	Creating Gnuplot Animations	399
6.2.4	Automating Simulation	401
6.2.5	Running Series of Experiments	402
6.3	A PSE Based on a Flexible GUI	405
6.3.1	Using Notebooks for Specifying Functions	405
6.3.2	The BLT Graph Widget	409
6.3.3	Time and Grid Parameters	421
6.3.4	UserDrawn Bottom Functions	421
6.3.5	Collecting the Pieces in a Full GUI	425
6.4	Adding Computational Steering to the PSE	426
6.5	Scripting Interface to an Advanced Simulation Code	427

A	Setting up the Required Software Environment	431
A.1	Installing Software Packages; Unix Systems	431
A.1.1	A Suggested Directory Structure	431
A.1.2	Installing Tcl/Tk and Additional Modules	432
A.1.3	Installing Python and Additional Modules	433
A.1.4	Installing Perl and Additional Modules	436
A.1.5	Installing Gnuplot	439
A.1.6	Installing SWIG	439
A.1.7	Installing SDF and Related Tools	440
A.1.8	Testing the Installation of Scripting Utilities	440
A.2	Installing Software Packages; Windows Systems	441

B	Elements of Software Engineering	442
B.1	Verification of Scripts (Regression Testing)	442
B.1.1	Automating Regression Tests	442
B.1.2	Implementing a Tool for Regression Tests	446
B.1.3	Writing a Test Script	450
B.1.4	Verifying Output from Numerical Computations	451
B.1.5	Making a Regression Test for the Wave Simulator	455
B.2	Version Control Management	457
B.2.1	Getting Started with CVS	457
B.2.2	Building Scripts to Simplify the Use of CVS	461
B.3	Documenting Scripts	463
B.3.1	Writing Clean and Readable Code	464
B.3.2	Using SDF for Embedded Documentation	464
B.3.3	The Basic Structure of SDFDocumented Codes	465
B.3.4	Examples on Some Basic SDF Commands	466
B.3.5	Translating SDF to Common Formats	468
B.4	Exercises	470

C	Introduction to Perl	474
C.1	The First Perl Encounter	474
C.1.1	A Scientific Hello World Script	475
C.1.2	Reading and Writing Data Files	476
C.1.3	Automating Simulation and Visualization	481
C.1.4	There Is More Than One Way to Do It	485
C.2	A List of Common Tasks	489
C.2.1	File Reading and Writing	489
C.2.2	Running an Application	490
C.2.3	OneLine Perl Scripts	491
C.2.4	Array and List Operations	492
C.2.5	Hash Operations	494
C.2.6	Splitting and Joining Text	495
C.2.7	Text Processing	496
C.2.8	String Operations	496
C.2.9	Numerical Expressions	497
C.2.10	Environment Variables	497
C.2.11	Subroutines	498
C.2.12	File Globbing	501
C.2.13	Testing File Types	502
C.2.14	Copying and Renaming Files	503
C.2.15	Creating and Moving to Directories	503
C.2.16	Removing Files and Directories	503
C.2.17	Splitting a Pathname Into Directory, Filename, Etc	504
C.2.18	Traversing Directory Trees	504
C.2.19	Downloading a URL	505
C.2.20	CPUTime Measurements	506
C.2.21	Programming with Classes in Perl	507
C.2.22	Debugging Perl Scripts	507
C.2.23	Regular Expressions	509
C.2.24	Exercises	513
C.2.25	Building and Using Modules	516
C.2.26	A Quick Intro to POD	517
C.2.27	A Listing of Some Useful Perl Modules	519
C.2.28	Binary Input/Output	519
C.3	Perl Versus Python	520
C.3.1	Python's Advantages	520
C.3.2	Perl's Advantages	522
C.4	GUI Programming with Perl/Tk	523
C.4.1	GUI Versions of the Scientific Hello World Script	523
C.4.2	Web Interfaces; CGI Programming	527

Table of Contents