Includes appendix & index
Table of Contents (Summary)
1 The Basics: Getting Started Quickly 1
2 List Data: Working with Ordered Data 47
3 Structured Data: Working with Structured Data 95
4 Code Reuse: Functions and Modules 145
5 Building a Webapp: Getting Real 195
6 Storing and Manipulating Data: Where to Put Your Data 243
7 Using a Database: Putting Python’s DB-API to Use 281
8 A Little Bit of Class: Abstracting Behavior and State 309
9 The Context Management Protocol: Hooking into Python’s with Statement 335
10 Function Decorators: Wrapping Functions 363
11 Exception Handling: What to Do When Things Go Wrong 413
11¾ A Little Bit of Threading: Dealing with Waiting 461
12 Advanced Iteration: Looping like Crazy 477
A Installing: Installing Python 521
B Pythonanywhere: Deploying Your Webapp 529
C Top Ten Things We Didn’t Cover: There’s Always More to Learn 539
D Top Ten Projects Not Covered: Even More Tools, Libraries, and Modules 551
E Getting Involved: The Python Community 563
Table of Contents (the real thing)
Your brain on Python. Here you are trying to learn something, while here
your brain is, doing you a favor by making sure the learning doesn’t stick. Your brain’s
thinking, “Better leave room for more important things, like which wild animals to
avoid and whether naked snowboarding is a bad idea.” So how do you trick your brain
into thinking that your life depends on knowing how to program in Python?
Intro
Who is this book for? xxviii
We know what you’re thinking xxix
We know what your brain is thinking xxix
Metacognition: thinking about thinking xxxi
Here’s what WE did xxxii
Read me xxxiv
Acknowledgments xxxvii
table of contents
x
the basics
Getting Started Quickly
Get going with Python programming as quickly as possible.
In this chapter, we introduce the basics of programming in Python, and we do this in
typical Head First style: by jumping right in. After just a few pages, you’ll have run
your first sample program. By the end of the chapter, you’ll not only be able to run the
sample program, but you’ll understand its code too (and more besides). Along the way,
you’ll learn about a few of the things that make Python the programming language it is.
Understanding IDLE’s Windows 4
Executing Code, One Statement at a Time 8
Functions + Modules = The Standard Library 9
Data Structures Come Built-in 13
Invoking Methods Obtains Results 14
Deciding When to Run Blocks of Code 15
What “else” Can You Have with “if ”? 17
Suites Can Contain Embedded Suites 18
Returning to the Python Shell 22
Experimenting at the Shell 23
Iterating Over a Sequence of Objects 24
Iterating a Specific Number of Times 25
Applying the Outcome of Task #1 to Our Code 26
Arranging to Pause Execution 28
Generating Random Integers with Python 30
Coding a Serious Business Application 38
Is Indentation Driving You Crazy? 40
Asking the Interpreter for Help on a Function 41
Experimenting with Ranges 42
Chapter 1’s Code 46
1
table of contents
xi
list data
Working with Data
All programs process data, and Python programs are no exception.
In fact, take a look around: data is everywhere. A lot of, if not most, programming is all about
data: acquiring data, processing data, understanding data. To work with data effectively, you need
somewhere to put your data when processing it. Python shines in this regard, thanks (in no small
part) to its inclusion of a handful of widely applicable data structures: lists, dictionaries, tuples, and
sets. In this chapter, we’ll preview all four, before spending the majority of this chapter digging deeper
into lists (and we’ll deep-dive into the other three in the next chapter). We’re covering these data
structures early, as most of what you’ll likely do with Python will revolve around working with data.
0
D
-12
1
o
-11
2
n
-10
3
'
-9
4
t
-8
5
-7
6
p
-6
7
a
-5
8
n
-4
9
i
-3
10
c
-2
11
!
-1
Numbers, Strings...and Objects 48
Meet the Four Built-in Data Structures 50
An Unordered Data Structure: Dictionary 52
A Data Structure That Avoids Duplicates: Set 53
Creating Lists Literally 55
Use Your Editor When Working on More Than a Few Lines of Code 57
“Growing” a List at Runtime 58
Checking for Membership with “in” 59
Removing Objects from a List 62
Extending a List with Objects 64
Inserting an Object into a List 65
How to Copy a Data Structure 73
Lists Extend the Square Bracket Notation 75
Lists Understand Start, Stop, and Step 76
Starting and Stopping with Lists 78
Putting Slices to Work on Lists 80
Python’s “for” Loop Understands Lists 86
Marvin’s Slices in Detail 88
When Not to Use Lists 91
Chapter 2’s Code, 1 of 2 92
2
table of contents
xii
Name: Ford Prefect
Gender: Male
Occupation: Researcher
Home Planet: Betelgeuse Seven
structured data
Working with Structured Data
Python’s list data structure is great, but it isn’t a data
panacea. When you have truly structured data (and using a list to store it may not be
the best choice), Python comes to your rescue with its built-in dictionary. Out of the box,
the dictionary lets you store and manipulate any collection of key/value pairs. We look
long and hard at Python’s dictionary in this chapter, and—along the way—meet set and
tuple, too. Together with the list (which we met in the previous chapter), the dictionary,
set, and tuple data structures provide a set of built-in data tools that help to make Python
and data a powerful combination.
A Dictionary Stores Key/Value Pairs 96
How to Spot a Dictionary in Code 98
Insertion Order Is NOT Maintained 99
Value Lookup with Square Brackets 100
Working with Dictionaries at Runtime 101
Updating a Frequency Counter 105
Iterating Over a Dictionary 107
Iterating Over Keys and Values 108
Iterating Over a Dictionary with “items” 110
Just How Dynamic Are Dictionaries? 114
Avoiding KeyErrors at Runtime 116
Checking for Membership with “in” 117
Ensuring Initialization Before Use 118
Substituting “not in” for “in” 119
Putting the “setdefault” Method to Work 120
Creating Sets Efficiently 124
Taking Advantage of Set Methods 125
Making the Case for Tuples 132
Combining the Built-in Data Structures 135
Accessing a Complex Data Structure’s Data 141
Chapter 3’s Code, 1 of 2 143
3
table of contents
xiii
module
code reuse
Functions and Modules
Reusing code is key to building a maintainable system.
And when it comes to reusing code in Python, it all starts and ends with the humble
function. Take some lines of code, give them a name, and you’ve got a function (which
can be reused). Take a collection of functions and package them as a file, and you’ve
got a module (which can also be reused). It’s true what they say: it’s good to share, and
by the end of this chapter, you’ll be well on your way to sharing and reusing your code,
thanks to an understanding of how Python’s functions and modules work.
Reusing Code with Functions 146
Introducing Functions 147
Invoking Your Function 150
Functions Can Accept Arguments 154
Returning One Value 158
Returning More Than One Value 159
Recalling the Built-in Data Structures 161
Making a Generically Useful Function 165
Creating Another Function, 1 of 3 166
Specifying Default Values for Arguments 170
Positional Versus Keyword Assignment 171
Updating What We Know About Functions 172
Running Python from the Command Line 175
Creating the Required Setup Files 179
Creating the Distribution File 180
Installing Packages with “pip” 182
Demonstrating Call-by-Value Semantics 185
Demonstrating Call-by-Reference Semantics 186
Install the Testing Developer Tools 190
How PEP 8–Compliant Is Our Code? 191
Understanding the Failure Messages 192
Chapter 4’s Programs 194
4
table of contents
xiv
building a webapp
Getting Real
At this stage, you know enough Python to be dangerous.
With this book’s first four chapters behind you, you’re now in a position to productively
use Python within any number of application areas (even though there’s still lots of
Python to learn). Rather than explore the long list of what these application areas
are, in this and subsequent chapters, we’re going to structure our learning around the
development of a web-hosted application, which is an area where Python is especially
strong. Along the way, you’ll learn a bit more about Python.
Python: What You Already Know 196
What Do We Want Our Webapp to Do? 200
Let’s Install Flask 202
How Does Flask Work? 203
Running Your Flask Webapp for the First Time 204
Creating a Flask Webapp Object 206
Decorating a Function with a URL 207
Running Your Webapp’s Behavior(s) 208
Exposing Functionality to the Web 209
Building the HTML Form 213
Templates Relate to Web Pages 216
Rendering Templates from Flask 217
Displaying the Webapp’s HTML Form 218
Preparing to Run the Template Code 219
Understanding HTTP Status Codes 222
Handling Posted Data 223
Refining the Edit/Stop/Start/Test Cycle 224
Accessing HTML Form Data with Flask 226
Using Request Data in Your Webapp 227
Producing the Results As HTML 229
Preparing Your Webapp for the Cloud 238
Chapter 5’s Code 241
5
table of contents
xv
Form Data Remote_addr User_agent Results
ImmutableMultiDict([(‘phrase’, 127.0.0.1 Mozilla/5.0 (Macintosh; {‘e’, ‘i’}
‘hitch-hiker’), (‘letters’, ‘aeiou’)]) Intel Mac OS X 10_11_2) AppleWebKit/537.36 (KHTML,
like Gecko) Chrome/47.0.2526
.106 Safari/537.36
storing and manipulating data
Where to Put Your Data
Sooner or later, you’ll need to safely store your data somewhere.
And when it comes to storing data, Python has you covered. In this chapter, you’ll learn
about storing and retrieving data from text files, which—as storage mechanisms go—may
feel a bit simplistic, but is nevertheless used in many problem areas. As well as storing and
retrieving your data from files, you’ll also learn some tricks of the trade when it comes to
manipulating data. We’re saving the “serious stuff” (storing data in a database) until the next
chapter, but there’s plenty to keep us busy for now when working with files.
Doing Something with Your Webapp’s Data 244
Python Supports Open, Process, Close 245
Reading Data from an Existing File 246
A Better Open, Process, Close: “with” 248
View the Log Through Your Webapp 254
Examine the Raw Data with View Source 256
It’s Time to Escape (Your Data) 257
Viewing the Entire Log in Your Webapp 258
Logging Specific Web Request Attributes 261
Log a Single Line of Delimited Data 262
From Raw Data to Readable Output 265
Generate Readable Output With HTML 274
Embed Display Logic in Your Template 275
Producing Readable Output with Jinja2 276
The Current State of Our Webapp Code 278
Asking Questions of Your Data 279
Chapter 6’s Code 280
6
table of contents
xvi
Python’s
DB-API
The MySQLConnector/Python
Driver
MySQL Your code
using a database
Putting Python’s DB-API to Use
Storing data in a relational database system is handy. In this chapter,
you’ll learn how to write code that interacts with the popular MySQL database technology, using
a generic database API called DB-API. The DB-API (which comes standard with every Python
install) allows you to write code that is easily transferred from one database product to the next...
assuming your database talks SQL. Although we’ll be using MySQL, there’s nothing stopping you
from using your DB-API code with your favorite relational database, whatever it may be. Let’s
see what’s involved in using a relational database with Python. There’s not a lot of new Python in
this chapter, but using Python to talk to databases is a big deal, so it’s well worth learning.
Database-Enabling Your Webapp 282
Task 1: Install the MySQL Server 283
Introducing Python’s DB-API 284
Task 2: Install a MySQL Database Driver for Python 285
Install MySQL-Connector/Python 286
Task 3: Create Our Webapp’s Database and Tables 287
Decide on a Structure for Your Log Data 288
Confirm Your Table Is Ready for Data 289
Task 4: Create Code to Work with Our Webapp’s Database and Tables 296
Storing Data Is Only Half the Battle 300
How Best to Reuse Your Database Code? 301
Consider What You’re Trying to Reuse 302
What About That Import? 303
You’ve Seen This Pattern Before 305
The Bad News Isn’t Really All That Bad 306
Chapter 7’s Code 307
7
table of contents
xvii
a little bit of class
Abstracting Behavior and State
Classes let you bundle code behavior and state together.
In this chapter, you’re setting your webapp aside while you learn about creating Python classes.
You’re doing this in order to get to the point where you can create a context manager with the
help of a Python class. As creating and using classes is such a useful thing to know about
anyway, we’re dedicating this chapter to them. We won’t cover everything about classes, but we’ll
touch on all the bits you’ll need to understand in order to confidently create the context manager
your webapp is waiting for.
Hooking into the “with” Statement 310
An Object-Oriented Primer 311
Creating Objects from Classes 312
Objects Share Behavior but Not State 313
Doing More with CountFromBy 314
Invoking a Method: Understand the Details 316
Adding a Method to a Class 318
The Importance of “self ” 320
Coping with Scoping 321
Prefix Your Attribute Names with “self ” 322
Initialize (Attribute) Values Before Use 323
Dunder “init” Initializes Attributes 324
Initializing Attributes with Dunder “init” 325
Understanding CountFromBy’s Representation 328
Defining CountFromBy’s Representation 329
Providing Sensible Defaults for CountFromBy 330
Classes: What We Know 332
Chapter 8’s Code 333
8
table of contents
xviii
§
$ mysql -u vsearch -p vsearchlogDB
Enter password:
Welcome to MySQL monitor...
mysql> select * from log;
+----+---------------------+--------------------------+---------+-----------+----------------+----------------------+
| id | ts | phrase | letters | ip | browser_string | results |
+----+---------------------+--------------------------+---------+-----------+----------------+----------------------+
| 1 | 2016-03-09 13:40:46 | life, the uni ... ything | aeiou | 127.0.0.1 | firefox | {'u', 'e', 'i', 'a'} |
| 2 | 2016-03-09 13:42:07 | hitch-hiker | aeiou | 127.0.0.1 | safari | {'i', 'e'} |
| 3 | 2016-03-09 13:42:15 | galaxy | xyz | 127.0.0.1 | chrome | {'y', 'x'} |
| 4 | 2016-03-09 13:43:07 | hitch-hiker | xyz | 127.0.0.1 | firefox | set() |
+----+---------------------+--------------------------+---------+-----------+----------------+----------------------+
4 rows in set (0.0 sec)
mysql> quit
Bye
File Edit Window Help Checking our log DB
the context management protocol
Hooking into Python’s with Statements
It’s time to take what you’ve just learned and put it to work.
Chapter 7 discussed using a relational database with Python, while Chapter 8 provided an
introduction to using classes in your Python code. In this chapter, both of these techniques are
combined to produce a context manager that lets us extend the with statement to work with
relational database systems. In this chapter, you’ll hook into the with statement by creating a
new class, which conforms to Python’s context management protocol.
What’s the Best Way to Share Our Webapp’s Database Code? 336
Managing Context with Methods 338
You’ve Already Seen a Context Manager in Action 339
Create a New Context Manager Class 340
Initialize the Class with the Database Config 341
Perform Setup with Dunder “enter” 343
Perform Teardown with Dunder “exit” 345
Reconsidering Your Webapp Code, 1 of 2 348
Recalling the “log_request” Function 350
Amending the “log_request” Function 351
Recalling the “view_the_log” Function 352
It’s Not Just the Code That Changes 353
Amending the “view_the_log” Function 354
Answering the Data Questions 359
Chapter 9’s Code, 1 of 2 360
9
table of contents
xix
function decorators
Wrapping Functions
When it comes to augmenting your code, Chapter 9’s context
management protocol is not the only game in town. Python also lets you
use function decorators, a technique whereby you can add code to an existing function without
having to change any of the existing function’s code. If you think this sounds like some sort of
black art, don’t despair: it’s nothing of the sort. However, as coding techniques go, creating a
function decorator is often considered to be on the harder side by many Python programmers,
and thus is not used as often as it should be. In this chapter, our plan is to show you that, despite
being an advanced technique, creating and using your own decorators is not that hard.
Your Web Server (Not Your Computer) Runs Your Code 366
Flask’s Session Technology Adds State 368
Dictionary Lookup Retrieves State 369
Managing Logins with Sessions 374
Let’s Do Logout and Status Checking 377
Pass a Function to a Function 386
Invoking a Passed Function 387
Accepting a List of Arguments 390
Processing a List of Arguments 391
Accepting a Dictionary of Arguments 392
Processing a Dictionary of Arguments 393
Accepting Any Number and Type of Function Arguments 394
Creating a Function Decorator 397
The Final Step: Handling Arguments 401
Putting Your Decorator to Work 404
Back to Restricting Access to /viewlog 408
Chapter 10’s Code, 1 of 2 410
10
table of contents
xx
...
Exception
+-- StopIteration
+-- StopAsyncIteration
+-- ArithmeticError
| +-- FloatingPointError
| +-- OverflowError
| +-- ZeroDivisionError
+-- AssertionError
+-- AttributeError
+-- BufferError
+-- EOFError
...
exception handling
What to Do When Things Go Wrong
Things go wrong, all the time—no matter how good your code is.
You’ve successfully executed all of the examples in this book, and you’re likely confident all of
the code presented thus far works. But does this mean the code is robust? Probably not. Writing
code based on the assumption that nothing bad ever happens is (at best) naive. At worst, it’s
dangerous, as unforeseen things do (and will) happen. It’s much better if you’re wary while
coding, as opposed to trusting. Care is needed to ensure your code does what you want it to, as
well as reacts properly when things go south.
Databases Aren’t Always Available 418
Web Attacks Are a Real Pain 419
Input-Output Is (Sometimes) Slow 420
Your Function Calls Can Fail 421
Always Try to Execute Error-Prone Code 423
try Once, but except Many Times 426
The Catch-All Exception Handler 428
Learning About Exceptions from “sys” 430
The Catch-All Exception Handler, Revisited 431
Getting Back to Our Webapp Code 433
Silently Handling Exceptions 434
Handling Other Database Errors 440
Avoid Tightly Coupled Code 442
The DBcm Module, Revisited 443
Creating Custom Exceptions 444
What Else Can Go Wrong with “DBcm”? 448
Handling SQLError Is Different 451
Raising an SQLError 453
A Quick Recap: Adding Robustness 455
How to Deal with Wait? It Depends... 456
Chapter 11’s Code, 1 of 3 457
11
table of contents
xxi
Wait!
a little bit of threading
Dealing with Waiting
Your code can sometimes take a long time to execute.
Depending on who notices, this may or may not be an issue. If some code takes 30
seconds to do its thing “behind the scenes,” the wait may not be an issue. However,
if your user is waiting for your application to respond, and it takes 30 seconds,
everyone notices. What you should do to fix this problem depends on what you’re
trying to do (and who’s doing the waiting). In this short chapter, we’ll briefly discuss
some options, then look at one solution to the issue at hand: what happens if
something takes too long?
Waiting: What to Do? 462
How Are You Querying Your Database? 463
Database INSERTs and SELECTs Are Different 464
Doing More Than One Thing at Once 465
Don’t Get Bummed Out: Use Threads 466
First Things First: Don’t Panic 470
Don’t Get Bummed Out: Flask Can Help 471
Is Your Webapp Robust Now? 474
Chapter 11¾’s Code, 1 of 2 475
113
/4
table of contents
xxii
advanced iteration
Looping Like Crazy
It’s often amazing how much time our programs spend in loops.
This isn’t a surprise, as most programs exist to perform something quickly a whole heap of times.
When it comes to optimizing loops, there are two approaches: (1) improve the loop syntax (to
make it easier to specify a loop), and (2) improve how loops execute (to make them go faster).
Early in the lifetime of Python 2 (that is, a long, long time ago), the language designers added a
single language feature that implements both approaches, and it goes by a rather strange name:
comprehension.
Reading CSV Data As Lists 479
Reading CSV Data As Dictionaries 480
Stripping, Then Splitting, Your Raw Data 482
Be Careful When Chaining Method Calls 483
Transforming Data into the Format You Need 484
Transforming into a Dictionary Of Lists 485
Spotting the Pattern with Lists 490
Converting Patterns into Comprehensions 491
Take a Closer Look at the Comprehension 492
Specifying a Dictionary Comprehension 494
Extend Comprehensions with Filters 495
Deal with Complexity the Python Way 499
The Set Comprehension in Action 505
What About “Tuple Comprehensions”? 507
Parentheses Around Code == Generator 508
Using a Listcomp to Process URLs 509
Using a Generator to Process URLs 510
Define What Your Function Needs to Do 512
Yield to the Power of Generator Functions 513
Tracing Your Generator Function, 1 of 2 514
One Final Question 518
Chapter 12’s Code 519
It’s Time to Go… 520
12
table of contents
xxiii
installation
Installing Python
pythonanywhere
Deploying Your Webapp
First things first: let’s get Python installed on your computer.
Whether you’re running on Windows, Mac OS X, or Linux, Python’s got you covered. How you
install it on each of these platforms is specific to how things work on each of these operating
systems (we know...a shocker, eh?), and the Python community works hard to provide installers
that target all the popular systems. In this short appendix, you’ll be guided through installing
Python on your computer.
At the end of Chapter 5, we claimed that deploying your webapp to
the cloud was only 10 minutes away. It’s now time to make good on that promise.
In this appendix, we are going to take you through the process of deploying your webapp on
PythonAnywhere, going from zero to deployed in about 10 minutes. PythonAnywhere is a
favorite among the Python programming community, and it’s not hard to see why: it works exactly
as you’d expect it to, has great support for Python (and Flask), and—best of all—you can get
started hosting your webapp at no cost.
Install Python 3 on Windows 522
Check Python 3 on Windows 523
Add to Python 3 on Windows 524
Install Python 3 on Mac OS X (macOS) 525
Check and Configure Python 3 on Mac OS X 526
Install Python 3 on Linux 527
Step 0: A Little Prep 530
Step 1: Sign Up for PythonAnywhere 531
Step 2: Upload Your Files to the Cloud 532
Step 3: Extract and Install Your Code 533
Step 4: Create a Starter Webapp, 1 of 2 534
Step 5: Configure Your Webapp 536
Step 6: Take Your Cloud-Based Webapp for a Spin! 537
a
b
table of contents
xxiv
top ten things we didn’t cover
There’s Always More to Learn
It was never our intention to try to cover everything. This book’s goal was
always to show you enough Python to get you up to speed as quickly as possible. There’s a lot
more we could’ve covered, but didn’t. In this appendix, we discuss the top 10 things that—given
another 600 pages or so—we would’ve eventually gotten around to. Not all of the 10 things will
interest you, but quickly flip through them just in case we’ve hit on your sweet spot, or provided
an answer to that nagging question. All the programming technologies in this appendix come
baked in to Python and its interpreter.
1. What About Python 2? 540
2. Virtual Programming Environments 541
3. More on Object Orientation 542
4. Formats for Strings and the Like 543
5. Getting Things Sorted 544
6. More from the Standard Library 545
7. Running Your Code Concurrently 546
8. GUIs with Tkinter (and Fun with Turtles) 547
9. It’s Not Over ’Til It’s Tested 548
10. Debug, Debug, Debug 549
c
table of contents
xxv
top ten projects not covered
Even More Tools, Libraries, and Modules
We know what you’re thinking as you read this appendix’s title.
Why on Earth didn’t they make the title of the last appendix: The Top Twenty Things We Didn’t
Cover? Why another 10? In the last appendix, we limited our discussion to stuff that comes
baked in to Python (part of the language’s “batteries included”). In this appendix, we cast the net
much further afield, discussing a whole host of technologies that are available to you because
Python exists. There’s lots of good stuff here and—just like with the last appendix—a quick
perusal won’t hurt you one single bit.
1. Alternatives to >>> 552
2. Alternatives to IDLE 553
3. Jupyter Notebook: The Web-Based IDE 554
4. Doing Data Science 555
5. Web Development Technologies 556
6. Working with Web Data 557
7. More Data Sources 558
8. Programming Tools 559
9. Kivy: Our Pick for “Coolest Project Ever” 560
10. Alternative Implementations 561
d
table of contents
xxvi
getting involved
The Python Community
Python is much more than a great programming language.
It’s a great community, too. The Python Community is welcoming, diverse, open, friendly,
sharing, and giving. We’re just amazed that no one, to date, has thought to put that on a
greeting card! Seriously, though, there’s more to programming in Python than the language. An
entire ecosystem has grown up around Python, in the form of excellent books, blogs, websites,
conferences, meetups, user groups, and personalities. In this appendix, we take a survey of the
Python community and see what it has to offer. Don’t just sit around programming on your own:
get involved!
BDFL: Benevolent Dictator for Life 564
A Tolerant Community: Respect for Diversity 565
Python Podcasts 566
The Zen of Python 567
Which Book Should I Read Next? 568
Our Favorite Python Books 569