The Wayback Machine - http://web.archive.org/web/20160322023123/http://planetpython.org/

skip to navigation
skip to content

Planet Python

Last update: March 22, 2016 01:50 AM

March 21, 2016


Bhishan Bhandari

Cryptography with an interpreter

Hey Guys, it’s been a long time since I published my last article. Apologies for the delay. Anyway, straight into the topic, Cryptography. Well, cryptography with python. This semester(V of Bsc CS) I choose Cryptography as an elective over Neural Network and I am enjoying it. So far I have learned Ceasar Cipher, Playfair Cipher, […]

The post Cryptography with an interpreter appeared first on The Tara Nights.

March 21, 2016 08:11 PM


Mike Driscoll

Python 201 Book Outline

Over the weekend, I spent some time rearranging ideas for my latest book such that I have have four specific sections of the book. Here they are:

Part I – Intermediate Modules

  • Chapter 1 – The argparse module
  • Chapter 2 – The collections module
  • Chapter 3 – The contextlib module (Context Managers)
  • Chapter 4 – The functools module (Function overloading, caching, etc)
  • Chapter 5 – All about imports
  • Chapter 6 – The importlib module
  • Chapter 7 – The itertools module
  • Chapter 8 – The re module (An Intro to Regex in Python)
  • Chapter 9 – The typing module (Type Hinting)

Part II – Odds and Ends

  • Chapter 10 – generators / iterators
  • Chapter 11 – map, filter, reduce
  • Chapter 12 – unicode
  • Chapter 13 – benchmarking
  • Chapter 14 – encryption
  • Chapter 15 – Connecting to databases

Part III – Web

  • Chapter 16 – Web scraping
  • Chapter 17 – Working with web APIs
  • Chapter 18 – ftplib
  • Chapter 19 – urllib / httplib (client / server)

Part IV – Testing

  • Chapter 20 – Doctest
  • Chapter 21 – unittest
  • Chapter 22 – mock
  • Chapter 23 – coverage.py

I want to note that these are just the topics that will absolutely be included. I may add others. I will also be adding others should I reach my stretch goal. If you’re interested in getting early access to the book or just want to support the blog, you can do so at my Kickstarter for Python 201!

March 21, 2016 05:15 PM


Python Engineering at Microsoft

What do your users really think? Using Text Analytics to understand GitHub Issue Sentiment

Launch Notebook Now!

Ever get the feeling your users aren’t that happy with your project? We all get those issues that are real downers on our repository. So I thought, let’s take these issues and make something fun. Using the Text Analytics Service and the WordCloud Python package, we can make some pretty pictures out of otherwise negative comments. I also found it fun to make clouds of the more positive issues.

PTVS-Word-Cloud

Below you will find a few snippets on how to do this yourself. If you want to just run this against your favorite GitHub project you can open a Jupyter notebook using the notebook link above. The below code is shown just to give you an idea of what the notebook does. In order to make it function you would need to complete a few additional bits of code.

 

Step 1: Install some libraries

We use a few libraries and should start by installing them. CortanaAnalytics is a small library to wrap requests to Azure Data Market Services. PyGitHub serves a similar purpose for GitHub. WordCloud helps to make pretty pictures.

pip install CortanaAnalytics
pip install PyGitHub
pip install wordcloud

 

Step 2: Get API keys to access Azure Text Analytics and GitHub

We need two API keys so we can access some services.

You can get an account key for the Text Analytics Service by signing up at http://azure.microsoft.com/en-us/marketplace/partners/amla/text-analytics
You can get a GitHub API Key by creating a token at https://github.com/settings/tokens.

 

Step 3: Get some issues from GitHub

Once you have API Keys, you just need to get GitHub issues.

import github

g = github.Github(GITHUB_ACCESS_TOKEN)
r = g.get_repo(GITHUB_REPOSITORY)
issues = r.get_issues(state='open')

Step 4: Analyse an issue using the Text Analytics Service

Once you have the GitHub issues we can iterate on them arranging them into text bits that can be analysed by Text Analytics. We will batch Sentiment requests together for the issue to cut down on the overall number of requests.

from cortanaanalytics.textanalytics import TextAnalytics
text_bits_to_analyse = [
    { 'Id':0, 'Text':issue.title },
    { 'Id':1, 'Text':issue.body }
]

ta = TextAnalytics(AZURE_PRIMARY_ACCOUNT_KEY)
sentiments = ta.get_sentiment_batch(text_bits_to_analyse)

title_sentiment = sentiments[0]['Score']
body_sentiment = sentiments[1]['Score']

 

Step 5: Get Key Phrases

We can also get key phrases using the same issues we used for sentiment.

key_phrases = ta.get_key_phrases_batch([{ 'Id':i.number, 'Text':i.body }])[0]['KeyPhrases']

Step 6: Generate some pretty pictures of our data using WordCloud
And once we have all of that, we can go ahead and make word clouds.

from wordcloud import WordCloud
import matplotlib.pyplot as plt

def show_wordcloud(frequencies):
    frequencies_cleaned = [x for x in frequencies if x[0].lower() not in words_to_remove]
    
    wordcloud = WordCloud(width=1920, height=1080).generate_from_frequencies(frequencies_cleaned)
    plt.axis("off")
    plt.imshow(wordcloud)

show_wordcloud(key_phrases)

And that’s it. You can try to reproduce this on your own locally or run the notebook and experiment from the Azure Notebooks environment.

 

Launch Notebook Now!

March 21, 2016 05:00 PM


PyCon

Startup Row: UtilityAPI won the SF Python pitch event

A post by Don Sheu, one of our Startup Row Coordinators

PyCon 2016’s Startup Row got our campaign on the road on March 9th in San Francisco, meeting with the local SF Python user group at Yelp headquarters. Six early-stage companies that use Python gave their pitches, competing for an opportunity to exhibit in the PyCon Expo Hall on Startup Row. The roster of candidate startups included Alpaca, Bauxy, Beansprock, Opsulutely, Watt Time, and UtilityAPI.

UtilityAPI won! They convinced the judges that its services for the new energy economy held the most promise, edging out their high quality competitors. Founded by Daniel Roesler and Elena Lucas, UtilityAPI provides easy access to usage data for customers like PG&E, ConEdison, and the Los Angeles Department of Water & Power.

An outstanding panel of judges selected our winner. On the panel were several entrepreneurs with successful exits like Bethanye McKinney Blount, Bebe Chueh, and Leah Culver. Kat Manalac, a partner with Y Combinator, joined the panel, as did currently active founders: Startup Row alumna Christine Spang founder of Nylas, and Jessica Scorpio founder of Getaround.

Our judging panel! From left to right::
@leahculver @spang @bebechueh @jessicascorpio @KatManalac @bethanye


Tonight, March 21st, Startup Row continues its road trip with a visit to Seattle. The local Puget Sound Programming Python meetup and Techstars will meet at Startup Hall on the UW campus, Seattle, to select Seattle’s representative to PyCon in Portland. If you’re going to be in Seattle this evening, you can join and be part of the audience!

Photo credit: Jeremy Smith, Startup Row's California Director

March 21, 2016 04:10 PM


Doug Hellmann

datetime — Date and Time Value Manipulation — PyMOTW 3

datetime contains functions and classes for working with dates and times, separately and together. Read more… This post is part of the Python Module of the Week series for Python 3. See PyMOTW.com for more articles from the series.

March 21, 2016 01:00 PM


Mike Driscoll

PyDev of the Week: Massimo DiPierro

This week we welcome Massimo DiPierro (@mdipierro) as our PyDev of the Week! Massimo is the inventor and lead developer of web2py, but he’s also contributed to lots of other projects which you can see on his github profile. He is also the author of the following books: Annotated Algorithms in Python: with Applications in Physics, Biology, and Finance and web2py Complete Reference Manual. Let’s take some time to get to know him better!

Can you tell us a little about yourself (hobbies, education, etc):

I am 44, was born in Italy, and since when I was kid I loved math and science. I have a BS/MS in Physics from the University of Pisa and a Ph.D. in Theoretical Physics from the University of Southampton in UK. This means I can read most of the formulas that Sheldon writes in the Big Bang Theory. Although, while Sheldon works on String Theory, I worked on Lattice Quantum Chromo Dynamics, a computational approach to QCD, the model that describes how quarks bind together to form the majority of matter in the universe. This is what brought me to the US: I moved here to work as postdoctoral researcher at the Fermi National Accelerator Laboratory. Eventually I realized I had a better chance at an academic career in Computing than in Physics and I accepted a job offer from the School of Computing of DePaul University. I spend most of time programming, doing research, and consulting. I enjoy scuba diving, sailing, and skiing but I do not get to do as much of those as I would like.

Why did you start using Python?

I started using Python about 10 years ago. When I was a physicist I was known for a C++ library for Lattice Quantum Chromo-dynamics computations known as FermiQCD. I created a Python wrapper for it.

What other programming languages do you know and which is your favorite?

I know many programming languages. The ones I know best and I enjoy the most are C, C++, Python, and JavaScript. The first language I learned was COBOL. My father taught me. The second was BASIC. The third was Pascal. The fourth was Clipper and the first program I ever sold was in Clipper. The only language I learned in a formal course was Scheme. I learned it from the famous book “Structure and Interpretation of Computer Programs” and I coded symbolic differentiation. In my Ph.D. thesis I programmed a Cray T3E using High Performance FORTRAN and C++ with MPI. At DePaul I have taught C/C++, Java, Python, JavaScript, and OpenCL. Java is my least favorite language. JavaScript was my second least favorite but I have learned to appreciate it over time and I find myself using it more than Python recently.

What projects are you working on now?

I am working on many projects and some I cannot talk about. Most of the projects are related to the Python language and web2py in particular. I provide support to web2py users and some times help them bootstrap new projects. Other times I help them fix old projects. I am also constantly developing new Python modules to push the boundaries of what web2py can do and explore its future. One project that is taking a lot of my time and not web2py related is Camio. I am a senior engineer for http://camio.com and we have built a platform that turns any device with a camera into a security camera. Images and videos are uploaded and stored in the cloud, analyzed by neural networks, and made searchable using natural language. Most of the code is built with Python, including the Machine Learning.

Which Python libraries are your favorite (core or 3rd party)?

My favorite libraries are pickle and ast. Pickle is a core library that allows to serialize and to de-serialize almost any data structure in Python. That means it is trivial to save and restore state of any program at any time, effortlessly. There is nothing equivalent in compiled languages. Ast is another core library. It allows processing of Abstract Syntax Trees, i.e. a symbolic representation of the code. This enables, for example, real time conversion of Python code into other languages such as C/JavaScript/OpenCL like my project https://github.com/mdipierro/ocl, Cython, and Numba.

Is there anything else you’d like to say?

I love Python for many reasons. One is the easy syntax. Another is the power of its numerical libraries (numpy, scipy, sympy, matplotlib, scikit learn). Yet another reason is the fact that it is an interpreted language and, as such, it can do magical things. For example it can introspect itself with __code__, it can decompile itself and recompile itself into other languages.

If you are reading this is probably because you know me as the lead developer of web2py. You may have heard of it. There is a chance what you have heard does not quite do justice to it. In web2py we tried to do something unique and in may ways we took an approach very different than other Pythonic web frameworks. One unique feature is in its philosophy: in web2py “do not repeat yourself” trumps “explicit is better than implicit” and therefore everything has a default. This means code is very compact but it does not imply code cannot be customized. Another feature is fully batteries included: we ship web2py with a sqlite, a database abstraction layer, a multithreaded SSL-enabled web server, web based IDE, ticketing system for logging errors, caching libraries (redis, memcache), login methods (ldap, pam, openid, etc.), form widgets, validators, payment systems, etc. Web2py was the first in many respects and it was WSGI based, had automatic database migrations, had mandatory CRSF protection, and had a web based IDE since 2007. We are still backward compatible with the 2007 version. Yet we have moved on a lot from there. For example, we had support for Tornado based websockets and asynchronous background task queues for about 5 years. The bottom line is: perhaps you should take a second look into web2py and perhaps you can help us make it even better.

Thanks for doing the interview!

March 21, 2016 12:30 PM


Kay Hayen

Nuitka Release 0.5.20

This is to inform you about the new stable release of Nuitka. It is the extremely compatible Python compiler. Please see the page "What is Nuitka?" for an overview.

This release is mostly about catching up with issues. Most address standalone problems with special modules, but there are also some general compatibility corrections, as well as important fixes for Python3.5 and coroutines and to improve compatibility with special Python variants like AnaConda under the Windows system.

Bug Fixes

  • Standalone Python3.5: The _decimal module at least is using a __name__ that doesn't match the name at load time, causing programs that use it to crash.
  • Compatibility: For Python3.3 the __loader__ attribute is now set in all cases, and it needs to have a __module__ attribute. This makes inspection as done by e.g. flask working.
  • Standalone: Added missing hidden dependencies for Tkinter module, adding support for this to work properly.
  • Windows: Detecting the Python DLL and EXE used at compile time and preserving this information use during backend compilation. This should make sure we use the proper ones, and avoids hacks for specific Python variants, enhancing the support for AnaConda, WinPython, and CPython installations.
  • Windows: The --python-debug flag now properly detects if the run time is supporting things and error exits if it's not available. For a CPython3.5 installation, it will switch between debug and non-debug Python binaries and DLLs.
  • Standalone: Added plug-in for the Pwm package to properly combine it into a single file, suitable for distribution.
  • Standalone: Packages from standard library, e.g. xml now have proper __path__ as a list and not as a string value, which breaks code of e.g. PyXML. Issue#183.
  • Standalone: Added missing dependency of twisted.protocols.tls. Issue#288.
  • Python3.5: When finalizing coroutines that were not finished, a corruption of its reference count could happen under some circumstances.
  • Standalone: Added missing DLL dependency of the uuid module at run time, which uses ctypes to load it.

New Features

  • Added support for AnaConda Python on this Linux. Both accelerated and standalone mode work now. Issue#295.
  • Added support for standalone mode on FreeBSD. Issue#294.
  • The plug-in framework was expanded with new features to allow addressing some specific issues.

Cleanups

  • Moved memory related stuff to dedicated utils package nuitka.utils.MemoryUsage as part of an effort to have more topical modules.
  • Plug-ins how have a dedicated module through which the core accesses the API, which was partially cleaned up.
  • No more "early" and "late" import detections for standalone mode. We now scan everything at the start.

Summary

This release focused on expanding plugins. These were then used to enhance the success of standalone compatibility. Eventually this should lead to a finished and documented plug-in API, which will open up the Nuitka core to easier hacks and more user contribution for these topics.

March 21, 2016 08:34 AM


Talk Python to Me

#51 SigOpt: Optimizing Everything with Python

You've heard that machine intelligence is going to transform our lives any day now. This is usually presented in a way that is vague and non-descript. <br/> <br/> This week on Talk Python To Me you'll meet Patrick Hayes the CTO at SigOpt whose goal is to accelerate your machine learning by "optimizing everything". That's a pretty awesome goal! Listen in on this episode to learn all about it! This is episode number 51, recorded March 3rd 2016. <br/> <br/> Links from the show: <br/> <div style="font-size: .85em;"> <br/> <b>Patrick on Twitter</b>: <a href='https://twitter.com/pfjhayes' target='_blank'>@pfjhayes</a> <br/> <b>SigOpt on Twitter</b>: <a href='https://twitter.com/sigopt' target='_blank'>@sigopt</a> <br/> <b>SigOpt</b>: <a href='https://sigopt.com' target='_blank'>sigopt.com</a> <br/> <b>SigOpt Blog</b>: <a href='https://blog.sigopt.com' target='_blank'>blog.sigopt.com</a> <br/> <b>SigOpt Research</b>: <a href='https://sigopt.com/research' target='_blank'>sigopt.com/research</a> <br/> <b>MOE</b>: <a href='https://github.com/sigopt/moe' target='_blank'>github.com/sigopt/moe</a> <br/> <br/> <b>Michael's Video Course</b>: <br/> <a href='https://training.talkpython.fm/courses/details/python-language-jumpstart-building-10-apps' target='_blank'>https://training.talkpython.fm/courses/details/python-language-jumpstart-building-10-apps</a> <br/> </div>

March 21, 2016 08:00 AM


Vasudev Ram

Motto for Python newbies

By Vasudev Ram




Also see:

10,000 Hours of Practice

Outliers - Malcolm Gladwell book

The Python Interview

- Vasudev Ram - Online Python training and programming

Signup to hear about new products and services I create.

Posts about Python  Posts about xtopdf

My ActiveState recipes


March 21, 2016 03:15 AM


Thomas Guest

Sausages, sausages, sausages - slice, slice, slice

A friend asked for help reaching the next level of a puzzle game. The test which stalled her involves machine placement in a sausage factory.

… each sausage was branded with a letter for quality control purposes, thus: ypbtkizfgxptclcoirdsuhjwulqkoszrabfc

The string was then drawn through seven machines which rearranged the sausages in flavour enhancing ways.

Machine A: The Reversifier

Reverses the order of the sausages, so they get tastier as you go along.

Machine G: Secondhalffirstifier

move the second half of the string to the beginning, as the earlier sausages are too spicy to eat early in the morning.

He attached these machines in a certain sequence, though one of them was out for repair so only six were used. He then fed a string of sausages through and was surprised to discover the string that came out at the other end said lickyourlips. What order were the machines in?

It’s nicely phrased, but what’s really wanted is the sequence of simple transformations that takes input “ypbtkizfgxptclcoirdsuhjwulqkoszrabfc” and produces output “lickyourlips”.

It’s no doubt possible to work backwards and figure out a solution using no more than logic, pencil and paper. For example, only two of the machines change the length of the string, and — looking at the before and after lengths — these must both be used. It’s rather easier to write a short program to find a solution.

First we must simulate the sausage machines. If we represent our string as a list, the seven machines, A-G, perform the following operations.

  1. reverse the order of a list
  2. remove every other element of a list
  3. remove every third element of a list
  4. pairwise reverse elements of a list
  5. move even numbered elements to the front of a list
  6. move the last element of a list to the front
  7. swap the front and back half of a list

None of these is difficult, especially in a high-level language which builds in support for sequence operations. What I found noteworthy is that a solution can be found without any loops or if statements. What’s more, every operation can handled using nothing more than slice operations.

Here’s my solution. The machines consist of slice operations, helped by a couple of conditional expressions and recursive calls. The solution can then be brute-forced: there are only 5040 ways of permuting 6 out of 7 machines.

I’ve used reduce to apply a chain of functions to a string of sausages — an explicit loop might be clearer, but I want a loop-free solution. For this same reason I use recursion in the pairwise swapper and the element dropper. Generally in Python, recursion is a poor choice. In this case I know I’m starting with a string of just 36 elements which cannot get any longer; there’s no risk of exceeding the system recursion limit.

The list reversal s[::-1] is idiomatic but alarming to the uninitiated. Slices have [start:stop:stride] fields, any of which may be defaulted. Usually start and stop default to the start and end of the list, but in this case the negative stride reverses them. The equally idiomatic list(reversed(s)) would be more clear.

To rotate the last element of the list to the front, prefer:

return s[-1:] + s[:-1]

to:

return [s[-1]] + s[:-1]

because the latter raises an IndexError for an empty sequence.

Slicing is a formidable tool for sequence manipulation, especially when combined with the option of using negative indices to count back from the end of the list. Slices allow you to reverse, rotate and partition lists, to pairwise swap elements, and to drop every nth element.

The miniature recipes presented here don’t even use slice assignment, which gives me an excuse to reproduce this elegant prime sieve function, which does.

March 21, 2016 12:00 AM

March 20, 2016


Python Diary

Introducing PyCARS!

I recently purchased a second Raspberry Pi, the model 2, and a 7" touch screen. Originally I was going to use it to build a home automation HUD which I could place either on my wall or in my living room somewhere. However, after attaching the LCD to the Pi, I found it a bit too bulky for such an elegant task... My idea now is to build it out into a bedside HUD/alarm clock system. Of course, this HUD system will be built in the Python language, and so I am here to share my current implementation, and a progress report with my fellow Python peers.

During the development, I needed to figure out some sort of UX/UI to use for this, and since I have a good passion for the Star Trek series, I thought using their LCARS design ideas would be a good base to start with. Currently, there really is no design or anything trek related(besides the sound effects) in the project. However, as the project matures, I am hoping to develop it into more of an app platform, where I can easily extend the functionality for new use-cases. Currently, it can act as a basic clock with a huge display. The display itself can be dimmed easily, and can also be completely turned off when not in use or when your asleep. There is a basic alarm system in place that cannot be configured without updating the Python code. I also built in some hue control functionality into it, so that I can easily control my bedroom and master bathroom lights with ease.

To keep this open source project from copyright and legal issues, no assets, such as images or sound files will be included alongside the source code. Also, the project will never make mention to LCARS, only PyCARS, which isn't copyrighted or trademarked by CBS Interactive. CBS has been known to take down software with LCARS interfaces, such as the Tricorder app for Android. Although, they don't seem to take down every LCARS related fan project, which seems very weird... I am unsure why they target some projects, and leave others alone.

You can view and download the source code repository from my BitBucket page PyCARS

Since this page might be noticed by a CBS interactive lawyer one day, here are some other LCARS related projects which have been around for a lot longer, which legal action has not been taken yet:

Just in case: Star Trek, Star Trek: The Next Generation are trademarks of PARAMOUNT PICTURES and CBS. In NO CASE is use of any copyrighted material, or any related, derived or inferred ideas intended as a claim of ownership to those copyrights/trademarks. I am in NO way associated with Paramount Pictures or CBS.

Furthermore, I will not be making any money, nor selling any material copyrighted by CBS Interactive. This project will remain open source, under a license I still need to consider. This Python project will also never be distributed with copyrighted material, such as images and sounds created by CBS Interactive. If the end-user using this Python project so rightfully chooses to violate CBS Interactive copyright and trademarks and uses LCARS images, fonts, and sounds, the end-user will remain fully liable by law. However, I highly doubt CBS interactive will pursue a John Doe for using LCARS images and sounds in the personal confines on his own home. However, if this project ends up being used by a larger corporation for UI/UX(although I highly doubt that will ever happen), it is their duty to provide whatever image, font, and sound assets suite their particular interface requirements. I am being as clear as possible here, that I am absolutely not distributing any CBS Interactive copyrighted or trademarked material with this open source Python project. CBS Interactive can clearly download the entire codebase and fully examine it for any copyrighted or trademarked data. However, since the LCARS system used in the show isn't really a real computer program doing stuff, and I highly doubt that the LCARS system for the TV/movies is made in Python... I should be-able to fully rest my case that this project is absolutely and definitely 100% legal. So take your best shot CBS Interactive, you've got nothing on this project.

March 20, 2016 10:48 PM


Python 4 Kids

Python for Kids: Python 3 – Project 2

Some people want to use my book Python for Kids for Dummies to learn Python 3. Choosing Python 2.7 over Python 3 was a difficult decision and I have given reasons why in the book.* Nevertheless, if I write a new edition of the book, it definitely will be in Python 3, so I plan to work through the code in the existing book, highlighting changes from Python 2 to Python 3 and providing code that will work in Python 3.

I am working from the downloadable code samples (they are cross referenced to page numbers in the book), so it might be an idea to get a copy, although working from the hard copy should also be fine. Get a copy from the link in the right hand sidebar.

For Project 2, most of the code works exactly the same in Python 2.7 and Python 3. There are some changes though later in the Project (from page 50). Those changes are set out below (page numbers from the 2015 printing).

Code on Pages 36 to Page 50

All of the code on these pages works in Python 3 and gives the same output.

Code on Page 50

#Python 2.7 code:
>>> my_message = 'Hello World!'
>>> while True:
...       print(my_message),
...

Comment

In Python 2.7 you use the comma -> , to tell print to NOT include a new line at the end of what is printed.
In Python 3 print has become a function. Functions are not discussed till Project 5! Implementing this in Python 3 needs a lot of extra concepts, that I’m not going to explain here. Instead, I’m just going to give you working code. You will need to come back to this after you’ve done Project 5. Hopefully then it will make more sense.

#Python 3
>>> while True:
...       print(my_message, end=&amp;quot; &amp;quot;)
...

Code on Page 51

#Python 2.7 code:
>>> range(3)
[0, 1, 2]

Comment

In Python 2.7 range() creates a list. In Python 3 it makes something like a generator** – and generators are not even covered in the book! :( The main difference is that, with a list, all the items that you need are created ahead of time. However, with a generator, you only create the next item when you need it. Practically, the code will work in the same manner and you won’t be able to notice any difference (see the examples on the next page). For example, the numbers produced by range in Python 3 will start at 0 and run up to the number one less than the number you give to range. The good news though is that you can ignore my warning (on page 52) about using range() if you’re running Python 3. In Python 3 range doesn’t blow out your memory, so feel free to use it for numbers as big as you like.

#Python 3 code:
>>> range(3)
range(0, 3)

Code on Page 52

#Python 2.7 code:
>>> range(3,10)
[3, 4, 5, 6, 7, 8, 9]

>>> range(3,10,2)
[3, 5, 7, 9]

Comment

See comments on generators above. You won’t be able to see the practical difference until you cover for loops (see below).

#Python 3 code:
>>> range(3,10)
range(3, 10)

>>> range(3,10,2)
range(3, 10, 2)

Code on Page 53

#Python 2.7 code:
>>> range(13,10,-1)
[13, 12, 11]

Comment

Same comments on generators. See below.

#Python 3 code:
>>> range(13,10,-1)
range(13, 10, -1)

#Python 2.7 code:
>>> for i in range(3):
...        print(i)
...
0
1
2

#Python 3 code:
>>> for i in range(3):
...        print(i)
...
0
1
2

Comment

In this example both the code and the output from the Python 2.7 and Python 3 code was the same. However, the code created the output in different ways (the Python 2.7 code created a list, while the Python 3 code created a generator). Anywhere you use range in the book, you can use it when using Python 3.

Also, using this for loop structure you can test to see whether the earlier ranges are practically the same in Python 2.7 and Python 3.
For example (from page 52):

#Python 2.7 code:
>>> range(3,10)
[3, 4, 5, 6, 7, 8, 9]

>>> range(3,10,2)
[3, 5, 7, 9]

#Python 3 code:
>>> for i in range(3,10):
...        print(i)
...
3
4
5
6
7
8
9

# note same numbers as in [3, 4, 5, 6, 7, 8, 9]

>>> for i in range(3,10,2):
...        print(i)
...
3
5
7
9

# note: odd numbers from 3 to 10

Code on Page 54

#Python 2.7 code:
>>> my_message = &amp;quot;Hello World!&amp;quot;
>>> for i in range(300):
...          print(my_message),
...

Comment

See comments on print above.

#Python 3 code:

>>> my_message = &amp;quot;Hello World!&amp;quot;
>>> for i in range(300):
...          print(my_message, end=&amp;quot; &amp;quot;)
...

Note that, in Python 2.7 when the print run had finished the >>> prompt started on a new line. However, in Python 3 the prompt >>> starts immediately after the last Hello World! on the same line.

Note:
* I had dreams of extending the book for parents to include, for example, loading applications to the cloud. Right now (March 2016, a year after I finished writing the book) Google App Engine, perhaps the easiest way to get an app live on the internet, still does not support Python 3 (although it is available through a separate service – GAE Managed VM hosting).

** It creates an “immutable sequence type” according to the docs. See Terry’s comment.


March 20, 2016 10:23 PM


Grzegorz Śliwiński

Running python tests on Travis's OSX workers

I, and my colleages are running tests mostly on linux, since our packages usually are clean python implementation. However, for mirakuru, which we use to govern processes in tests, and by extension pytest-dbfixtures, which has predefined process fixtures for pytest, we wanted to run tests also on osx. Travis.ci for a long time had option to configure osx workers and linux, however python builds on osx are kind of broken by default.

Travis.ci has a long standing bug report for this issue, which will get 2 years old in this May. Apparently, just setting python as a language and osx among the system to test on won't work for apple's operating system.

Read more… (2 min remaining to read)

March 20, 2016 08:44 PM


Kracekumar Ramaraju

Permissions in Django Admin

Admin dashboard is one of the Django’s useful feature. Admin dashboard allows super users to create, read, update, delete database objects. The super users have full control over the data. Staff user can login into admin dashboard but can’t access data. In few cases, staff users needs restricted access . Super user can access all data from various in built and third party apps. Here is a screenshot of Super user admin interface after login.

Staff users don’t have access to data.

Allow staff user to access models

Django permissions determines access to models and allowed actions in admin interface. Every model has three permissions. They are <app_label>.add_<model>, <app_label>.change_<model>, <app_label>.delete_<model> allows user to create, edit and delete objects.

API and Admin interface allows assigning permissions to the user.

Staff user can perform various tasks on allowed models after assigning permissions.

Filtering objects in model

Conference management system hosts many conferences in a single instance. Each conference has different set of moderators. System allows only conference specific moderators to access the data. To achieve the functionality, Django provides an option to override queryset. Admin requires custom implementation of get_queryset method. Here is how a sample code looks like.

class ConferenceAdmin(AuditAdmin):
    list_display = ('name', 'slug', 'start_date', 'end_date', 'status') + AuditAdmin.list_display
    prepopulated_fields = {'slug': ('name',), }

    def get_queryset(self, request):
        qs = super(ConferenceAdmin, self).get_queryset(request)
        if request.user.is_superuser:
            return qs
        return qs.filter(moderators=request.user)

class ConferenceProposalReviewerAdmin(AuditAdmin, SimpleHistoryAdmin):
    list_display = ('conference', 'reviewer', 'active') + AuditAdmin.list_display
    list_filter = ('conference',)

    def get_queryset(self, request):
        qs = super(ConferenceProposalReviewerAdmin, self).get_queryset(
        request)
        if request.user.is_superuser:
            return qs
        moderators = service.list_conference_moderator(user=request.user)
        return qs.filter(conference__in=[m.conference for m in moderators])

Filtered moderator objects for staff user.

Unfiltered moderator objects for superusers.

Note the difference in total number of objects (23, 30) in the view.

March 20, 2016 05:07 PM


Sylvain Hellegouarch

An asynchronous CherryPy server based on asyncio

CherryPy is a minimalist web application server written in Python. Hundreds of people have relied on it for more than fourteen years now. Recently, I’ve gained interest in the native asynchronous support Python has gained through the implementation of PEP-3156 and PEP-0492. Basically, Python now supports coroutines natively and in a friendly API interface. In addition, thanks to the built-in asyncio module, you can naturally develop concurrent applications.

CherryPy has always used a multi-threaded engine to support concurrent web applications. This may sound surprising, but this is working very well still in 2016. Yet, I love a puzzle and I was interested in making CherryPy run as a set of coroutines rather than a bunch of threads. So a couple of days ago, I set myself on the task to make it happen.

And so here it is, CherryPy on asyncio!

This a branch on my fork, not an official part of the CherryPy project yet.

Now, you can run code like this:

import cherrypy
import cherrypy.async

class Root:
    @cherrypy.expose
    async def index(self):
        return "hello"

    @cherrypy.expose
    async def echo(self, msg):
        return msg
    
if __name__ == '__main__':
    cherrypy.quickstart(Root())

The only differences are:

That is all.

The idea is that the cherrypy.async patches all the internals of CherryPy for you and turn it into an async-aware server.

Note that the code currently runs only on Python 3.5+ as we use the async/await keywords.

Has it been easy?

Turning a project that was not designed for coroutines is not that complicated thanks to the simple interface provided by async/await. However, anytime an I/O operation is performed, it is necessary to transform a call like:

data = fp.readline()

to

data= await fp.readline()

This mundane line is sometimes is part of a function call, in that case, you have to overwrite and copy/paste the whole function to change that one line. Mind you, this can’t be avoided because you must also re-declare the function as a coroutine anyway by prefixing it with async.

As usual, the difficulty lies in the entanglement of your code. The more you made simple to comprehend, the simpler and faster it will be to change with confidence.

What was changed?

Mostly the HTTP server, the machinery of CherryPy: the internal engine/bus, the dispatcher, the request handling.

If we can find a way to re-organise the existing code, actually few lines would eventually be changed.

Note, I made the decision not to make this server WSGI aware because I find that rather counter-intuitive with an async-based server somehow.

Is it production ready?

Not at all. It hasn’t been really tested (this will require to re-write many tests so that they play along with coroutines).

It is also, for some unknown reason yet, much slower than the multithreaded version. Profiling will need to be performed.

Still, if you are feeling like testing it:

$ hg clone https://Lawouach@bitbucket.org/Lawouach/cherrypy
$ cd cherrypy
$ hg update async

I don’t know if this code will go further than this but, maybe this will interest the community enough so that it moves forward. This would make CherryPy more suitable for HTTP2 and websockets.

March 20, 2016 03:15 PM


Programando Ciência

[EVENT] SciPy Latin America 2016

Hey scientist! Have you ever thought about participating in the largest Latin America event about scientific Python, and connecting to the members of the coolest programming community of the world? That’s right: this year will be easier for us Brazilians to go to SciPyLA! The event will be hosted in Florianópolis, the island of magic! […]

March 20, 2016 06:47 AM


Kushal Das

Day Two of FOSSASIA 2016

I had to leave early to the venue for day two, as I had a welcome talk in the Python track. The morning started with the “Introduction to GSOC, and GCI” talk from Stephanie Taylor. The room was full with many ex-GSOC and GCI students, and mentors. The students of GCI last year completed more than 4k tasks, among them 1k+ was done by the students under FOSSASIA organization.

After this talk I moved to the Python track room. With a small welcome talk we started the first talk in the track, “Test driven development with pytest” from Ivan Zimine. The Goat book was there in the slides. He gave example of a very simple Flask application, and then how to test it with pytest. At the end few users asked about preference between Django and Flask :)

Next Sayan started his talk about Fedora Infrastructure. He gave a nice overview of the different applications running in the infrastructure, and how one can join the team. During the QA session, one person asked how do we choose which projects to work on? I think this answer should get it’s own blog post.

Next, I was up for the talk about Tunir. I demoed the latest features (again, more details are coming next week). My talk slides are here.

After lunch I moved to the “Internet, Society, Community” track as Anwesha had her talk there. I managed to stay inside the room for the most part of Anivar’s talk about India’s fight for Netneutrality and Open Internet, but Py was crying, so I had to leave the room. Luckily she slept just before Anwesha’s talk, so we managed to get back into the room. Anwesha gave her talk on a FOSS conference titled Anatomy of a Software Patent for a foss developer. I am sure she will write more about it in her blog.

From 4PM I had my “Python3 101” workshop. There were many newcomers in the workshop, so I tried to explain basic things more. We saw few differences between Python 2, and Python 3. At the end of the workshop we wrote our own “ls” replacement in Python. Below is a picture few students who came down from Dubai to attend FOSSASIA, they participated in my workshop along with their teacher.

March 20, 2016 03:09 AM

March 19, 2016


Julien Tayon

I still find with the examples used for teaching OOP counterproductive

After the stupid example with the taxonomy of the species/car/employees that makes you do a lot of inheritance one of the valid use case for OOP is 2D geometry : points & rectangles.

This is the python example in wiki.python.org
https://wiki.python.org/moin/PointsAndRectangles

Don't worry, python is one of the many language doing this class. It is an actual use case for learning classes.

200 lines of code and you do nothing. Not even fun

Except in python complex is a base type, and I learned the use of complex numbers when I was  in public high school in my sweet banlieue. (a place said to be full of delinquents and uneducated kids)

Then I thought : with just simple math how hard would it be to just draw two polygons one of which rotated with the python base types? 
How hard is programming when you keep it simple?

Here is my answer in less than 10% of the initial number of lines of code and a drawing to illustrate how hard it is :


from PIL import Image, ImageDraw
# short constant names a la fortran
from cmath import pi as PI, e as E
I = complex("j")
 
to_x_y = lambda cpl: (cpl.real, cpl.imag)
im = Image.new("RGB", (512, 512), "white")
draw = ImageDraw.Draw(im)
rotation = E**(I*PI/3)
homothetia = min(im.size[0], im.size[1])
trans = homothetia/2
homothetia /= 2.5
polygone = (complex(0,0), complex(0,1), complex(1,1), complex(1,0), complex(0,0))
bigger = map(lambda x: x * homothetia + trans, polygone)
bigger_rotated = map(lambda x: x * rotation, bigger)
draw.line(map(to_x_y, bigger), "blue")
draw.line(map(to_x_y, bigger_rotated), "green")

im.save("this.png", "PNG")

I don't say classes are useless, I use them. But, I still have a hard time thinking of a simple  pedagogic example of their use. However, I see a lot of overuse of Object Oriented Programming to try to make people able to use concepts that are not accessible without basic knowledge. OOP is no shortcut for avoiding to learn math, or anything else. If you don't understand geometry and math, you will probably be whatever the quality of the class given unable to do any proper geometrical operations.

We need OOP. But the basic complex type far outweigh in usefulness any 2D geometry classes I see so far without requiring any dependencies.  And I never saw in 5 years complex types used even for doing 2D geometry, however it seems the alternative buzzword to OOP -including numpy- is so making you look data scientist and pro! 

So what is the excuse again for not using something that works and is included in base types with no requirements for doing geometry when we say that games could be a nice way to bring teenagers to coding? Are we also gonna tell them learning is useless and that reinventing the square wheel is a good idea? Are we gonna tell them to not learn because we have libraries for everything?

For the fun I did an OOP facade to complex as point/rectangle but seriously, it is just pedantic and useless. But still 50% less lines and more useful than the initial example.




March 19, 2016 11:05 PM


Catalin George Festila

Free ebook from O'Reilly - Functional Programming in Python.

You can download your free ebook from O'Reilly.
The Functional Programming in Python by David Mertz - publisher: O'Reilly - released: June 2015.
David Mertz is a director of the Python Software Foundation, and chair of its Trademarks and Outreach & Education Committees. He wrote the columns Charming Python and XML Matters for IBM developerWorks and the Addison-Wesley book Text Processing in Python. David has spoken at multiple OSCON and PyCon events.
This is the download link.

 

March 19, 2016 06:38 PM


PyPy Development

PyPy 5.0.1 bugfix released

PyPy 5.0.1


We have released a bugfix for PyPy 5.0, after reports that the newly released lxml 3.6.0, which now supports PyPy 5.0 +, can crash on large files. Thanks to those who reported the crash. Please update, downloads are available at

pypy.org/download.html

The changes between PyPy 5.0 and 5.0.1 are only two bug fixes: one in cpyext, which fixes notably (but not only) lxml; and another for a corner case of the JIT.

What is PyPy?


PyPy is a very compliant Python interpreter, almost a drop-in replacement for CPython 2.7. It’s fast (PyPy and CPython 2.7.x performance comparison) due to its integrated tracing JIT compiler.
We also welcome developers of other dynamic languages to see what RPython can do for them.
This release supports x86 machines on most common operating systems (Linux 32/64, Mac OS X 64, Windows 32, OpenBSD, FreeBSD), newer ARM hardware (ARMv6 or ARMv7, with VFPv3) running Linux, and the big- and little-endian variants of PPC64 running Linux.

Please update, and continue to help us make PyPy better.

Cheers
The PyPy Team

March 19, 2016 05:44 PM


Python 4 Kids

Python for Kids Book: Project 2

In these posts I outline the contents of each project in my book Python For Kids For Dummies.  If you have questions or comments about the project listed in the title post them here. Any improvements will also be listed here.

What’s in Project 2

Project 2 is a Hello World project that covers some important basics, including what literals are, how you name a literal in order to store it (ie variables). It shows the different ways to make a string literal.If you’re going to name a value you need to know about Python’s naming rules (and PEP8 naming conventions – yah but I don’t actually mention PEP8). Those naming rules themselves rely on you not using a keyword as a name, so you also get a list of Python keywords in this project.

It also covers Python’s print statement, looping with while (including the notion of a conditional and a code block). looping with for and counting with range. By the end of the Project you can fill the screen with “Hello World!”.  To do so you repeat something 300 times. I introduce the concept of magic numbers and explain why they should be avoided. I offer the use of variables in ALL_CAPS to use as constants.

Update March 2016: The book is designed to teach you Python 2.7. However, Python 3 is on the way, and if you’d like to see what’d be different in this project if you use Python 3 see this post.


March 19, 2016 12:40 PM


Weekly Python StackOverflow Report

(xi) stackoverflow python report

These are the ten most rated questions at Stack Overflow last week.
Between brackets: [question score / answers count]
Build date: 2016-03-19 09:09:01 GMT


  1. Counterintuitive behaviour of int() in python - [31/5]
  2. Is there a need to close files that have no reference to them? - [30/6]
  3. Why did Django 1.9 replace tuples () with lists [] in settings and URLs? - [22/2]
  4. Edit the value of every Nth item in a list - [12/7]
  5. What is a DynamicClassAttribute and how do I use it? - [10/1]
  6. Class inheritance in python - [8/10]
  7. Returning Dictionary-length of words in string - [8/7]
  8. Python 'with' not deleting object - [7/4]
  9. Linear shift between 2 sets of coordinates - [7/3]
  10. Sum of multiple list of lists index wise - [7/3]

March 19, 2016 09:09 AM


BangPypers

March 2016 BangPypers Workshop Report

March Bangpypers meetup happened at the calm.io. 30 people attended the meetup. Session started around 10:30.

Jitendra Agrawal conducted workshop on Solr and Indrajit Rajtilak volunteered for the session.

Apache Solr is a fast, open source and easy to use search platform built on top of Apache Lucene. He explained the limitations of database search functionality and why solr is required for searching. Then they helped participants to install and run basic queries on solr. Later speaker about customising solr.

Next, they created a simple django app. Using django-haystack, they built an index from django models. Once index is built, they quired index from CLI and also created a view to query the index.

There are lot of questions from participants and session was very interactive. You can checkout slides from here and django app code is available here.

Here are a few pictures from workshop.

I1

I2

I3

Thanks to calm.io for sponsoring lunch & t-shirts.

If you wanted to give talks or conduct workshop, please leave a comment on meetup page.

March 19, 2016 06:25 AM


Vasudev Ram

Python generators are pluggable

By Vasudev Ram


Generator image attribution

While working on a Python project, it crossed my mind that generators could be of use in it. A little research made me realize that generators are pluggable, i.e. they can be passed to functions, and then be used within those functions. This is because generators are a kind of Python object, and any Python object can be passed as an argument to a function.

This in turn is because almost everything in Python is an object (including generators), similar to how almost everything in Unix is a file. Both those concepts can enable some powerful operations.

Here is a program that demonstrates passing generator objects as arguments to another function, and then using those generators inside it:
# Program to show that generators are pluggable, i.e.,
# can be passed as function arguments, and then used
# inside those functions to which they are passed.
# Author: Vasudev Ram - http://jugad2.blogspot.com
# Copyright 2016 Vasudev Ram

def gen_squares(fro, to):
'''A generator function that returns a generator
that returns squares of values in a range.'''
for val in range(fro, to + 1):
yield val * val

def gen_cubes(fro, to):
'''A generator function that returns a generator
that returns cubes of values in a range.'''
for val in range(fro, to + 1):
yield val * val * val

def use(gen):
print "In use() function:"
print "Using:", gen
print "Items:",
for item in gen:
print item,
print

print "Pluggable Python generators.\n"
print "In main module:"
print "type(use): ", type(use)
print "use:", use
print
print "type(gen_squares): ", type(gen_squares)
print "gen_squares: ", gen_squares
print "type(gen_squares(1, 5)): ", type(gen_squares(1, 5))
print "gen_squares(1, 5): ", gen_squares(1, 5)
print
print "type(gen_cubes): ", type(gen_cubes)
print "gen_cubes: ", gen_cubes
print "type(gen_cubes(1, 5)): ", type(gen_cubes(1, 5))
print "gen_cubes(1, 5): ", gen_cubes(1, 5)
print
for gen_obj in (gen_squares(1, 5), gen_cubes(1, 5)):
use(gen_obj)
print
Run the program with:
python pluggable_generators.py
Here is the output:
Pluggable Python generators.

In main module:
type(use): <type 'function'>
use: <function use at 0x0202C3B0>

type(gen_squares): <type 'function'>
gen_squares: <function gen_squares at 0x0207BF30>
type(gen_squares(1, 5)): <type 'generator'>
gen_squares(1, 5): <generator object gen_squares at 0x020869B8>

type(gen_cubes): <type 'function'>
gen_cubes: <function gen_cubes at 0x0207BFB0>
type(gen_cubes(1, 5)): <type 'generator'>
gen_cubes(1, 5): <generator object gen_cubes at 0x020869B8>

In use() function:
Using: <generator object gen_squares at 0x020869B8>
Items: 1 4 9 16 25

In use() function:
Using: <generator object gen_cubes at 0x020869E0>
Items: 1 8 27 64 125
As you can see, I've printed both type(obj) and obj for many of the objects shown, to make it more clear what is going on. Also, a generator function and a generator object (the result of calling a generator function), are two different things, so they are printed separately as well.

A few points about generators and their use:

They can potentially lead to less memory usage, since values are only generated on demand, i.e. evaluation is lazy.

They can help with separation of concerns, a key technique that leads to program modularity; the code for the actual generator functions like gen_squares and gen_cubes does not have to be embedded in the use() function, which makes both the generators and the use() function more reusable.

Someone could say here that we could write gen_squares and gen_cubes as regular functions instead of as generator functions, and then just call them from use(), so their code still does not have to be embedded in the use() function, and that would be right. But in that case, the calls to them would return lists, and if the lists were very large, that would use a lot of memory, and maybe crash or slow down the program. Those issues will not happen with generators, though, because each item is generated just before it is used, and then it is thrown away, not stored. So the memory needed is not proportional to the number of items generated.

Here are some links about Python generators:

Generators - Python Wiki

Stack Overflow - Understanding generators in Python

The image at the top of the post is a Ferranti two-phase AC generator set.

- Vasudev Ram - Online Python training and programming

Signup to hear about new products and services I create.

Posts about Python  Posts about xtopdf

My ActiveState recipes

March 19, 2016 01:31 AM

March 18, 2016


Chris Hager

Find broken hyperlinks in a PDF document with PDFx

PDFx is a free command-line tool to extract references, links and metadata from PDF files. You can also use it to find broken links in a PDF file, using pdfx -c:

PDFx Link Checker

For each URL and PDF reference, pdfx performs a HEAD request and checks the status code. It there are broken links, PDFx print the link with the page number where the link was found in the original pdf:

$ pdfx https://weakdh.org/imperfect-forward-secrecy.pdf -c
Document infos:
- CreationDate = D:20150821110623-04'00'
...

Summary of link checker:
33 working
1 broken (reason: 303)
  - http://www.nytimes.com/interactive/2013/11/23/us/politics/23nsa-sigint-strategy-document.html (page 13)
1 broken (reason: 403)
  - http://www.nytimes.com/interactive/2013/11/23/us/ (page 13)
1 broken (reason: 404)
  - https://github.com/bumptech/stud/blob/ (page 13)

Installing PDFx

You can simply install PDFx with easy_install or pip and run it like this:

$ sudo easy_install -U pdfx
...
$ pdfx <pdf-file-or-url>

Run pdfx -h to see the help output:

$ pdfx -h
usage: pdfx [-h] [-d OUTPUT_DIRECTORY] [-c] [-j] [-v] [-t] [-o OUTPUT_FILE]
            [--version]
            pdf

Extract metadata and references from a PDF, and optionally download all
referenced PDFs. Visit https://www.metachris.com/pdfx for more information.

positional arguments:
  pdf                   Filename or URL of a PDF file

optional arguments:
  -h, --help            show this help message and exit
  -d OUTPUT_DIRECTORY, --download-pdfs OUTPUT_DIRECTORY
                        Download all referenced PDFs into specified directory
  -c, --check-links     Check for broken links
  -j, --json            Output infos as JSON (instead of plain text)
  -v, --verbose         Print all references (instead of only PDFs)
  -t, --text            Only extract text (no metadata or references)
  -o OUTPUT_FILE, --output-file OUTPUT_FILE
                        Output to specified file instead of console
  --version             show program's version number and exit

For more examples and infos, take a look at the PDFx project page. You can find the code on Github, the code is released under the Apache license.


Feedback, ideas and pull requests are welcome! You can also reach me on Twitter via @metachris.

March 18, 2016 11:00 PM