Learning Python with the Python Challenge (levels 0 – 6)

Posted on January 16th, 2011

One of my professional goals, as I’ve mentioned, is to learn and get good at Python. Today it’s cold out and having left my wallet in a cab the previous evening, I haven’t been in a good mood for leaving the house. A friend suggested that I try out the Python challenge (http://www.pythonchallenge.com/) as a fun way to learn the language. And what a wonderful waste of an afternoon! Jesse and I cozied up on the couch and turned on the fake fireplace and got to it. I’m sure these are not the best solutions, but they are mine as a complete newb to Python, and so far I’ve been having a blast. All were done directly in the Python interpreter.

ZERO:
As it should be, the challenge starts with level 0. It’s a picture of a monitor with a picture of 2 to the power of 38 on the screen. The hint is right in the page display instructing you to change the URL. This one of course is easy, 2**38 gives you the number to copy and paste in the URL.

ONE:
Now we get to start playing around with some fun brainstorming. This one has a a string of text that doesn’t make sense along with the clue, which is K->M, O->Q and E->G. As the clue suggests, you are supposed to have all letters in the string of text advance by 2.

First, I assigned the text to a variable:

jumble = """g fmnc wms bgblr rpylqjyrc gr zw fylb. rfyrq ufyr amknsrcpq ypc dmp. bmgle gr gl zw fylb gq glcddgagclr ylb rfyr'q ufw rfgq rcvr gq qm jmle. sqgle qrpgle.kyicrpylq() gq pcamkkclbcb. lmu ynnjw ml rfc spj."""

Next I imported the string library since I figured we would need to work with it. One thing I think is really rad about Python is the dir function, which gives you a list of functions and attributes available in a library or on an object. So, after importing string, entering dir(string) into the interpreter gives you this:

['Formatter', 'Template', '_TemplateMetaclass', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '_float', '_idmap', '_idmapL', '_int', '_long', '_multimap', '_re', 'ascii_letters', 'ascii_lowercase', 'ascii_uppercase', 'atof', 'atof_error', 'atoi', 'atoi_error', 'atol', 'atol_error', 'capitalize', 'capwords', 'center', 'count', 'digits', 'expandtabs', 'find', 'hexdigits', 'index', 'index_error', 'join', 'joinfields', 'letters', 'ljust', 'lower', 'lowercase', 'lstrip', 'maketrans', 'octdigits', 'printable', 'punctuation', 'replace', 'rfind', 'rindex', 'rjust', 'rsplit', 'rstrip', 'split', 'splitfields', 'strip', 'swapcase', 'translate', 'upper', 'uppercase', 'whitespace', 'zfill']

Awesome! I decided to use string.letters, which is a list of the alphabet first in lowercase and then upper. To use it, I cycled through every letter in jumble, found its location in string.letters, and then the letter that was in the location 2 after it and strung those all together:

nonjumble = ""
for letter in jumble:
  a = string.letters.find(letter)
  nonjumble += string.letters[a+2]

After doing this, I saw right away that I had an easier to read string, but with b’s where all the spaces should be. That’s because it would return -1 when looking for the space, advance that by 2, and string.letters[2] is the letter ‘b’. Easy fix:

nonjumble.replace("b"," ")

This works just fine, and prints off this:
“i hope you didnt trAnslAte it By hAnd thAts whAt computers Are for doing it in By hAnd is inefficient And thAt s why this text is so long using string mAketrAns is recommended now Apply on the url”

We did play around with this one a bit more, mostly so Jesse could show off how to make Python look like Perl and we came up with this that can be used in place of the above code:

''.join(string.letters[string.letters.find(x)+2] if x != ' ' else x for x in jumble)

And that worked well enough as well. It’s a lot of work to just find out that you need to advance the URL letters by 2, so http://www.pythonchallenge.com/pc/def/map.html becomes http://www.pythonchallenge.com/pc/def/ocr.html. Someday I’ll read more about maketrans, but I was excited to move on.

TWO:
This one tells you straight off the bat to look at the page source for clues. The source tells you to look for the rare characters and then gives a very long string of stuff like this:
“%%$@_$^__#)^)&!_+]!*@&^}@[@%]()%+$&[(_@%+%$*^@$^!+]!&_#)_*}{}}!}_]$[%}@[{_@#_^{“

First off I pulled the page source and copied the string to a variable:

import urllib
fd = urllib.urlopen("http://www.pythonchallenge.com/pc/def/ocr.html")
mess = fd.read()
messarr = mess.split("--")

You can look at the source to see what I’m doing here. I just want the actual string that I’m evaluating, and it was the second set in between comment tags (and “--” wasn’t used anywhere within it). Anyways, at this point, the string that needs to be dealt with is in messarr[3].

Just looking at it, we didn’t see any actual letters, so we figured to find out which of the characters were “rare”, meaning not used as much in the string. We could have made this a lot easier on ourselves. But first off, we created a frequency distribution:

import collections
histogram = collections.defaultdict(lambda: 0)
for char in messarr[3]:
    histogram[char] += 1

After doing that, histogram is now:

defaultdict( at 0xa2a780c>, {'\n': 1221, '!': 6079, '#': 6115, '%': 6104, '$': 6046, '&': 6043, ')': 6186, '(': 6154, '+': 6066, '*': 6034, '@': 6157, '[': 6108, ']': 6152, '_': 6112, '^': 6030, 'a': 1, 'e': 1, 'i': 1, 'l': 1, 'q': 1, 'u': 1, 't': 1, 'y': 1, '{': 6046, '}': 6105})

So, obviously there were some letters in there! This is all that one needs to do to solve #2 after all:
(regular expressions!)

re.sub(r'[^a-z]', '', messarr[3])

Which gives the word “equality”, and the answer to challenge #2.

THREE:
The third one also has a long text of junk in the source. This time the clue says “One small letter, surrounded by EXACTLY three big bodyguards on each of its sides.” Sounds like another use for the regex library!

page = urllib.urlopen("http://www.pythonchallenge.com/pc/def/equality.html").read()

This time I didn’t bother parsing out the messy string since I figured the pattern I was looking for probably didn’t exist anywhere in the html and I would find out quickly if it did. Next we did a findall for the lowercase letters that were surrounded by three and only three uppercase letters, and then joined them all together:

''.join(re.findall(r'[^A-Z][A-Z]{3}([a-z])[A-Z]{3}[^A-Z]', page))

That gave the answer to problem number three, “linkedlist”

FOUR:
Four is where it starts to get a little weird, and AWESOME. The page has an image with a link to “linkedlist.php?nothing=12345” and in the HTML comments gives the clue “urllib may help. DON'T TRY ALL NOTHINGS, since it will never end. 400 times is more than enough.” Clicking on the link brings you to a page that says: “and the next nothing is 92512”, and so on.

First off, I created a function called get_nothing in order to use a number to read from the URL and return the next number. This is the first get_nothing function:

def get_nothing(next_nothing):
    contents = urllib.urlopen("http://www.pythonchallenge.com/pc/def/linkedlist.php?nothing={next_nothing}".format(next_nothing=next_nothing)).read()
    new_nothing = re.search(r'\d+', contents)
    return new_nothing

The main mistake here was that I assumed that each result would have only one number in it.

To use the function, I created a list to keep track of what URLs it is visiting, and then populated it. I figured that since it said 400 would be more than enough, that seemed like a safe number to loop through, and of course it would stop before then anyways:

nothing_collection = []
nothing_collection.append('12345')
for a in xrange(400):
    last_nothing = nothing_collection[a]
    new_nothing = get_nothing(last_nothing).group()
    nothing_collection.append(new_nothing)

After doing that, I looked at nothing_collection and saw that the last number in it was 61067. I took a look at http://www.pythonchallenge.com/pc/def/linkedlist.php?nothing=61067 and it let me know that “You've been misleaded [sic] to here. Go to previous one and check.” Jerks! The previous one, 61066, was a trick since it has 2 numbers: “There maybe [sic] misleading numbers in the text. One example is 61067. Look only for the next nothing and the next nothing is 53522”

Easy enough, get_nothing’s regular expression just needed a little adjustment; this is what I replaced it with:

def get_nothing(next_nothing):
    contents = urllib.urlopen("http://www.pythonchallenge.com/pc/def/linkedlist.php?nothing={next_nothing}".format(next_nothing=next_nothing)).read()
    new_nothing = re.search(r'the next nothing is (\d+)', contents)
    return new_nothing.group(1)

Since I had it call new_nothing.group this time, I altered the nothing_collection code a little too:

nothing_collection = []
nothing_collection.append('12345')
for a in xrange(400):
    last_nothing = nothingcollection[a]
    new_nothing = getnothing(last_nothing)
    nothing_collection.append(new_nothing)

This one takes a while to run! But, it ends at 92118. http://www.pythonchallenge.com/pc/def/linkedlist.php?nothing=92118 instructs you to divide by two and keep going. Really?

start = len(nothing_collection)
nothing_collection.append('46059')
for a in xrange(start, 400):
    last_nothing = nothing_collection[a]
    new_nothing = get_nothing(last_nothing)
    nothing_collection.append(new_nothing)

print nothing_collection[-1]

Which brings me to 65667, and gives me the address to the next challenge, found at peak.html.

FIVE:
This one gives us a clue basically saying the hint sounds like “peak hell”. This is where I was lucky to have Jesse along for the ride. Since I’m not familiar with Python libraries, I had no idea there was one called pickle, but Jesse did and that helped out a ton. Along with that hint was this:

<peakhell src="banner.p"/>

So, we figured we should probably download banner.p and use pickle on it. First I read this to figure out exactly how to do that: http://docs.python.org/library/pickle.html

test = pickle.load(urllib.urlopen('http://www.pythonchallenge.com/pc/def/banner.p'))

test is now a list of lists of tuple pairs. Each pair consists of either ‘#’ or ‘ ‘ and an integer. It was early in the day, we were smart, and figured out fairly quickly that this would make a picture for us, so we did this:

def print_line(pair_list):
    print ''.join(pair[0] * pair[1] for pair in pair_list)

for pair_list in test:
    print_line(pair_list)

It’s actually pretty cool. Here’s a screenshot (and the solution to number five):

SIX:

This one was difficult for me mostly because it involved so many steps and took some lucky guesses. To be totally honest, it took us about an hour, although to also be honest, there were a lot of distractions. There was a picture of a zipper and the clue was just “zip”. After staring at the screen for awhile, I figured out that I just had to change the URL’s extension to “zip” in order to download a zip file to work with. Not before reading about Python’s zip library though! http://docs.python.org/library/zipfile.html

I started by grabbing that zip file (which I had to download first since I couldn’t pass an URL to zipfile.ZipFile()):

myzip = zipfile.ZipFile('/home/jen/Downloads/channel.zip', 'r')

After doing that, we decided to take a quick peak at the file names and contents:

[(f.filename, myzip.open(f.filename).read()) for f in myzip.filelist]

The filenames are all numbers followed by “.txt” and the contents bring back our friend from challenge #4: “Next nothing is xxxxx” There is a readme file with our next 2 hints, which are: “hint1: start from 90052 hint2: answer is inside the zip”

We put the filenames and contents in a giant dictionary, which I named bigdict and then giggled for about 45 minutes.

bigdict = dict((f.filename, (myzip.open(f.filename).read(), f.comment)) for f in myzip.filelist)

NOTE: this is our final code. When we ran this first, I didn’t include the comments and the file it ended at said “collect the comments”, so it had to be redone.

This was all right since I had already written some code to handle the similar situation in #4 and was able to start by altering that to fit this situation:

bigstring = []

def get_num(start_num):
    key = "%s.txt" % start_num
    next_num = re.search(r'Next nothing is (\d+)', bigdict[key][0]).group(1)
    bigstring.append(bigdict[key][1])
    return next_num

I used a different way to interpolate my string here, just for practice. Anyone with opinions on which is best is welcome to add them to my comments, I’m curious what people think is the best.

Using the same functionality as in #4, I start with 90052 and cycle through get_num():

output = []
output.append('90052')
for a in xrange(911):
   new_value = get_num(output[a])
    output.append(new_value)

I used 911 since that’s how big the dictionary was. Next was the fun part, printing off what I came up with:

print(''.join(bigstring))

Here’s the screenshot:

Almost there! Going to http://www.pythonchallenge.com/pc/def/hockey.html gave me this message:

“it's in the air. look at the letters.”

Using the letters used to make the word ‘HOCKEY’ I went to http://www.pythonchallenge.com/pc/def/oxygen.html and completed challenge #6.

We stared at #7 for awhile, played around with the image in it using the Python Imaging Library (http://www.pythonware.com/library/pil/handbook/introduction.htm), didn’t get anywhere and decided to call it a night.

That concludes this blog post. Hope it was fun for someone to read! I would love to hear about other peoples experiences with this and other online programming challenges!

Cheers,
Jen

Click here to view and leave comments!

Start of Pantry Application (db design with MySQL Workbench)

Posted on January 1st, 2011

One common problem I have is determining what I have in my pantry. For example, I’ll go grocery shopping after work and not remember whether I have any apple cider vinegar and if so, if I have enough for the recipe I want to make. Often I’ll buy a bottle and then get home to discover that I already have plenty. Or, I’ll assume I have something I don’t and get home to find out that I am now missing a key ingredient. Clearly, I need a software solution to my problem and just as clearly, I want to create it myself regardless of it already existing.

There are several decisions I need to make, platform being probably the hardest. For now, I decided to start out by designing the data model so I don’t have to worry about the hard stuff. Obviously, I need to keep track of the food items, categorize them in some way and track the amount on hand.

Then I decided that I wanted my ERD to be pretty for this blog post, and I decided to try doing this with MySQL Workbench (http://www.mysql.com/products/workbench/). Good news is it comes for linux! Bad news is that I was kind of a flake and tried installing the wrong version the first time, but 10 minutes later when I figured that out, installation was very easy (Windows easy, even!).

To test how easy the software is to use, I decided not to read the manual before starting. I opened it up, clicked on File -> New Model. From there I built some tables. One thing I realized right away was that my computer is not strong enough at all to make this a pleasure to do. Also, for some reason it goes way off my screen (and doesn’t maximize correctly).

A few hours later (hours because my computer does not handle this software very well *) I broke down and had to read the manual. Starting my diagram using the tool was pretty easy, you drag and drop the tables. However, the EED tool didn’t work the way I thought it would :) Oh well! Had the tool that I needed in order to create fk relationships using existing columns not been off the screen then I might not have had to resort to checking out the manual. Also, it shows the table relationships, but the lines don’t match up with the actual columns which can be confusing if you are just looking at it. Within the tool, you can hover over the relationship and it will show what fields are being used. Another thing to note is that since it’s not easy to alter the look of the relationships (and define them further), I couldn’t define the cardinality/modality in the manner that I am used to with crow’s foot notation (http://www2.cs.uregina.ca/~bernatja/crowsfoot.html). Of course, maybe it is possible and my lack of excitement about spending any more time on this just got in the way :) Anyone who knows more and would like to share their expertise in my comments, please feel welcome.

So, my first go at using the tool to define my data structure and making it a picture is here:

I think it’s all pretty self-explanatory. When food is purchased, the purchase is put into the purchases table and the amount in the food table increases. I’m not keeping track of each use, that’ll just be taken care of programmatically.

My next step is to determine the technology that I’ll use. I’m leaning towards django but am open to suggestions. At some point I’ll want to make a phone application (android) for it as well, since that’s where its best use will be.

That’s it for now!
Jen

* I’m sure that the software would be wonderful and fast to use on anything besides my netbook.

Click here to view and leave comments!