donkirkby

Tuesday, November 13, 2012

Live Coding in Python v2

I've built a tool that lets you run your Python code as you type it. For example, this code draws a 100x100 pixel square.

When I change the forward distance to 50, the square immediately changes. I don't even have to save the file.

In this tutorial, I'll demonstrate two things. Live turtle graphics that make a fun learning tool, and a live coding display that can be used with regular code to show you what's happening inside it. To try it yourself, visit donkirkby.github.com. To see it in action, watch my demo video, or read on.

Python already comes with a turtle module, so what's the difference? To use the regular turtle, I need to add a little start up code, and then I need to save and run. Every time I make a change to the code, I need to save and run to see the result. Of course, I don't do that every time. Instead, I predict the result by running through the code in my head. One of this project's main goals for live coding is to let programmers' brains focus on writing code instead of running code. If you can see the code's results laid out in front of you, you don't have to hold it all in your head.

Still, I sometimes like to run the regular turtle graphics code to see the animation of how the turtle moves along its path. I can make my code run both ways by putting it into a function, and then calling it different ways depending on whether I'm running as the main script or in live coding mode.

def draw(t):
    for i in range(4):
        t.forward(50)
        t.right(90)

if __name__ == '__main__':
    from Tkinter import mainloop
    from turtle import Turtle
    t = Turtle()
    draw(t)
    mainloop()
elif __name__ == '__live_coding__':
    global __live_turtle__
    
    t = __live_turtle__
    draw(t)

Another benefit to live coding like this is that I can be creative in a different way, by reacting to the results of my changes. How about an example? When I added the feature for filling polygons, I played with triangles, squares, and pentagons. Then I tried a star, and the middle wasn't filled. After the surprise wore off, I realized that the centre is actually "outside" the polygon when you draw a star this way. That gave me the idea to see how it would deal with a spiral, so I made the turtle go around the star five times, and made the sides longer and longer. That was cool, a striped star! Then I made it go around 50 times, and it filled the screen.

At this point, I wondered what would happen if I changed the angle, and the results blew my mind!

I didn't set out to draw a pinwheel pattern and work out how to achieve that, I just stumbled across it while exploring how filled polygons work. When you combine live coding's rapid response with an intuitive interface like turtle graphics, it's easier to learn and create with. I think that was Bret Victor's point in his Inventing on Principle video that inspired me to build this tool.

That was the fun learning tool, now what can you do with real code? I did create a turtle class that writes to PDF, so that will let you use turtle graphics in a few more situations, but the main feature is a different view that helps you visualize what's happening inside your code so you don't have to keep running it in your head. I'll start with a trivial chunk of code where I assign a variable, and then modify it.

s = 'Hello'
s += ', World!'

That's easy to step through in your head and see that s is now 'Hello, World!' Remember, though, that I want to let your brain focus on writing code instead of stepping through it.

I open the live coding display on the right, and it shows me what's in the variable after each change.

s = 'Hello'                             
s += ', World!'
s = 'Hello'
s = 'Hello, World!'

Let's do something more interesting and write a library function that does binary search for a value in a sorted array. The live coding will show us what's happening in our code so we don't have to hold it all in our heads.

def search(n, a):                       
    return -1
                                        

It's a bad search function that never finds anything, but let's see how it works when we call it.

def search(n, a):                       
    return -1

i = search(2, [1, 2, 4])
n = 2 a = [1, 2, 4]                     
return -1

i = -1 

You can see the input parameters at the start of the function, and the return value at the end.

We'll start looking for the value in the array, and the first place to look is the middle item.

def search(n, a):                       
    low = 0
    high = len(a) - 1
    mid = low + high / 2
    if n == a[mid]:
        return mid
    return -1

i = search(2, [1, 2, 4])
n = 2 a = [1, 2, 4]                     
low = 0
high = 2
mid = 1

return 1


i = 1

That was lucky! It was in the first place we looked, and you can see the calculations as it goes. You see an abstract formula in the code, like high = len(a) - 1, and you see the concrete result in the live coding display, like high = 2. However, a search function usually won't find the item we're searching for on the first try. Let's ask for an item earlier in the list and use a while loop to find it.

def search(n, a):                       
    low = 0
    high = len(a) - 1
    while True:
        mid = low + high / 2
        v = a[mid]:
        if n == v:
            return mid
        if n < v:
            high = mid - 1
    return -1

i = search(1, [1, 2, 4])
n = 1 a = [1, 2, 4]                     
low = 0
high = 2
         |
mid = 1  | mid = 0
v = 2    | v = 1
         |
         | return 0
         |
high = 0 |


i = 0

The loop runs twice, and each run adds a column to the display showing the calculations. That's a good example of how this tool differs from a debugger. With a debugger, you're always looking at a single moment in time. Here, you can see the whole history of the search laid out on the screen, and you move back and forth through time just by moving your eye. It's a lot like the difference that makes static visualizations of sorting algorithms easier to follow than animated sorting algorithms.

Now let's look for an item later in the list.

def search(n, a):                       
    low = 0
    high = len(a) - 1
    while True:
        mid = low + high / 2
        v = a[mid]
        if n == v:
            return mid
        if n < v:
            high = mid - 1
        else:
            low = mid + 1
    return -1

i = search(4, [1, 2, 4])
n = 4 a = [1, 2, 4]                     
low = 0
high = 2
        |
mid = 1 | mid = 3
v = 2   | IndexError: list index out of
        |
        | 
        |
        |
        |
low = 2 |


IndexError: list index out of range

Oops, I get an IndexError. Without the live coding display, I would just get a traceback that shows where the error happened, but not how it happened. Now, I can walk back from the error to see where things went wrong. mid is the index value, and it's calculated at the top of the loop. The two values that go into it are both 2, so they should average to 2. Oh, I need parentheses to calculate the average.

def search(n, a):                       
    low = 0
    high = len(a) - 1
    while True:
        mid = (low + high) / 2
        v = a[mid]
        if n == v:
            return mid
        if n < v:
            high = mid - 1
        else:
            low = mid + 1
    return -1

i = search(4, [1, 2, 4])
n = 4 a = [1, 2, 4]                     
low = 0
high = 2
        |
mid = 1 | mid = 2
v = 2   | v = 4
        |
        | return 2
        |
        |
        |
low = 2 |


i = 2

What happens if we try to find a value that's not in the list?

def search(n, a):                       
    low = 0
    high = len(a) - 1
    while True:
        mid = (low + high) / 2
        v = a[mid]
        if n == v:
            return mid
        if n < v:
            high = mid - 1
        else:
            low = mid + 1
    return -1

i = search(3, [1, 2, 4])
n = 3 a = [1, 2, 4]                     
low = 0
high = 2
        |          |         |         |
mid = 1 | mid = 2  | mid = 1 | mid = 1 |
v = 2   | v = 4    | v = 4   | v = 4   |
        |          |         |         |
        |          |         |         |
        |          |         |         |
        | high = 1 |         |         |
        |          |         |         |
low = 2 |          | low = 2 | low = 2 |


RuntimeError: live coding message limit

I guess that while True wasn't such a good idea, we're stuck in an infinite loop. If you want to see some of the later loop runs, you can scroll over to the right.

From the third run on, the values in the loop don't change, so we probably want to exit from the second or third run. If you look at the end of the second run, you can see that high is lower than low. That means that we've searched all the way from both ends to meet in the middle, and it's time to give up.

def search(n, a):                       
    low = 0
    high = len(a) - 1
    while low <= high:
        mid = (low + high) / 2
        v = a[mid]
        if n == v:
            return mid
        if n < v:
            high = mid - 1
        else:
            low = mid + 1
    return -1

i = search(3, [1, 2, 4])
n = 3 a = [1, 2, 4]                     
low = 0
high = 2
        |
mid = 1 | mid = 2
v = 2   | v = 4
        |
        |
        |
        | high = 1
        |
low = 2 |
return -1

i = -1

At this point, I think I'm done. I can add a few entries and search for them to make sure everything is working. Also, if this were a real library module, I wouldn't want to execute a call at the end of the file, so I only do it when I'm in live coding mode.

if __name__ == '__live_coding__':
    i = search(3, [1, 2, 4])

Remember, you can try this tool yourself by visiting donkirkby.github.com. Help me test it and report your bugs. I'd also love to hear about any other projects working on the same kind of tools.

Friday, August 17, 2012

Unit Tests

This is a discussion of the benefits and philosophy of test-driven development that I wrote many years ago. I don't think there's anything ground-breaking here, but it's a nice compilation of what I've read and learned over the years.

Whaddya mean, test?

A whale is not a fish, it's an insect. -- Peter Cook
A unit test is not a test, it's a specification. -- Agile development mindset

One of these is a joke, and one is a useful way to think about the world. When you write one unit test, you actually get two things.

  1. A safety net - If you or another developer make some changes that break the code, that unit test will start failing. If you run the test right after you make the change, then it's obvious that your change is the one that broke the code.
  2. A working example - If you write your test clearly, then other developers can look at it to see how your class should be used. Not only that, but error condition tests explicitly show how not to use it.

An example

There's a nice, simple example of a unit test in this blog post by Jeff Langr. It tests an implementation of the exponent function. Here are some excerpts:

First, a simple case of squaring the numbers from 1 to 10:

   public void testSquares() {
      for (int i = 1; i < 10; i++)
         assertEquals(i + " squared:", i * i, MathUtil.power(i, 2));
   }

Then, some more exotic cases:

   public void testOneRaisedToAnythingIsAlwaysOne() {
      assertEquals(1, MathUtil.power(1, 1));
      assertEquals(1, MathUtil.power(1, 2));
      assertEquals(1, MathUtil.power(1, LARGE_NUMBER));
   }

And, finally, some error conditions:

   public void testNegativeExponentsUnsupported() {
      try {
         MathUtil.power(1, -1);
         fail("should not be supported");
      }
      catch (UnsupportedOperationException expected) {
      
      }
   }

These three cases are typical of the kinds of things to put in your tests. You've specified how to use the class, as well as how not to use the class and shown what will happen when the class is abused.

Whaddya mean, test driven?

OK, I wrote my code and it works; now I have to write a unit test? That can be pretty painful. It's even worse if you get asked to try and write unit tests for existing code that someone else wrote. Why is it painful? Because there are two types of code: easy-to-test and hard-to-test code. If you write the tests after the code is finished, you're gambling. You might have written easy-to-test code, but maybe not. Once you find out, it's depressing to have to go back and change code that works, just so you can write the stupid test.

Why gamble? Write easy-to-test code the first time. The simplest way to be sure the code is easy to test is to write the test at the same time as the code. That's what the "driven" means in test-driven development. You write a little bit of test, then enough code to make the test compile and pass, but no more. Then you write a little bit more test, and just enough code to make it compile and pass. When you have enough tests to specify all the requirements for your class, your class is finished.

What makes code hard to test?

Hard-to-test code tends to have larger classes and methods that do several things at once. It can also have fuzzy relationships between those classes. Objects create instances of each other willy-nilly and messages fly back and forth. Testing a big class is hard because you probably need to set up a lot of input data, and there are probably a lot of different scenarios to test. If you have several big classes that you have to test as a group, the number of scenarios will often multiply together.

Easy-to-test code tends to have small classes that do one thing. It will often have more classes collaborating to accomplish the task, but the relationships and responsibilities of those classes are clear. Techniques like dependency injection mean that objects don't have to worry about creating the other objects they work with and that allows you test one class at a time instead of testing the whole group at once.

There are no guarantees, but easy-to-test code has a lot in common with easy-to-maintain code.

Writing clearly

If unit tests are going to be your class's specification document, they had better be easy to read. There are four things that every test should have to make it readable: a name that describes it, data that describes the starting conditions, the execution of the method being tested, and verification that the method did the right thing.

Diminishing returns

When are you done? You're done when you're comfortable that the code is nestled safely in its testing blanket. The hardest thing to learn is when to stop writing a test. Start small and learn from your experience. It's better to have a readable, stable test that only covers half an object's code than to have a big mess that covers all of the code but breaks all the time for bogus reasons.

An advantage of writing the test first is that the test and the code finish around the same time. If you write a failing test before you add each feature, you should have a pretty good test when you finish the last feature, and you won't have any existential angst over whether you've tested enough.

Refactoring

As you build the test and the class together, the design may evolve. As you add features, a class may need to be split into smaller classes. You can use refactoring techniques to do this safely. You may also need to refactor the test to make it more readable. Refactoring is a big topic, and the best place to start is Martin Fowler's book or his web site. Refactoring is a lot safer when you have the tests to catch mistakes.

Further Reading

Read this series of blog posts on writing unit tests. Find further discussion of unit tests as specs or specification by example. Some people talk about getting test infected when your perspective shifts and you can't imagine coding without tests.

All of this is covered in detail in Gerard Meszaros's excellent book, xUnit Test Patterns. Unsurprisingly, it's part of Martin Fowler's book series.

Labels:

Wednesday, June 27, 2012

Live Coding in Python

I saw Bret Victor's Inventing on Principle video and thought, "Wow, I want those tools!" I couldn't find them, so I built my own version for Python. To try it yourself, visit donkirkby.github.com. To see how my version works, watch my demo video, or read on. I'll start with a trivial chunk of code where I assign a variable, and then modify it.

s = 'Hello'
s += ', World!'

That's easy to step through in your head and see that s is now 'Hello, World!' But why do I have to do it in my head when the computer could do it for me?

I add a special comment to show the live coding display on the left, and add another one to set it to a fixed width.

                                        

s = 'Hello'
s = 'Hello, World!'
# echo on                               
# echo width 40
s = 'Hello'
s += ', World!'

Let's do something more interesting and write a library function that does binary search for a value in a sorted array. The live coding will show us what's happening in our code so we don't have to hold it all in our heads.

                                        
# echo on                               
# echo width 40
def search(n, a):
    return -1

It's a bad search function that never finds anything, but let's see how it works when we call it.

                                        

n = 2 a = [1, 2, 4]
return -1

i = -1 
# echo on                               
# echo width 40
def search(n, a):
    return -1

i = search(2, [1, 2, 4])

You can see the input parameters at the start of the function, and the return value at the end.

We'll start looking for the value in the array, and the first place to look is the middle item.

                                        

n = 2 a = [1, 2, 4]
low = 0
high = 2
mid = 1

return 1


i = 1
# echo on                               
# echo width 40
def search(n, a):
    low = 0
    high = len(a) - 1
    mid = low + high / 2
    if n == a[mid]:
        return mid
    return -1

i = search(2, [1, 2, 4])

That was lucky! It was in the first place we looked, and you can see the calculations as it goes. You see an abstract formula in the code, like high = len(a) - 1, and you see the concrete result in the live coding display, like high = 2. However, a search function usually won't find the item we're searching for on the first try. Let's ask for an item earlier in the list and use a while loop to find it.

                                        

n = 1 a = [1, 2, 4]
low = 0
high = 2
         |
mid = 1  | mid = 0
v = 2    | v = 1
         |
         | return 0
         |
high = 0 |


i = 0
# echo on                               
# echo width 40
def search(n, a):
    low = 0
    high = len(a) - 1
    while True:
        mid = low + high / 2
        v = a[mid]:
        if n == v:
            return mid
        if n < v:
            high = mid - 1
    return -1

i = search(1, [1, 2, 4])

The loop runs twice, and each run adds a column to the display showing the calculations.

Now let's look for an item later in the list.

                                        

n = 4 a = [1, 2, 4]
low = 0
high = 2
        |
mid = 1 | mid = 3
v = 2   | IndexError: list index out of
        |
        | 
        |
        |
        |
low = 2 |


IndexError: list index out of range
# echo on                               
# echo width 40
def search(n, a):
    low = 0
    high = len(a) - 1
    while True:
        mid = low + high / 2
        v = a[mid]
        if n == v:
            return mid
        if n < v:
            high = mid - 1
        else:
            low = mid + 1
    return -1

i = search(4, [1, 2, 4])

Oops, I get an IndexError. Without the live coding display, I would just get a traceback that shows where the error happened, but not how it happened. Now, I can walk back from the error to see where things went wrong. mid is the index value, and it's calculated at the top of the loop. The two values that go into it are both 2, so they should average to 2. Oh, I need parentheses to calculate the average.

                                        

n = 4 a = [1, 2, 4]
low = 0
high = 2
        |
mid = 1 | mid = 2
v = 2   | v = 4
        |
        | return 2
        |
        |
        |
low = 2 |


i = 2
# echo on                               
# echo width 40
def search(n, a):
    low = 0
    high = len(a) - 1
    while True:
        mid = (low + high) / 2
        v = a[mid]
        if n == v:
            return mid
        if n < v:
            high = mid - 1
        else:
            low = mid + 1
    return -1

i = search(4, [1, 2, 4])

What happens if we try to find a value that's not in the list?

                                        

n = 3 a = [1, 2, 4]
low = 0
high = 2
        |          |         |         |
mid = 1 | mid = 2  | mid = 1 | mid = 1 |
v = 2   | v = 4    | v = 4   | v = 4   |
        |          |         |         |
        |          |         |         |
        |          |         |         |
        | high = 1 |         |         |
        |          |         |         |
low = 2 |          | low = 2 | low = 2 |


RuntimeError: live coding message limit
# echo on                               
# echo width 40
def search(n, a):
    low = 0
    high = len(a) - 1
    while True:
        mid = (low + high) / 2
        v = a[mid]
        if n == v:
            return mid
        if n < v:
            high = mid - 1
        else:
            low = mid + 1
    return -1

i = search(3, [1, 2, 4])

I guess that while True wasn't such a good idea, we're stuck in an infinite loop. If you want to see some of the later loop runs, you can add another special comment.

# echo scroll 30

From the third run on, the values in the loop don't change, so we probably want to exit from the second or third run. If you look at the end of the second run, you can see that high is lower than low. That means that we've searched all the way from both ends to meet in the middle, and it's time to give up.

                                        

n = 3 a = [1, 2, 4]
low = 0
high = 2
        |
mid = 1 | mid = 2
v = 2   | v = 4
        |
        |
        |
        | high = 1
        |
low = 2 |
return -1

i = -1
# echo on                               
# echo width 40
def search(n, a):
    low = 0
    high = len(a) - 1
    while low <= high:
        mid = (low + high) / 2
        v = a[mid]
        if n == v:
            return mid
        if n < v:
            high = mid - 1
        else:
            low = mid + 1
    return -1

i = search(3, [1, 2, 4])

At this point, I think I'm done. I can add a few entries and search for them to make sure everything is working. Also, if this were a real library module, I wouldn't want to execute a call at the end of the file, so I use a slight tweak of the standard library module pattern to only make the call when I've got the live coding display on.

if __name__ == '__live_coding__':
    i = search(3, [1, 2, 4])

Remember, you can try this tool yourself by visiting donkirkby.github.com. Help me test it and report your bugs. I'd also love to hear about any other projects working on the same kind of tools.

Labels:

Sunday, November 20, 2011

Akron board milling project

I finally finished the milling project that I started back in March. It's a board game called Akron that I milled into a sheet of acrylic. We had a milling course at work for people to learn how to use the computer-controlled milling machine, and this is what I chose to do as my project.
The game is one of my favourites, a connection game designed by Cameron Browne where you can stack marbles up and over your opponent's path. In the second photo, you can see that the white player has connected from left to right, although the path is a bit twisty. Black was close to cutting off white by crossing over the top, but couldn't get there in time. To learn more about the game, read the rules, or play it on Richard's play-by-e-mail server.
The project took a lot longer than I expected, mostly because I made it way more complicated than necessary. On future projects, I'll try to keep things simpler, but it's tempting for me to push into what is possible with the tools, instead of staying with what is practical. I found it an interesting transition moving from writing software to designing something physical. In software, I'm used to writing in small pieces and testing each piece along the way. With a mill, it takes a long time to get set up, so I probably did more dry runs than I needed to. If you're interested in the gcode that I wrote, you can find it in my Google Code project. Writing gcode was interesting, in some ways it's even more primitive than classic BASIC. I think on future projects, I'll write Python code to generate the gcode, and skip gcode's loops and conditional logic as much as possible.
The 1" diameter marbles came from two copies of Abalone, and one copy of Balanx.

Friday, August 05, 2011

Sly Corporation

Here's a board game design project that turned into an art project. In the distant past of 2003, I found a copy of Sly at the Salvation Army and picked it up because I'm a big fan of the designer. The box lid for SlyIt's a game system designed in the 1970's and distributed by Amway of all things. Funky colours like avocado and harvest gold, and the box is pure nostalgia.

The game system came with six games, but of course I went looking for new games that other people had designed or adapted to play on it. I never found any, but I did find it mentioned in an article by Ron Hale-Evans on game systems, along with Rock Paper Scissors Spock Lizard and the piecepack. I've since published a few piecepack games, but I never found any new games for Sly. Even my plaintive cries for help went unanswered.

Since the following eight years haven't produced any more games for Sly, I recently decided to do something myself. I thought I would try adapting Eric Solomon's Corporation. Corporation being played on the Sly game systemIt's a nice-looking game published by nestorgames. They have a great set of abstract games for sale, and they manufacture everything on demand.

The game adapted very easily to Sly, so I foolishly said to myself, "Wouldn't it be fun to publish the adapted rules and make them look just like the rules for the original six games in Sly?" Since I enjoyed designing the programmer cake in Inkscape, I decided to use it again for this project. Of course, it took me ten times longer to lay out the rules than it did to do the original adaptation. Learning how to use Inkscape feels a bit like learning pointers in C. Very useful once you understand, but you can't help feeling that your brain was slightly damaged by the experience. Rules for Corporation next to rules for an original game from SlyI'm pleased with the result, it looks pretty close to the original. If you have Sly, you can download the rules and add them to your copy. If you're really keen, you can design or adapt your own games, and use my files as a template to format the rules. Just don't make me wait another eight years, please.

Saturday, May 14, 2011

Birthday cakes for programmers

I recently designed this birthday cake for a friend who is also a programmer. Here's the code that appears on the cake:
    int i=4;
while (i--)
/* 0100010 */
printf("%s%s\n",
"Happy birthday"
,i-1? " to you":
" dear Aki" );
/*TODO:use a
fork()*/
As you can see in the picture, the zeros and ones were replaced by candles on the cake. (Can you figure out how old the birthday boy is?)

Tools & Techniques

I got the original inspiration for this cake when I read the "Surprise for a programmer on Birthday" question on Stack Overflow. Sadly, that question has been deleted because too many people hate fun. However, you can see a shadow of its former glory on the Stack Printer archive site.
I used Péter Török's idea of code that prints the words to the Happy Birthday song and wrote my own version of the code laid out in a circular shape. Yes, I tested my code. I also included Josh's idea of binary birthday candles, although they were frustrating because I didn't space them far enough apart. The lit candles would ignite their unlit neighbours.
Once I had the code written and laid out, I was trying to think how I could ice the cake neatly enough to keep it legible. Then I remembered that some shops sell cakes with your photo on them. Could I draw an image of the code in a paint program and e-mail it to them? A call to my local Dairy Queen made me happy. Turns out they have edible paper and edible ink, and they just print it in a regular ink jet printer, how cool is that?
So all I had to do was draw the code, right? Not so fast, buddy! I wanted to make it look like an old dot-matrix printout, so I searched for a free dot-matrix font. There are a few of them, but they're all proportional instead of monospaced. Well, after much fruitless searching and head scratching I discovered Inkscape's vertical text orientation. With that, I could use the dot-matrix font I found, and force it to be monospaced. I had to type the text in columns instead of the usual lines - weird, but achievable. After that, I just added the green bars and I was done.
If you want to adapt my design for a friend's cake, you can download my SVG file and the font, then edit it in Inkscape.

Prior Art

Lots of other people have done similar things, here's a survey of some that I found. Sew Tara might have the most popular choice with a classic XKCD strip.
Penelope/Tania asked the stack overflow question that was my direct inspiration, and this is her cake.
Chantastic and team made this internationalized XML cake, along with a Spring XML cake, a SQL cake, and a UML cake. The UML cake looks like they figured out the edible paper trick before I did.
I love the syntax colouring on this one. Not sure who the original baker was, but the earliest source I could find was onehumor.com.
The Good Eats community describe this as a mistake that happened when the bakery's mail system mangled some MS Outlook HTML, but I think it would be funny to do intentionally.
Janet McKnight made this QR code cake and successfully decoded it with a phone camera.
I have an irrational desire for this pi cake. fungus amungus even posted instructions to make your own.

Update: My birthday cake

Aki returned the favour by designing a gamer cake for me. The cake itself looked like an old VT100 terminal, but he rigged up an LCD projector to display the screen contents on the surface of the cake. Then he wrote an interactive fiction piece about me at the office on my birthday. I've never played a text-adventure game with a room full of people watching before; it's a little distracting. When the little game finished, it triggered an animation of Conway's Game of Life that segued into "Happy Birthday Don". All in all, very impressive, Aki.

Labels:

Monday, June 26, 2006

Television Listings in Ruby

For years, I've been poking at the idea of building a program that retrieves TV listings from the web and generates a personalized TV guide for me. For example, it would highlight shows that I like and just not display shows that I hate. When zap2it published a web service, I knew the time had come. I've played with it a bit using .NET and java clients, but now I'm using this project as an excuse to try out the ruby language.
I'd heard that Ruby is a neat little scripting language that makes programming fun. I decided to try it out on this project. It actually is pretty fun, but I'm not sure I'd want to lead a team of programmers on a Ruby project. You still need to be a bit of a hacker to figure out how to use it. Some of the API documentation is pretty slim, so you need to dig into the source code to see what you're supposed to do.
The first version of RubyTubyTeeVee is now working and up on source forge.

What's Next?

I have lots more features I'd like to add, but I think the first thing I'll do is look at moving it from a desktop application to a Ruby on Rails web application.

References

Here are the most useful references I found while working on this project.
  • Ruby home page - the centre of the Ruby universe.
  • Programming Ruby - A good introductory book that's available online.
  • Core API and Standard libraries - The most useful API documentation I've found.
  • Ruby installer for Windows - I'll be developing on the Windows platform, so this makes an easy starting point. If that link breaks, try the RubyInstaller home page.
  • FX Ruby - a GUI framework for Ruby. Seems to be included in the distribution. Check out the examples, particularly the raabrowser.
  • Printing - a sample that shows how to print from a Ruby app. Looks like the FXDCPrint and FXPrintDialog classes are a cleaner way to accomplish it, but they might not be finished yet. There's a code snippet that seems to do something similar by calling the Win32 API directly. I'm going to try Ruby RTF next. If that doesn't work, maybe these Ruby RTF Tools will.
  • REXML - a module for accessing XML from Ruby. Check out the tutorial or find more details in the library API. Because I'm processing large XML files, I decided to use an event model instead of parsing the whole thing into a Document Object Model (DOM). The StreamListener class shows the methods you need to implement for a listener. NOTE: the documentation for tag_start is incorrect: attrs is a Hash, not an array of arrays. There's an example of how to use your listener in the tutorial.
  • strftime specs - They're actually on the Time class instead of the Date class. Maybe it's one of those strange mixin things that I don't quite understand yet.
  • HTTP library documentation may help me make the request from the zap2it web service. This presentation might be helpful, or even this web service tutorial, although it might be outdated.
  • Zap2it web service specs. I think this requires a login.
  • Ruby web service client example.
  • Digest authentication is a big pain, but zap2it requires it. I found a ruby snippet that seems to generate the header for you, and a general description of digest authentication. Wow, was that a pain! The snippet didn't quite work for me, but I used the internet wayback machine to find the source code for the Java libraries that were handling the authentication in zap2it's java sample. I compared that to the Ruby snippet and found that the Ruby snippet was missing the algorithm and opaque fields, and it had hard coded the GET method so that POST requests would always fail. But now I have a Ruby class that sucessfully requests TV listings!