Introduction

In this assignment you will write a class ColumnDict that takes a table in the form of a list of lists (e.g. from a CSV file) and provides a key-value data structure (dictionary) where the values are the columns of the table.

For full marks in this assignment your solutions should not use any loops, but only comprehensions (and other builtin functions as needed). Your solution should also work with arbitrarily large input.

• Make sure you commit and push all your work using coursegit before 16:30 on Thursday November 27.

Dictionary functionality

Provide a constructor, and appropriate “dunder” methods so that the following tests pass. You should only store the data once.

def test_one_row():
    table= ColumnDict([["alice", "bob"],
                       [27, 32]])
    assert table["bob"] == [32]
    assert table["alice"] == [27]


def test_two_rows():
    table2= ColumnDict( [["alice", "bob"],
                         [1, 2],
                         [False, True]])
    assert table2["bob"] == [2,True]
    assert table2["alice"] == [1,False]

def test_three_cols():
    table3= ColumnDict( [["alice", "mallory", "bob"],
                         [1, 3, 2],
                         [False, None, True]])
    assert table3["bob"] == [2,True]
    assert table3["alice"] == [1,False]
    assert table3["mallory"] == [3,None]

def test_update():
      table3= ColumnDict( [["alice", "mallory", "bob"],
                           [1, 3, 2],
                           [False, None, True]])
      table3["bob"]=[42,False]
      assert table3["bob"] == [42,False]
      assert table3["alice"] == [1,False]
      assert table3["mallory"] == [3,None]

Selecting subsets of columns

Write a select method for you ColumnDict class that produces table (list of lists) table with subset of the columns. Your select method should handle any number of key arguments, including zero.

def test_select1():
   table3= ColumnDict( [["alice", "mallory", "bob"],
                        [1, 3, 2],
                        [False, None, True]])

   out=table3.select("alice", "mallory")
   assert out == [["alice", "mallory"],
                  [1, 3],
                  [False, None]]

def test_select2():
   table3= ColumnDict( [["alice", "mallory", "bob"],
                        [1, 3, 2],
                        [False, None, True]])

   assert table3.select() == []

Iterator

Provide appropriate dunder methods to support iteration. Do not copy or traverse the data more times than necessary. Do not use generators (i.e. yield). Your code should pass (at least) the following tests.

def test_iter1():
    rows= [["alice", "mallory", "bob"],
                         [1, 3, 2],
                         [False, None, True]]
    expected = [["alice", "bob", "mallory"],
                         [1, 2, 3],
                         [False, True, None]]
    table3= ColumnDict( rows )
    assert list(table3) == expected

def test_iter2():
    rows= [["alice", "mallory", "bob"],
                         [1, 3, 2],
                         [False, None, True]]
    expected = [["alice", "bob", "mallory"],
                         [1, 2, 3],
                         [False, True, None]]
    table3= ColumnDict( rows )
    out = []
    for row in table3:
        out.append(row)

    assert out == expected

def test_iter3():
    rows= [["alice", "mallory", "bob"],
           [1, 3, 2],
           [False, None, True]]
    expected = [["alice", "bob", "mallory"],
                [1, 2, 3],
                [False, True, None]]
    table3= ColumnDict( rows )
    assert [ row for row in table3 ] == expected
    assert list(table3) == expected

Introduction

Debian is currently collecting buildinfo but they are not very conveniently searchable. Eventually Chris Lamb's buildinfo.debian.net may solve this problem, but in the mean time, I decided to see how practical indexing the full set of buildinfo files is with sqlite.

Hack

1. First you need a copy of the buildinfo files. This is currently about 2.6G, and unfortunately you need to be a debian developer to fetch it.

    $ rsync -avz mirror.ftp-master.debian.org:/srv/ftp-master.debian.org/buildinfo .
   

2. Indexing takes about 15 minutes on my 5 year old machine (with an SSD). If you index all dependencies, you get a database of about 4G, probably because of my natural genius for database design. Restricting to debhelper and dh-elpa, it's about 17M.

    $ python3 index.py
   

   You need at least python3-debian installed

3. Now you can do queries like

    $ sqlite3 depends.sqlite "select * from depends where depend='dh-elpa' and depend_version<='0106'"
   

   where 0106 is some adhoc normalization of 1.6

Conclusions

The version number hackery is pretty fragile, but good enough for my current purposes. A more serious limitation is that I don't currently have a nice (and you see how generous my definition of nice is) way of limiting to builds currently available e.g. in Debian unstable.

I could not find any nice examples of using the vobject class to filter an icalendar file. Here is what I got to work. I'm sure there is a nicer way. This strips all of the valarm subevents (reminders) from an icalendar file.

import vobject
import sys

cal=vobject.readOne(sys.stdin)

for ev in cal.vevent_list:
    if ev.contents.has_key(u'valarm'):
       del ev.contents[u'valarm']

print cal.serialize()