In Octave we can multiply every element of a matrix by a scalar using the
.* operator
A=[1,2,3;4,5,6];
B=A.*2
In general .* supports any two arguments of the same size.
C=A .* [2,2,2;2,2,2]
It turns out these are actually the same operation, since Octave converts the first into the second via
broadcasting
Quoting from the Octave docs, for element-wise binary operators and functions
The rule is that corresponding array dimensions must either be equal, or one of them must be 1.
In the case where one if the dimensions is 1, the smaller matrix is
tiled to match the dimensions of the larger matrix.
Here's another example you can try.
x = [1 2 3;4 5 6;7 8 9];
y = [10 20 30];
x + y
Reshaping arrays
One potentially surprising aspect of Octave arrays is that the number
of dimensions is independent from the number of elements. We can add
as many dimensions as we like, as long as the only possible index in
those dimensions is 1. This can be particularly useful when trying
to broadcast with higher dimensional arrays.
Complete the following function. You may want to copy the definitions of A and B into the REPL
to understand the use of cat.
## usage: scale_layers(array, weights)#### multiply each layer of a 3D array by the corresponding weightfunction out =scale_layers(array, weights)
out =endfunction%!test
%! onez =ones(3,3);%! A=cat(3,onez,2*onez,3*onez);%! B=cat(3,onez,6*onez,15*onez);%!assert(scale_layers(A,[1;3;5]),B)
Scaling a colour channel
Save the image above left as
~/cs2613/labs/L21/paris.jpg (make sure you get the
full resolution image, and not the thumbnail).
Run the following demo code; you can change the weight vector for different colourization.
This assignment is based on the material covered in Lab 15
and Lab 16.
The goal of the assignment is to develop a simple query language that
lets the user select rows and columns from a
CSV File, in
effect treating it like database.
Make sure you commit and push all your work using coursegit before
16:30 on Thursday March 20.
General Instructions
Every non-test function should have a docstring
Feel free to add docstrings for tests if you think they need
explanation
Use list and dictionary comprehensions as much as reasonable.
Your code should pass all of the given tests, plus some of your own
with different data. If you want, you can use some of the sample
data from the US Government College Scorecard. I’ve selected some of
the data into smaller files:
defread_csv(filename):'''Read a CSV file, return list of rows'''import csv
withopen(filename,'rt',newline='')as f:
reader = csv.reader(f, skipinitialspace=True)return[ row for row in reader ]
Save the following as “~/fcshome/assignments/A4/test1.csv”; we will use
it several tests. You should also construct your own example CSV files
and corresponding tests.
name, age, eye colour
Bob, 5, blue
Mary, 27, brown
Vij, 54, green
Here is a test to give you the idea of the returned data structure from
read_csv.
The first row most in most CSV files consists of column labels. We will
use this to help the user access columns by name rather than by counting
columns.
Write a function header_map that builds a dictionary from labels to
column numbers.
Sometimes it’s more convenient to work with rows of the table as
dictionaries, rather than passing around the map of column labels
everwhere. Write a function row2dict that takes the output from
headermap, and a row, and returns a dictionary representing that row
(column order is lost here, but that will be ok in our application).
We are going to write a simple query languge where each query is a
3-tuple (left, op, right), and op is one of =, <, and >. In
the initial version, left and right are numbers or strings. Strings
are interpreted as follows: if they are column labels, retrieve the
value in that column; otherwise treat it as a literal string. With this
in mind, write a function check_row that takes a row in dictionary
form, and checks if it matches a query tuple.
Extend check_row so that it supports operations AND and OR. For
these cases both left and right operands must be queries. Hint: this
should only be a few more lines of code.
defcountdown(n):print('Counting down from', n)while n >0:yield n
n -=1
Referring to Section
6.1
an equivalent iterator class. Here we will implement the entire protocol with one class, but multiple classes like we did for JavaScript is also reasonable.
Modify only the __next__ method of the following skeleton class so that given
tests pass.
defcountdown(n):yield'starting'while n >0:yield n
n -=1deftest_generator():
counter =countdown(3)assert[ t for t in counter ] == \
['starting',3,2,1]class Counter:"Iterator class simulating countdown"def__init__(self,n):
self.n = n
self.first =Truedef__iter__(self):return self
def__next__(self):# insert code here
self.n -=1return self.n+1deftest_class():
counter =Counter(3)assert[ t for t in counter ] == \
['starting',3,2,1]
What is __iter__ for?
Time
25 minutes
Activity
individual
Update the __iter__ method of your previous solution so that the following additional test passes
deftest_twice():
counter =Counter(3)assert[ t for t in counter ] == \
['starting',3,2,1]assert[ t for t in counter ] == \
['starting',3,2,1]
Question for your journal: what is an important functionality difference
between generators and more general iterators?
Download Work.zip and unzip in your newly created directory.
Higher order functions
Time
20 minutes
Activity
Assemble given pieces
Complete Exercise
3.5. Note
that you are given an initial version of fileparse.py containing a
function parse_csv in Exercise 3.4 just above. Make sure that your
modified function passes the following test.
Question for your journal: what is the type of the argument types?
Refactoring a function
Time
30 minutes
Activity
Refactor function to add feature
Complete Exercise
3.6. You only need to write a few lines of code, but you
will need to make several blocks of code conditional on
has_headers. Be careful when re-indenting code, remember Python is
picky about indentation.
Make sure the following test (and your previous tests) pass
Exercise 2.2. Instead of using the REPL for this one, write functions parse_row, compute_cost, and update_dict so that the following tests pass.
deftest_cost():
d =parse_row(row)
cost =compute_cost(d)assert cost == pytest.approx(3220.000,abs=0.00000001)deftest_d():
d =parse_row(row)update_dict(d)assert d == {'name':'AA','shares':75,'price':32.2,'date': (6,11,2007),'account':12345}
Containers: read_portfolio
Time
25 minutes
Activity
Write function based on template
Exercise 2.4. Embed the following tests in report.py, and make sure they pass.
deftest_read():
portfolio =read_portfolio('Data/portfolio.csv')assert portfolio == \
[('AA',100,32.2), ('IBM',50,91.1),('CAT',150,83.44), ('MSFT',200,51.23),('GE',95,40.37), ('MSFT',50,65.1), ('IBM',100,70.44)]deftest_total():
portfolio =read_portfolio('Data/portfolio.csv')
total =0.0for name, shares, price in portfolio:
total += shares*price
assert total == pytest.approx(44671.15,abs=0.001)
Exercise 2.5. Call this version report2.py
Embed the following tests in report2.py, and make sure they pass.
Exercise 2.20. Start with the following skeleton, and convert the given examples into pytest tests.
import pytest
from report2 import read_portfolio
prices = {'GOOG':490.1,'AA':23.45,'CAT':35.46,'IBM':91.1,'MSFT':34.23,'GE':13.48,}
Note that the value test will need to be adjusted for a value of
approximately 31167.10, since our price list is different from the one
used in the book.
Modify one of the tests so it fails. Run pytest again.
Question for your journal: what are three ways failing tests are
indicated?
Lists and Pytest
Time
20 minutes
Activity
Individual programming from template
For each of the remaining interactive evaluations in Exercise
1.19,
add a corresponding test to listex.py.
make sure each test_* function (def) has a unique name.
if you want, you can have multiple asserts in a single test.
Questions for your journal:
what are some differences between == in JavaScript and Python?
based on what you have seen so far, what are similarities and
differences between the node test framework and pytest
Functions and coverage
Time
20 minutes
Activity
Individual programming, modify previous solution
We have already been using python functions for pytest, without
really thinking about how they work. In Part 1.7 of Practical
Python,
functions are explained.
Debian is currently collecting
buildinfo
but they are not very conveniently searchable. Eventually
Chris Lamb's buildinfo.debian.net may
solve this problem, but in the mean time, I decided to see how
practical indexing the full set of buildinfo files is with sqlite.
Hack
First you need a copy of the buildinfo files. This is currently about 2.6G, and unfortunately you need
to be a debian developer to fetch it.
Indexing
takes about 15 minutes on my 5 year old machine (with an SSD). If you index all dependencies,
you get a database of about 4G, probably because of my natural genius for database design.
Restricting to debhelper and dh-elpa, it's about 17M.
$ python3 index.py
You need at least python3-debian installed
Now you can do queries like
$ sqlite3 depends.sqlite "select * from depends where depend='dh-elpa' and depend_version<='0106'"
where 0106 is some adhoc normalization of 1.6
Conclusions
The version number hackery is pretty fragile, but good enough for my
current purposes. A more serious limitation is that I don't currently
have a nice (and you see how generous my definition of nice is) way of
limiting to builds currently available e.g. in Debian unstable.
I could not find any nice examples of using the vobject class to
filter an icalendar file. Here is what I got to work. I'm sure there
is a nicer way. This strips all of the valarm subevents (reminders) from
an icalendar file.
import vobject
import sys
cal=vobject.readOne(sys.stdin)
for ev in cal.vevent_list:
if ev.contents.has_key(u'valarm'):
del ev.contents[u'valarm']
print cal.serialize()