This feed contains pages with tag "python".
Overview
This assignment is based on the material covered in Lab 14 and Lab 15.
The goal of the assignment is to develop a simple query language that lets the user select rows and columns from a CSV File, in effect treating it like database.
- Make sure you commit and push all your work using coursegit before 16:30 on Thursday November 21.
General Instructions
Every non-test function should have a docstring
Feel free to add docstrings for tests if you think they need explanation
Use list and dictionary comprehensions as much as reasonable.
Your code should pass all of the given tests, plus some of your own with different data. If you want, you can use some of the sample data from the US Government College Scorecard. I’ve selected some of the data into smaller files:
- A marking rubric is available.
Reading CSV Files
We will use the builtin Python CSV module to read CSV files.
def read_csv(filename): '''Read a CSV file, return list of rows''' import csv with open(filename,'rt',newline='') as f: reader = csv.reader(f, skipinitialspace=True) return [ row for row in reader ]
Save the following as “~/fcshome/assignments/A4/test1.csv”; we will use it several tests. You should also construct your own example CSV files and corresponding tests.
name, age, eye colour Bob, 5, blue Mary, 27, brown Vij, 54, green
Here is a test to give you the idea of the returned data structure from
read_csv.def test_read_csv(): assert read_csv('test1.csv') == [['name', 'age', 'eye colour'], ['Bob', '5', 'blue'], ['Mary', '27', 'brown'], ['Vij', '54', 'green']]
Parsing Headers
The first row most in most CSV files consists of column labels. We will use this to help the user access columns by name rather than by counting columns.
Write a function
header_mapthat builds a dictionary from labels to column numbers.table = read_csv('test1.csv') def test_header_map_1(): hmap = header_map(table[0]) assert hmap == { 'name': 0, 'age': 1, 'eye colour': 2 }
Transforming rows into dictionaries
Sometimes it’s more convenient to work with rows of the table as dictionaries, rather than passing around the map of column labels everwhere. Write a function
row2dictthat takes the output from headermap, and a row, and returns a dictionary representing that row (column order is lost here, but that will be ok in our application).def test_row2dict(): hmap = header_map(table[0]) assert row2dict(hmap, table[1]) == {'name': 'Bob', 'age': '5', 'eye colour': 'blue'}
Matching rows
We are going to write a simple query languge where each query is a 3-tuple
(left, op, right), andopis one of=,<, and>. In the initial version,leftandrightare numbers or strings. Strings are interpreted as follows: if they are column labels, retrieve the value in that column; otherwise treat it as a literal string. With this in mind, write a functioncheck_rowthat takes a row in dictionary form, and checks if it matches a query tuple.def test_check_row(): row = {'name': 'Bob', 'age': '5', 'eye colour': 'blue'} assert check_row(row, ('age', '=', 5)) assert not check_row(row, ('eye colour', '=', 5)) assert check_row(row, ('eye colour', '=', 'blue')) assert check_row(row, ('age', '>', 4)) assert check_row(row, ('age', '<', 1000))
Extending the query language
Extend
check_rowso that it supports operationsANDandOR. For these cases both left and right operands must be queries. Hint: this should only be a few more lines of code.def test_check_row_logical(): row = {'name': 'Bob', 'age': '5', 'eye colour': 'blue'} assert check_row(row, (('age', '=', 5),'OR',('eye colour', '=', 5))) assert not check_row(row, (('age', '=', 5),'AND',('eye colour', '=', 5)))
Filtering tables
Use you previously developed functions to impliment a function
filter_tablethat selects certain rows of the table according to a query.def test_filter_table1(): assert filter_table(table,('age', '>', 0)) == [['name', 'age', 'eye colour'], ['Bob', '5', 'blue'], ['Mary', '27', 'brown'], ['Vij', '54', 'green']] assert filter_table(table,('age', '<', 28)) == [['name', 'age', 'eye colour'], ['Bob', '5', 'blue'], ['Mary', '27', 'brown']] assert filter_table(table,('eye colour', '=', 'brown')) == [['name', 'age', 'eye colour'], ['Mary', '27', 'brown']] assert filter_table(table,('name', '=', 'Vij')) == [['name', 'age', 'eye colour'], ['Vij', '54', 'green']] def test_filter_table2(): assert filter_table(table,(('age', '>', 0),'AND',('age','>','26'))) == [['name', 'age', 'eye colour'], ['Mary', '27', 'brown'], ['Vij', '54', 'green']] assert filter_table(table,(('age', '<', 28),'AND',('age','>','26'))) == [['name', 'age', 'eye colour'], ['Mary', '27', 'brown']] assert filter_table(table,(('eye colour', '=', 'brown'), 'OR', ('name','=','Vij'))) == [['name', 'age', 'eye colour'], ['Mary', '27', 'brown'], ['Vij', '54', 'green']]
Getting started
- Make a directory
~/cs2613/labs/L18 - Download Work.zip and unzip in your newly created directory.
- Download Part of the textbook solution to Exercise 3.18
Methods
- Time
- 25 minutes
- Activity
- Write accessor and mutator methods.
- Start from the following skeleton class
class Stock: def init(self, name, shares, price): self.name=name self.shares=shares self.price=price
- Complete Exercise
4.2;
in particular add methods
costandsellto yourStockclass so that the following tests pass.
def test_cost2(): s = Stock('GOOG', 100, 490.10) assert s.cost() == pytest.approx(49010.0,0.001) def test_sell(): s = Stock('GOOG', 100, 490.10) s.sell(25) assert s.shares == 75 assert s.cost() == pytest.approx(36757.5, 0.001)
Special Methods
- Time
- 25 minutes
- Activity
- Code to test, learn about "dunder" methods.
- Complete Exercise 4.9. In particular add the appropropriate method to the
Stockclass so that the following test passes.
def test_repr(): goog = Stock('GOOG', 100, 490.1) assert repr(goog) == "Stock('GOOG', 100, 490.1)"
Static methods
- Time
- 25 minutes
- Activity
- Transform code, use template for static methods
- Using the code of
Exercise 4.3 as a hint,
add the static method
read_portfolioto theStockclass.
@staticmethod def read_portfolio(filename): # code from 4.3 goes here
Your completed static method should pass
def test_read_portfolio(): portfolio = Stock.read_portfolio('Data/portfolio.csv') assert repr(portfolio[0:3]) == \ "[Stock('AA', 100, 32.2), Stock('IBM', 50, 91.1), Stock('CAT', 150, 83.44)]"
- Question for your journal: what feature of the
reprfunction does this test demonstrate?
Inheritance
- Time
- 25 minutes
- Activity
- Refactor given code, work with class hierarchy
Start with following class hierarchy based on Exercises 4.5 and 4.6
class TableFormatter: def headings(self, headers): ''' Emit the table headings. ''' raise NotImplementedError() def row(self, rowdata): ''' Emit a single row of table data. ''' raise NotImplementedError() class TextTableFormatter(TableFormatter): ''' Emit a table in plain-text format ''' def headings(self, headers): output = '' for h in headers: output += f'{h:>10s} ' output+='\n' output+=(('-'*10 + ' ')*len(headers)) output += '\n' return output def row(self, rowdata): output = '' for d in rowdata: output+=f'{d:>10s} ' output += '\n' return output def test_text_2(): portfolio=stock.Stock.read_portfolio('Data/portfolio.csv') formatter= TextTableFormatter() output= formatter.headings(['Name','Shares','Price', 'Cost']) for obj in portfolio[0:3]: output +=formatter.row([obj.name,f'{obj.shares}', f'{obj.price:0.2f}',f'{obj.cost():0.2f}']) assert '\n' + output == ''' Name Shares Price Cost ---------- ---------- ---------- ---------- AA 100 32.20 3220.00 IBM 50 91.10 4555.00 CAT 150 83.44 12516.00 '''
Define an appropriate method so that the
strfunction forStockinstances returns the same output as the in one (non-heading) row of the given test.Define a new class
StockTableFormatterthat uses thestrbuiltin function forStockobjects and passes the following test
def test_string_1(): portfolio=stock.Stock.read_portfolio('Data/portfolio.csv') formatter= StockTableFormatter() output= formatter.headings(['Name','Shares','Price', 'Cost']) for obj in portfolio[0:3]: output +=formatter.row(obj) assert '\n' + output == ''' Name Shares Price Cost ---------- ---------- ---------- ---------- AA 100 32.20 3220.00 IBM 50 91.10 4555.00 CAT 150 83.44 12516.00 '''
- Note that both the test
test_text_2andtest_string_1are sensitve to whitespace, so be careful when copying the strings.
Before next lab
- Complete Exercise
4.2;
in particular add methods