grattan icon indicating copy to clipboard operation
grattan copied to clipboard

What to do in case of update

Open HughParsonage opened this issue 6 years ago • 2 comments

HughParsonage avatar Mar 08 '19 03:03 HughParsonage

All internal data is defined and saved via data-raw/put-data.R.

The script is very complicated. While some simplification is possible, care should be taken not to change too much because it's important to have a fairly clean record of each change. Experience has shown that if the changes are too large when viewed in git it's quite difficult to detect breaking changes.

The most difficult update will be that associated with a new ATO sample file. The internal data supporting project includes tables with inflators for each variable. Some of these variables have a considerable chunk of code in data-raw/put-data.R just to create their inflators.

One annoyance is that the ABS changes historical data (even as early 1980s), so nearly every time you run the script there will be spurious changes.


Before each update to sysdata, review open pull requests, take note of https://cran.r-project.org/web/checks/check_results_grattan.html (i.e. fix any problems there).

ABS data From time to time or whenever there are updates to the relevant series:

  • Source data-raw/put-data.R
  • Fix the documentation according to the unit tests (e.g. if the lf_trend is updated, the lf_inflator_fy documentation needs to be updated to indicate the date it was updated.

The data updated includes

  • Data relevant to tax modelling:
    • cpi_...
    • wages_trend
    • lf_trend
  • as well as
    • abs_key_aggregates (for GDP, GNI)
    • aus_pop_by_yearqtr etc
    • residential_property_prices

DSS data

  • On windows with wget64 capabilities, run https://github.com/HughParsonage/grattan/blob/master/data-raw/put-guides-dss-gov-au.R

    Other platforms may be able to use just wget.

  • Source data-raw/put-data.R.

ATO sample files

  • Add to the top of data-raw/put-data.R calls to read in the new sample file.
  • Change the assignment sample_files_all <- accordingly
  • Copy and modify generic_inflators_1516 to define a new generic_inflators_

Check the diffs in data-raw/sysdata/ to verify the data has been updated.

On changes to the tax scales (typically Budget Night):

  • Change the files that are read to produce the following tables:
    • lito_tbl,
    • tax_tbl,
    • medicare_tbl,
    • sapto_tbl,
    • hecs_tbl

HughParsonage avatar Mar 08 '19 04:03 HughParsonage

Note: after updating medicare-tables.xlsx and running put-data.R, you will also need to update the hard-coded values for the thresholds in src/MedicareLevySingle.cpp. If this is not done, a test in tests/testthat/test_income_tax_cpp.R will fail.

MattCowgill avatar Apr 04 '19 03:04 MattCowgill