simonmichael
Replace YYYY/MM/DD with YYYY-MM-DD in all documentation and in all printing commands
This has already been mentioned in https://github.com/simonmichael/hledger/issues/210, but there has been no further progress since then, so I wanted to pick it up again.
Is there still any discussion needed that this is the right way to go? Otherwise I'd skip that part and gather a TODO list 😅:
Change output of
- [x]
hledger print - [x]
hledger rewrite - [x]
hledger add - [x]
hledger balancesheet - [x]
hledger close - [x]
hledger register - [x]
hledger print-unique - [x]
hledger register-match
Did I miss something?
hledger stats already uses the ISO format.
If somebody is using rewrite in a automated fashion or similar, this would be a severe breaking change.
So maybe introduce the feature via a config flag first and issue a warning for all commands, that the default will change in the future and that in order to keep the current formatting one needs to add the config flag (e.g. hledger --output-date-format yyyy/mm/dd rewrite)?
I'd be happy to create a PR as soon as we're in agreement how this should be handled.
Looking forward to your feedback!
I too prefer yyyy-mm-dd format and would like to see this. I'd even be happy with a non-breaking change for this, such as a configuration option (does hledger support or plan to support rc files?).
I've added a comment to #210. With regard to output from all hledger tools, and all docs, I agree that we should switch to ISO YYYY-MM-DD as the default/preferred date format. There's still the question of whether to always show leading zeros, eg 0001-01-01 not 1-1-1, I'm not sure if the latter would ever be preferable; probably not. I feel adding options or a config file are a last resort, to be avoided unless really needed.
Trying to focus this issue more on the output side, with #210 covering input. "Documentation" of course involves both input and output examples.
What does "using rewrite in a automated fashion" mean - parsing hledger's output ? That's not a supported API (unlike hledger-api) and I'm not expecting this issue to cause too much compatibility pain.
From #210:
What I miss, is --output-date-format for hledger output (e.g. the register command's output). I think that would be useful to produce printed reports that are not awkward for "normal" clerks in Germany.
I suppose you're right. If ISO dates in output are really a problem for people ? Ledger calls it --date-format.
I'm strongly for always showing leading zeros in ISO dates. Because that is what makes ISO dates and also they are not sortable otherwise.
I think I'll get used to the dash version.
But aren't there downsides of changing the hledger journal format and going different ways than the ledger file format? Compatibility? I'm new to hledger and don't know legder. This are just my considerations.
I think an --output-date-format option would be nice for reports and also a way to keep backwards compatibility for scripts that parse hledger output.
I'd definitely always show leading zeros. This ensures that also people unfamiliar with this format can easily pick up which number is the year. Also, removing the leading zeros is not ISO8601 compliant.
automated fashion
E.g. having git hooks to always use rewrite before commiting, or some kind of CI which checks for changes after using rewrite. But that's probably not a common use case 😅, so yeah, probably not that important.
But aren't there downsides of changing the hledger journal format and going different ways than the ledger file format?
That's a topic for #210, but that's not really a problem as the hledger journal format (and I assume Ledger's before it) already allow ISO dates.
@simonmichael
I suppose you're right. If ISO dates in output are really a problem for people ?
Most people I know in Germany would never write down an ISO date and almost never read ISO dates (except in some IT professions). When I show hledger reports to people - clerks or customers - I'm pretty sure they are unsettled by the ISO dates. Especially customers are already demanded by the many numbers ;)
I think it best to have input data in the YYYY-MM-DD (ISO8601) format, and that be the default in documentation and print commands.
However, perhaps, there ought to be a way to create reports in other custom date formats (like DD/MM/YYYY or DD.MM.YYYY, which seem to be the most popular formats in countries like India and Germany)? Of course, one potential argument against this would be that people can use sed or other tools to edit csv outputs themselves, and that hledger need not handle this. However, if this doesn't create much complexity, having a --output-date=DD.MM.YYYY flag of some sort might be useful.
I'm a bit confused about the scope of this issue. Imo, the internal date format should be the international standardized ISO 8601 (referenced here as "ISO format"). This means, the journal file should use this notation, as well as all in- and output. Is this how it's meant to be ?
Yes, we've agreed on #210, and here,
- ~to keep supporting only ISO 8601 dates (YYYY-MM-DD with leading zeros) plus our slight punctuation variants (YYYY/MM/DD, YYYY.MM.DD, and optional leading zeros) in input (all file formats and UIs) [clarification: UI also accepts smart dates, we're talking about simple dates here]~
- ~to use ISO 8601 dates as the default for all output~
- to use ISO 8601 dates as the default in docs
- (lower priority feature) to allow customisation of output date format with
--output-date-formator (shorter, Ledger-compatible)--date-format
1 was covered by #210 and is the status quo; this issue, #933, will cover at least 2 and 3.
@simonmichael I've been thinking about this a little more and I still wonder if it's a good idea that Hledeger supports YYYY/MM/DD and YYYY.MM.DD. Normally when somebody uses a slash it's understood as MM/DD/YYYY and when somebody uses a dot it's understood as DD.MM.YYYY. I guess this can't be changed due to backwards compatibility. However, maybe we should print a warning whenever somebody uses a slash or a dot. Something like
Attention:
Dates with `/` are understood as YYYY/MM/DD.
To avoid this warning please use YYYY-MM-DD instead.
I hear you but I'm not convinced. D/M/Y is the norm in Ireland, eg. I suspect https://en.wikipedia.org/wiki/Date_format_by_country will confirm that the punctuation character can't be relied on as a guide.
Yeah, true. But that is basically confiriming that Hledger should print a warning whenever the format is not YYYY-MM-DD, right?
I don't think it's causing enough trouble to be worth the annoyance of a warning, is it ?
Mh. Ok I guess there is little ambiguity with YYYY/MM/DD and YYYY.MM.DD.
But at least if it has the format XX/YY/ZZ or XX.YY.ZZ a warning should be printed.
11/12/13
john 10 €
anna
This could mean anything otherwise and is really bad practice.
Yes, I can see that requiring a four digit year could make sense. On the other hand it's an easy mistake to notice and fix, and if some users are really attached to a two-digit year format I'm not sure we should deny them. Like Ledger, we also allow yearless dates (11/12) which again can be misinterpreted. Re warnings generally, we have so far resisted adding any; we have errors that stop the program, but not warnings, which keeps our output very predictable and avoids the need for more infrastructure to control and customise the warnings. I'm not sure if we should add warnings to hledger, or if we should keep them confined in some kind of lint command.
Haha, that's not the kind of reasoning I'd expect from a Haskell developer. Sounds like: "Null / undefined / NaN is easy to fix. If some developers are attached to it, they should be able to use it" 😛
I think it should be warned about …
- To protect users from themselves. It's much easier to make a mistake e.g. copy and pasting and forgetting to switch from m/d/y to y/m/d.
- To promote consistency among journal files
- To educate about correct usage of ISO date formats
we also allow yearless dates (11/12)
But how can yearless dates be used in meaningful way? Sounds like it's only useful for short example journals.
have so far resisted adding any warnings
They are just printed to stderr and normally are a non-braking change. So the impact should be really low, right?
some kind of lint command
That would be a good start already!
The year for yearless dates can be set by a preceding directive in the ledger file.
I think a short warning message wouldn't be so bad. I think like ad-si that the risk and impact of this kind of error is high. But please no warning for dates with 4-digit-years.
Concerning warnings about yearless dates, I'm unsure.
Keep in mind that a warning have to be at the end of the normal output to be visible. And the warning would not be visible in a pager like less.
Right now I'll just say that we chose to be highly compatible with Ledger from the outset, which dictated many things. If you were designing a format from scratch intended for maximum error prevention and "educating" users, some things would be different. And there's nothing stopping that! But the journal format should remain close to Ledger's format I think.
... that's why I created Transity. ISO 8601 in UTC only and all error detection features are activated per default 😉. So yeah, I understand if you don't want to go all the way, just sharing my 2 cents here.
Also as I've said I don't see that either the risk or impact of this kind of error is high in practice. If anyone has experience to the contrary that would be interesting to hear more about.
To give a little more data, here's my experience: in 11 years of hledgering I have
- never written a date with parts in the wrong order
- never had to parse a journal written by someone else with date parts in the wrong order
- frequently had to parse CSV with weird date formats, requiring me to set the right
date-formatonce, when setting up the CSV rules file - once or twice a month, forgot to change the date when copy-pasting an old transaction, which causes disagreement at next reconcile, at which time I notice and fix the problem. (check-dates, a lint-y addon, could help prevent this.)
- once in a while written a malformed date due to a typo, which hledger complains about if it's obviously bad, otherwise I notice and fix at reconcile time
Actually check-dates is a built in command, I forgot (it checks that transactions are in date order). We have a small convention that lint-y command names begin with "check".
@ad-si are you still interested in working on this ? If you want help let me know.
Oh sorry, I was still waiting for some kind of consensus, but I guess that was reached with https://github.com/simonmichael/hledger/issues/933#issuecomment-451855349. I'll open a first pull request the coming days!
I know @simonmichael is leaning against configuration and options but I'd add that - warnings are generally OK for risky situations, like yearless dates and xx/yy/zz date format. But only if a date format is not specified earlier (in option or configuration ;). This is normally one of the first things I configure (--date-format %Y-%m-%d), working daily with different date/time formats, timezones, languages, and currencies.
@MisterY I hear you. Warnings (and config files) aren't out of the question, but they require strong justification. If you run into concrete examples where hledger's current behaviour caused you a real problem, do share them (but as a new issue please - let's refocus this one).
Status: https://github.com/simonmichael/hledger/issues/933#issuecomment-451855349
No problem, but if you'd ever like to learn, come on by #hledger to find help.
Update: #1157 has implemented YYYY-MM-DD output, more testers welcome.
Thanks to @bwignall, ISO date output has landed in master. \o/
Next steps in the plan:
- update all remaining documentation and examples on hledger.org to use ISO dates (maybe ? as needed ?)
- add an --output-date-format or --date-format option to allow customisation of output
Is there currently in master no way to use the old default date format in print output? I'm a strong supporter of the new default and will be migrating to it myself as soon as I can, but even I'm hesitant to build the master now and switch if there isn't a way to –at least temporarily– keep outputting the old format. I have a lot of other scripts which muck with my records that will need updating to match.
Currently: no, there really isn't. Guess we need someone to work on that --output-date-format..
For what it's worth, to those considering implementing --output-date-format, here are the three substantive changes that went into porting to yyyy-mm-dd:
- https://github.com/simonmichael/hledger/commit/35481a665b1c81fec6661bbaf9d0607c813bb702
- https://github.com/simonmichael/hledger/commit/ab12ff916fab9d91ccc3ea3eb93f5d18c0f11046
- https://github.com/simonmichael/hledger/commit/24026d629a73b512274ae634c83662992d06862c
Sorry, I have a question on the status.
In current hledger (version 1.16) at least print and add use the new ISO format. Because hledger uses show for Days now. But is it possible that version 1.16.2 on hackage still generate the old date format with showTransaction?
The program buchhaltung still generates the old format despite I updated its dependenciest in compiled/installed it...
Edit: @simonmichael thanks, my fault. I have the release installed and the git version. The git version was first on the path...
Perhaps the title of this issue should be changed to "add customization of date output format" to more accurately reflect what's remaining to be done?
I suggest we limit this issue's scope again: it was about switching to ISO format, and that's done except for consideration and possible action on
- update all remaining documentation and examples on hledger.org to use ISO dates (maybe ? as needed ?)
For other things above (--output-date-format, possible warnings/more strict behaviour when parsing ambiguous dates, ...) let's open new PRs when someone is ready to work on them.