pimutils
Performance issues with calendar from Office365
I use khal for multiple vdir calendars: work, private, academic, etc. I experience serious performance issues with khal interactive, where jumping to the next week usually takes up to 2 seconds.
I pinpointed the issue (as already expected) to the work Office365 calendar I sync via vdirsyncer and davmail. I don't know if it is the sheer size of some events or the format of the .ics files that produces this issue. If I disable this calendar, my other calendars work just fine.
I used the discover type for many folders, but even when I disable all other calendars and explicitly define my work calendar with the respective vdir, nothing changes. So this is not the cause.
Now this issue alone would be bearable if khal cached the interactive view. What makes it so painful is that this happens on every restart of khal interactive. Is caching just broken for me or is this default behaviour?
Anyway, I don't know how to get to the root cause of the performance issue. Any help on how to further debug would be appreciated.
Some ideas to pinpoint the cause:
- Can you have a look if those calendar files really are very large (and, if so, send an example, redacted of course)?
- How many files are in that vdir folder?
- In case you run vdirsyncer quite often, try running khal before you run ikhal to queck if it is a caching issue
Thank you for your suggestions. I did not find much time for github in the past months.
In case you run vdirsyncer quite often, try running khal before you run ikhal to queck if it is a caching issue
It would be good to know first what to expect from the cache. Is khal even supposed to cache the previous selected dates after I quit it? It was rather a question because I have no idea.
How many files are in that vdir folder?
The vdir of my office365 calendar contains 57 .ics files.
Can you have a look if those calendar files really are very large...
Following table lists line, word and character count of these files:
lines words chars
5097 6928 222219
1731 2599 78149
833 1041 38927
732 929 34394
706 926 28429
647 650 18654
647 650 18582
622 625 17931
339 543 16282
313 657 14325
369 491 14190
249 514 13152
328 436 12535
302 406 12068
316 423 11907
245 288 10636
365 374 10608
317 326 9200
220 291 8470
190 268 8066
127 318 6201
117 252 5681
116 151 5341
172 179 4926
110 182 4761
165 168 4737
102 167 4275
98 176 4197
94 140 4077
142 145 4075
91 139 3755
93 124 3723
99 105 2723
98 101 2717
98 101 2698
94 97 2634
75 78 2024
74 77 2024
74 77 2019
74 77 2013
73 76 1986
69 72 1921
60 69 1855
51 64 1361
51 56 1359
51 54 1343
50 59 1313
50 53 1313
50 53 1289
49 52 1283
49 58 1279
49 58 1263
45 48 1229
44 48 1194
0 1 8
So there are 2 files which are pretty big. I moved them away and it improves performance greatly. So the large files are probably the issue. Thanks for asking these questions, because somehow it did not occur to me to check the line and word counts :)
What is the expected line count of ics files for khal? How large are the .ics files approximately that are used to test khal?
There is still some lag when going down multiple weeks, even if all files have line_count < 800. More specifically, if I go down more than 7 weeks quickly, I don't know where the cursor is anymore and I have to wait ~2s. So if I'm in January and want to go to August, it's not the best experience.
If there was a) some cache that is saved even after the program quits or b) any way to jump to a specific month then this would not be an issue for me. I don't know if a) or b) is possible though. I apologize if I have overlooked it in the docs.
...(and, if so, send an example, redacted of course)?
I will post the content of the large files as soon as I find time to redact them. Probably tomorrow.
Ok, I checked the file and it is a recurring event that has been recurring since 2016. It's a monthly event at work. I'm not an expert with .ics. Do you know if it is normal for recurring events to grow this large?
Edit: I did not find a quick way to redact the large file. I cut out one of the many VEVENTs and redacted it. The other VEVENTs are basically just repeating anyway. Here is the excerpt:
BEGIN:VCALENDAR
PRODID:Microsoft Exchange Server 2010
VERSION:2.0
BEGIN:VTIMEZONE
TZID:Europe/Berlin
BEGIN:STANDARD
DTSTART:16010101T030000
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=-1SU;BYMONTH=10
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:16010101T020000
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=-1SU;BYMONTH=3
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
ORGANIZER;CN="":mailto:
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN="":mailto:
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN="":mailto:
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN="":mailto:
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN="":mailto:
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN="":mailto:
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN="":mailto:
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN="":mailto:
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN="":mailto:
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN="":mailto:
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN="":mailto:
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN="":mailto:
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN="":mailto:
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN="":mailto:
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN="":mailto:
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN="":mailto:
ATTENDEE;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN="":mailto:
ATTENDEE;ROLE=OPT-PARTICIPANT;PARTSTAT=NEEDS-ACTION;RSVP=TRUE;CN="":mailto:
DESCRIPTION:\nXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\n\nXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\n\nXXXXXXXXXXXXXXXXXXXXX\nXXXX
XXXXXXXXXXXXX\n\n\n
RRULE:FREQ=MONTHLY;INTERVAL=1;BYMONTHDAY=13
UID:040000008200E00074C5B7101A82E00800000000901E313D9C91D10100000000000000001
00000003545932E423DBB4680103138780BCA60
SUMMARY:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXX
DTSTART;TZID="Europe/Berlin":20160413T090000
DTEND;TZID="Europe/Berlin":20160413T093000
CLASS:PUBLIC
PRIORITY:5
DTSTAMP:20181213T115325Z
TRANSP:OPAQUE
STATUS:TENTATIVE
SEQUENCE:28
X-MICROSOFT-CDO-APPT-SEQUENCE:28
X-MICROSOFT-CDO-OWNERAPPTID:2055751648
X-MICROSOFT-CDO-BUSYSTATUS:TENTATIVE
X-MICROSOFT-CDO-INTENDEDSTATUS:BUSY
X-MICROSOFT-CDO-ALLDAYEVENT:FALSE
X-MICROSOFT-CDO-IMPORTANCE:1
X-MICROSOFT-CDO-INSTTYPE:1
X-MICROSOFT-DONOTFORWARDMEETING:FALSE
X-MICROSOFT-DISALLOW-COUNTER:FALSE
X-MICROSOFT-LOCATIONS:[]
ATTENDEE;CN="";PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT:mailto:
ATTENDEE;CN="";PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT:mailto:
ATTENDEE;CN="";PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT:mailto:
ATTENDEE;CN="";PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT:mailto:
ATTENDEE;CN="";PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT:mailto:
ATTENDEE;CN="";PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT:mailto:
ATTENDEE;CN="";PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT:mailto:
ATTENDEE;CN="";PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT:mailto:
ATTENDEE;CN="";PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT:mailto:
ATTENDEE;CN="";PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT:mailto:
ATTENDEE;CN="";PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT:mailto:
ATTENDEE;CN="";PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT:mailto:
ATTENDEE;CN="";PARTSTAT=ACCEPTED;ROLE=REQ-PARTICIPANT:mailto:
ATTENDEE;CN="";PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT:mailto:
ATTENDEE;CN="";PARTSTAT=NEEDS-ACTION;ROLE=REQ-PARTICIPANT:mailto:
ATTENDEE;CN="";PARTSTAT=ACCEPTED;ROLE=REQ-PARTICIPANT:mailto:
ATTENDEE;CN="";PARTSTAT=NEEDS-ACTION;ROLE=OPT-PARTICIPANT:mailto:
LAST-MODIFIED:20200901T133813Z
X-MOZ-LASTACK:20191122T121856Z
BEGIN:VALARM
DESCRIPTION:REMINDER
TRIGGER;RELATED=START:-PT15M
ACTION:DISPLAY
END:VALARM
END:VEVENT