cpython icon indicating copy to clipboard operation
cpython copied to clipboard

Timestamp conversion on windows fails with timestamps close to EPOCH

Open c3939d93-56b7-43ae-a865-17e381711bc8 opened this issue 6 years ago • 10 comments

BPO 37527
Nosy @pfmoore, @abalkin, @tjguk, @jleclanche, @zware, @zooba, @animalize, @ammaraskar, @pganssle, @Dschoni
PRs
  • python/cpython#15498
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2019-07-09.10:32:25.056>
    labels = ['3.8', 'type-bug', '3.7', '3.9', 'OS-windows']
    title = 'Timestamp conversion on windows fails with timestamps close to EPOCH'
    updated_at = <Date 2020-01-10.12:29:05.237>
    user = 'https://github.com/Dschoni'
    

    bugs.python.org fields:

    activity = <Date 2020-01-10.12:29:05.237>
    actor = 'pingchaoc'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Windows']
    creation = <Date 2019-07-09.10:32:25.056>
    creator = 'Dschoni'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 37527
    keywords = ['patch']
    message_count = 5.0
    messages = ['347541', '347596', '355819', '355824', '355832']
    nosy_count = 10.0
    nosy_names = ['paul.moore', 'belopolsky', 'tim.golden', 'jleclanche', 'zach.ware', 'steve.dower', 'malin', 'ammar2', 'p-ganssle', 'Dschoni']
    pr_nums = ['15498']
    priority = 'normal'
    resolution = None
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue37527'
    versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']
    

    A long description of the issue can be found on SO here: https://stackoverflow.com/questions/56931738/python-crash-on-windows-with-a-datetime-close-to-the-epoch?noredirect=1#comment100413591_56931738

    TL;DR:

    This fails on windows:

    from datetime import datetime
    datetime.fromtimestamp(1).timestamp()
    

    Looks like a similar problem to bpo-29097.

    >>> from datetime import datetime
    >>> d = datetime.fromtimestamp(1)
    >>> d.timestamp()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    OSError: [Errno 22] Invalid argument
    

    Neijwiert tried to analyzed it in stackoverflow: https://stackoverflow.com/questions/56931738/

    This indeed seems to be a duplicate of 29097, which is fixed in Python 3.7, so we can close this bug. Thank you for your report Dschoni, and thank you for finding the duplicate Ma Lin!

    pganssle avatar Nov 01 '19 14:11 pganssle

    bpo-29097 fixed bug in datetime.fromtimestamp(). But this issue is about datetime.timestamp(), not fixed yet.

    Ah, my mistake. The examples all use datetime.fromtimestamp, so I didn't notice that it was failing only on the timestamp side. Re-opening, thanks!

    pganssle avatar Nov 01 '19 18:11 pganssle

    Just narrowed down the problem little bit:

    the error happens in the method timestamp()

    • only on Windows

    • On the native datetime object (without timezone info)

    • On the dates a couple of days after 1 January 1970 and all the dates before that

    import datetime
    
    dt = datetime.datetime(1970,1,3)
    
    print(dt) # 1970-01-03 00:00:00
    
    print(dt.timestamp()) #169200.0
    
    
    
    dt = datetime.datetime(1970,1,1)
    
    print(dt)  # 1970-01-01 00:00:00
    
    print(dt.timestamp()) # OSError: [Errno 22] Invalid argument
    
    
    tz =datetime.timezone.utc
    
    dt = datetime.datetime(1970,1,1, tzinfo=tz) 
    
    print(dt) # 1970-01-01 00:00:00+00:00
    
    print(dt.timestamp()) # 0.0
    
    
    tz =datetime.timezone.utc
    
    dt = datetime.datetime(1960,1,1, tzinfo=tz) #1960-01-01 00:00:00+00:00
    
    print(dt)
    
    print(dt.timestamp()) # -315619200.0
    
    

    This was tested on Windows 10 Python 3.10.5

    On Ubuntu all of the tests just run

    Ev2geny avatar Oct 18 '22 19:10 Ev2geny

    I think I found what the issues is. The problem seems to be in the c code function _PyTime_localtime(time_t t, struct tm *tm) in the pytime.c.

    For Windows it uses localtime_s function, which appears not to accept negative time (time before 1 January 1970), however for none-windows it uses localtime_r, which does accept negative values.

    To reproduce on Windows

    // https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/localtime-s-localtime32-s-localtime64-s?view=msvc-170
    
    #include <stdio.h>
    #include <string.h>
    #include <time.h>
    
    int main(void)
    {
        struct tm newtime;
        long long int long_time;
        char timebuf[26];
        errno_t err;
    
        long_time = -1; // If this is 0 or above, everything just works fine
    
        // Convert to local time.
        err = localtime_s(&newtime, &long_time);
        if (err)
        {
            printf("Invalid argument");
            exit(1);
        }
    
        err = asctime_s(timebuf, 26, &newtime);
        if (err)
        {
            printf("Invalid argument to asctime_s.");
            exit(1);
        }
        printf("%s\n", timebuf);
    }
    

    However on Ubuntu

    //https://en.cppreference.com/w/c/chrono/localtime
    
    #include <time.h>
    #include <stdio.h>
    #include <stdlib.h>   
     
    int main(void)
    {
        time_t t = -1000;
    
        struct tm buf;
        char str[26];
    
        // https://manpages.ubuntu.com/manpages/bionic/man3/ctime.3.html
        asctime_r(localtime_r(&t, &buf), str);
        printf("Date / time:     %s", str);   // Date / time:     Thu Jan  1 00:43:20 1970
    }
    

    Ev2geny avatar Oct 19 '22 21:10 Ev2geny

    @zooba , are you the right person to look at this issue, as it seems to be related to the Microsoft way of implementation of the localtime_s function ?

    Ev2geny avatar Oct 19 '22 21:10 Ev2geny

    No, this needs someone who's more of an expert in time functions than me. Unless you want to wait for me to get up to speed (no promises, my learning list is pretty long).

    zooba avatar Oct 20 '22 13:10 zooba

    The core problem of this issue likely also impact https://github.com/python/cpython/issues/94414

    At least both the timestamp() and astimezone() functions are impacted on Windows 10 and Windows 11

    achoum avatar Mar 21 '24 08:03 achoum

    Duplicates: https://github.com/python/cpython/issues/94414, https://github.com/python/cpython/issues/94757

    StanFromIreland avatar Mar 09 '25 09:03 StanFromIreland

    I encountered this issue as well. datetime.timetuple() doesn't seem to be affected, so using it in combination with time.mktime() is a workaround:

    >>> datetime.fromtimestamp(1).timestamp()
    Traceback (most recent call last):
      File "<python-input-12>", line 1, in <module>
        datetime.fromtimestamp(1).timestamp()
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
    OSError: [Errno 22] Invalid argument
    >>> time.mktime(datetime.fromtimestamp(1).timetuple())
    1.0
    

    The above drops microseconds, but taking them into account isn't that hard:

    >>> dt = datetime.fromtimestamp(0.123)
    >>> time.mktime(dt.timetuple())
    0.0
    >>> time.mktime(dt.timetuple()) + dt.microsecond / 1e6
    0.123
    

    pekkaklarck avatar Apr 30 '25 13:04 pekkaklarck

    This fails on windows:

    from datetime import datetime
    datetime.fromtimestamp(1).timestamp()
    

    Issue fixed by https://github.com/python/cpython/pull/143463.

    Python 3.15.0a5+ (heads/main:f5685a266b2, Jan 15 2026, 10:57:22) [MSC v.1944 64 bit (AMD64)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> from datetime import datetime
    >>> datetime.fromtimestamp(1).timestamp()
    1.0
    

    vstinner avatar Jan 15 '26 10:01 vstinner