libc Fix newlib

So i found several incorrect definitions in the newlib module. ~~I cannot trace these values to anything except #646 which was done for devkitpro, yet definitions there don't seem to be different from fixed ones.~~ devkitPro/libctru has these wrong values.

PR checks

[ ] Edit corresponding file(s) under libc-test/semver when you add/remove item(s)
[x] Your PR doesn't contain any unstable values like *LAST or *MAX (see #3131)
[ ] If your PR increments version number, it must not contain any other changes
[x] rustc ci/style.rs && ./style src
[ ] cd libc-test && cargo test
- (this might fail on your env due to environment difference between your env and CI. Ignore failures if you are not sure)

I'm not sure how to do all of these.

Sep 08 '23 15:09 zetanumbers

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @JohnTitor (or someone else) soon.

Please see the contribution instructions for more information. Namely, in order to ensure the minimum review times lag, PR authors and assigned reviewers should ensure that the review label (S-waiting-on-review and S-waiting-on-author) stays updated, invoking these commands when appropriate:

@rustbot author: the review is finished, PR author should check the comments and take action accordingly
@rustbot review: the author is ready for a review, this PR will be queued again in the reviewer's queue

Sep 08 '23 15:09 rustbot

One other thing I think is worth mentioning: target_os = "horizon" also applies to the aarch64-nintendo-switch-freestanding target, but I'm not sure if these values need to be different on that platform as well (it doesn't look like they set env: "newlib" so maybe it is irrelevant). I don't think it would necessarily be a regression but tagging the maintainers of that platform @leo60228 @jam1garner just in case they have input for how this change might affect it.

Sep 23 '23 16:09 ian-h-chamberlain

Freestanding using newlib is a combination that doesn't currently have any meaning, there's no port of newlib targeting freestanding environments (and I'd go as far as to say there never will be) so this shouldn't affect the freestanding target.

Sep 23 '23 18:09 jam1garner

:umbrella: The latest upstream changes (presumably #3284) made this pull request unmergeable. Please resolve the merge conflicts.

Sep 24 '23 15:09 bors

@jam1garner What about this target that's on CI for this repo? https://github.com/rust-lang/libc/blob/main/ci/switch.json

Looks like it's for the switch with newlib, added by @leo60228 4 years ago

Sep 29 '23 22:09 pheki

yeah idk leo can probably shed more light on that but my best guess is that target json is aimed at homebrew environments not freestanding

Sep 29 '23 22:09 jam1garner

I don't really remember after four years, sorry. I don't think the configuration that .json is for is still supported, though.

Sep 29 '23 22:09 leo60228

Could anyone give me a summary of the discussion here? I'd like to help merge this but don't have enough context.

Jan 04 '24 09:01 JohnTitor

Could anyone give me a summary of the discussion here? I'd like to help merge this but don't have enough context.

A lot of wrong definitions, particularly for newlib targets. The main reason, I would say, is that it has definitions from libctru for nintendo 3DS, which are super different from the most og newlib. This is because newlib code was introduced precisely for nintendo 3DS.

I no longer have time to untangle this problem to check field offests, struct alignments and sizes, etc. I would be happy to pass this torch to anyone interested. I thought of making some instrument to generate test C code, but it bacame too unwieldy to me. I also had hard time configuring ESP32 toolchain, it being a bash shell you have to dive into before getting access to compiler, so I haven't managed to check it either.

Jan 04 '24 10:01 zetanumbers

Keep in mind horizon os of 3DS as to preserve these libctru definitions.

Jan 04 '24 10:01 zetanumbers

Indeed. Some wrong definitions had been introduced for newlib targets due to them being needed by horizon OS for the N3DS target (and not being present for any other newlib target). For what concerns HorizonOS, all the "current" definitions should be kept via cfg_if or by moving them into the horizon module, since they are valid in our context.

Any changes made should not disrupt our current implementation.

Jan 04 '24 10:01 Meziu

I wrote the script to check some definitions, here are results for ps vita

sizeof (struct pollfd_replica) == 12 != 8 == sizeof (struct pollfd)
sizeof (struct statvfs_replica) == 56 != 1 == sizeof (statvfs)
alignof (struct statvfs_replica) == 8 != 2 == alignof (statvfs)
sizeof (struct sigaction_replica) == 12 != 1 == sizeof (sigaction)
alignof (struct sigaction_replica) == 4 != 2 == alignof (sigaction)
sizeof (struct dirent_replica) == 344 != 352 == sizeof (struct dirent)
alignof (struct dirent_replica) == 1 != 8 == alignof (struct dirent)
sizeof (struct passwd_replica) == 24 != 28 == sizeof (struct passwd)
sizeof (struct sem_t_replica) == 16 != 4 == sizeof (sem_t)
alignof (struct sem_t_replica) == 1 != 4 == alignof (sem_t)
sizeof (struct utsname_replica) == 390 != 325 == sizeof (struct utsname)
sizeof (struct cpu_set_t_replica) == 128 != 4 == sizeof (cpu_set_t)
sizeof (struct pthread_cond_t_replica) == 8 != 4 == sizeof (pthread_cond_t)
alignof (struct pthread_cond_t_replica) == 8 != 4 == alignof (pthread_cond_t)
alignof (struct pthread_attr_t_replica) == 1 != 4 == alignof (pthread_attr_t)
alignof (struct pthread_rwlockattr_t_replica) == 1 != 4 == alignof (pthread_rwlockattr_t)
sizeof (vita-samples/hello_world/src/newlib_assert.h:409): 4 != 8
O_CLOEXEC == 262144 != 524288
PF_INET6 == 24 != 23
AI_NUMERICSERV == 8 != 0
AI_ADDRCONFIG == 1024 != 0
NI_NUMERICSERV == 8 != 0
NI_DGRAM == 16 != 0
EAI_AGAIN == -3 != 2
EAI_BADFLAGS == -1 != 3
EAI_FAIL == -4 != 4
EAI_FAMILY == -6 != 5
EAI_MEMORY == -10 != 6
EAI_NONAME == -2 != 8
EAI_SERVICE == -8 != 9
EAI_SOCKTYPE == -7 != 10
EAI_SYSTEM == -11 != 11
EAI_OVERFLOW == -12 != 14

Here's the script

#include <stdlib.h>
#include <pthread.h>
#include <errno.h>
#include <sys/socket.h>
#include <fcntl.h>
#include <time.h>
#include <netdb.h>
#include <netinet/tcp.h>
#include <netinet/in.h>
#include <sys/resource.h>
#if !defined (__3DS__)
#include <dlfcn.h>
#endif
#include <sys/unistd.h>
#include <time.h>
#include <sys/types.h>
#include <sys/utsname.h>
#include <semaphore.h>
#include <pwd.h>
#include <dirent.h>
#include <sys/statvfs.h>
#include <locale.h>
#include <sys/poll.h>
#include <sys/socket.h>
#include <netdb.h>

void assert_eq_impl(const char* name, long long got, long long expected);
#define ASSERT_EQ(C, V) assert_eq_impl(#C, C, V)

void assert_bytes_eq_impl(const char* name, long long line, const void* lhs_ptr, long long lhs_size, const void* rhs_ptr, long long rhs_size);
#define ASSERT_BYTES_EQ(A, B) assert_bytes_eq_impl(__FILE__, __LINE__, A, sizeof(*A), B, sizeof(*B))

void assert_t_eq_impl(const char* lhs_name, long long lhs_size, long long lhs_align, const char* rhs_name, long long rhs_size, long long rhs_align);
#define ASSERT_T_EQ(T, U) assert_t_eq_impl(#T, sizeof (T), _Alignof (T), #U, sizeof (U), _Alignof (U))

static void assert_definitions() {

#if defined (__vita__) | defined (__3DS__) | defined (ESP_PLATFORM)
#define POINTER_SIZE 4
#endif

  ASSERT_EQ(POINTER_SIZE, sizeof (void*));

	ASSERT_T_EQ(blkcnt_t, int32_t);
	ASSERT_T_EQ(blksize_t, int32_t);

  ASSERT_T_EQ(clockid_t, unsigned long);

#if defined (__3DS__)
  ASSERT_T_EQ(dev_t, uint32_t);
  ASSERT_T_EQ(off_t, int64_t);
#else
  ASSERT_T_EQ(dev_t, short);
  ASSERT_T_EQ(off_t, long);
#endif

#if defined (__3DS__)
  ASSERT_T_EQ(ino_t, uint32_t);
#elif defined (__sparc__)
  ASSERT_T_EQ(ino_t, unsigned long);
#else
  ASSERT_T_EQ(ino_t, unsigned short);
#endif

	ASSERT_T_EQ(fsblkcnt_t, uint64_t);
	ASSERT_T_EQ(fsfilcnt_t, uint32_t);
	ASSERT_T_EQ(id_t, uint32_t);

#if defined (__3DS__)
  ASSERT_T_EQ(key_t, int);
#else
  ASSERT_T_EQ(key_t, long);
#endif

#if !defined (__vita__)
	ASSERT_T_EQ(loff_t, long long);
#endif

#if defined (__3DS__)
  ASSERT_T_EQ(mode_t, unsigned int);
  ASSERT_T_EQ(nfds_t, uint32_t);
#else
  ASSERT_T_EQ(mode_t, uint32_t);
  ASSERT_T_EQ(nfds_t, unsigned int);
#endif

	ASSERT_T_EQ(nlink_t, unsigned short);

#if defined (__3DS__)
  ASSERT_T_EQ(pthread_t, unsigned long);
#else
  ASSERT_T_EQ(pthread_t, uint32_t);
#endif
	ASSERT_T_EQ(pthread_key_t, unsigned int);
	ASSERT_T_EQ(rlim_t, uint32_t);

#if defined (__3DS__)
  ASSERT_T_EQ(sa_family_t, uint16_t);
#else
  ASSERT_T_EQ(sa_family_t, uint8_t);
#endif

	ASSERT_T_EQ(socklen_t, uint32_t);
#if !defined (__vita__)
	ASSERT_T_EQ(speed_t, uint32_t);
#endif
	ASSERT_T_EQ(suseconds_t, int32_t);
#if !defined (__vita__)
	ASSERT_T_EQ(tcflag_t, unsigned int);
#endif
	ASSERT_T_EQ(useconds_t, uint32_t);

#if defined (__3DS__)
  ASSERT_T_EQ(time_t, long long);
#else
  ASSERT_T_EQ(time_t, int32_t);
#endif

  // The order of the `ai_addr` field in this struct is crucial
  // for converting between the Rust and C types.
  struct addrinfo_replica {
    int ai_flags;
    int ai_family;
    int ai_socktype;
    int ai_protocol;
    socklen_t ai_addrlen;

#if defined (ESP_PLATFORM)
    sockaddr* ai_addr;
#endif

    char* ai_canonname;

#if !defined (ESP_PLATFORM) // & !defined (PPC_NINTENDO or smth)
    struct sockaddr* ai_addr;
#endif

    struct addrinfo* ai_next;
  };
  ASSERT_T_EQ(struct addrinfo_replica, struct addrinfo);

  struct ip_mreq_replica {
    struct in_addr imr_multiaddr;
    struct in_addr imr_interface;
  };
  ASSERT_T_EQ(struct ip_mreq_replica, struct ip_mreq);

  struct linger_replica {
    int l_onoff;
    int l_linger;
  };
  ASSERT_T_EQ(struct linger_replica, struct linger);

#if !defined (__vita__)
  struct in_addr_replica {
    in_addr_t s_addr;
  };
  ASSERT_T_EQ(struct in_addr_replica, struct in_addr);
#endif

  struct hostent_replica {
    char* h_name;
    char** h_aliases;
    int h_addrtype;
    int h_length;
    char** h_addr_list;
    // char* h_addr;
  };
  ASSERT_T_EQ(struct hostent_replica, struct hostent);

  struct pollfd_replica {
    int fd;
    int events;
    int revents;
  };
  ASSERT_T_EQ(struct pollfd_replica, struct pollfd);

  struct lconv_replica {
    char* decimal_point;
    char* thousands_sep;
    char* grouping;
    char* int_curr_symbol;
    char* currency_symbol;
    char* mon_decimal_point;
    char* mon_thousands_sep;
    char* mon_grouping;
    char* positive_sign;
    char* negative_sign;
    char int_frac_digits;
    char frac_digits;
    char p_cs_precedes;
    char p_sep_by_space;
    char n_cs_precedes;
    char n_sep_by_space;
    char p_sign_posn;
    char n_sign_posn;
    char int_n_cs_precedes;
    char int_n_sep_by_space;
    char int_n_sign_posn;
    char int_p_cs_precedes;
    char int_p_sep_by_space;
    char int_p_sign_posn;
  };
  ASSERT_T_EQ(struct lconv_replica, struct lconv);

  struct tm_replica {
    int tm_sec;
    int tm_min;
    int tm_hour;
    int tm_mday;
    int tm_mon;
    int tm_year;
    int tm_wday;
    int tm_yday;
    int tm_isdst;
  };
  ASSERT_T_EQ(struct tm_replica, struct tm);

  struct statvfs_replica {
    unsigned long f_bsize;
    unsigned long f_frsize;
    fsblkcnt_t f_blocks;
    fsblkcnt_t f_bfree;
    fsblkcnt_t f_bavail;
    fsfilcnt_t f_files;
    fsfilcnt_t f_ffree;
    fsfilcnt_t f_favail;
    unsigned long f_fsid;
    unsigned long f_flag;
    unsigned long f_namemax;
  };
  ASSERT_T_EQ(struct statvfs_replica, statvfs);

  struct sigaction_replica {
    void* sa_handler;
    sigset_t sa_mask;
    int sa_flags;
  };
  ASSERT_T_EQ(struct sigaction_replica, sigaction);

  struct dirent_replica {
#if !defined (__vita__)
    ino_t d_ino;
    unsigned char d_type;
#else
    uint8_t __offset[88];
#endif
    char d_name[256];
  };
  ASSERT_T_EQ(struct dirent_replica, struct dirent);

  struct stack_t_replica {
    void* ss_sp;
    int ss_flags;
    size_t ss_size;
  };
  ASSERT_T_EQ(struct stack_t_replica, stack_t);

#define ULONG_SIZE 32
  ASSERT_EQ(ULONG_SIZE, sizeof (unsigned long) * 8);

  struct fd_set_replica { // Unverified
    unsigned long fds_bits[FD_SETSIZE / ULONG_SIZE];
  };
  ASSERT_T_EQ(struct fd_set_replica, fd_set);

  struct passwd_replica { // Unverified
    char* pw_name;
    char* pw_passwd;
    uid_t pw_uid;
    gid_t pw_gid;
    char* pw_gecos;
    char* pw_dir;
    char* pw_shell;
  };
  ASSERT_T_EQ(struct passwd_replica, struct passwd);

#define NCCS 32
#if !defined (__vita__)
  struct termios_replica { // Unverified
    tcflag_t c_iflag;
    tcflag_t c_oflag;
    tcflag_t c_cflag;
    tcflag_t c_lflag;
    cc_t c_line;
    cc_t c_cc[NCCS];
  };
  ASSERT_T_EQ(struct termios_replica, termios);
#endif

  struct sem_t_replica { // Unverified
    char __size[16];
  };
  ASSERT_T_EQ(struct sem_t_replica, sem_t);

#if !defined(__vita__)
  struct Dl_info_replica { // Unverified
    const char* dli_fname;
    void* dli_fbase;
    const char* dli_sname;
    void* dli_saddr;
  };
  ASSERT_T_EQ(struct Dl_info_replica, Dl_info);
#endif

  struct utsname_replica { // Unverified
    char sysname[65];
    char nodename[65];
    char release[65];
    char version[65];
    char machine[65];
    char domainname[65];
  };
  ASSERT_T_EQ(struct utsname_replica, struct utsname);

  struct cpu_set_t_replica { // Unverified
        uint32_t bits[32];
  };
  ASSERT_T_EQ(struct cpu_set_t_replica, cpu_set_t);

#if defined (ESP_PLATFORM)
#define __PTHREAD_INITIALIZER_BYTE 0xff
#define __SIZEOF_PTHREAD_ATTR_T 32
#define __SIZEOF_PTHREAD_MUTEX_T 4
#define __SIZEOF_PTHREAD_MUTEXATTR_T 12
#define __SIZEOF_PTHREAD_COND_T 4
#define __SIZEOF_PTHREAD_CONDATTR_T 8
#define __SIZEOF_PTHREAD_RWLOCK_T 4
#define __SIZEOF_PTHREAD_RWLOCKATTR_T 12
#define __SIZEOF_PTHREAD_BARRIER_T 32
#elif defined (__vita__)
#define __PTHREAD_INITIALIZER_BYTE 0xff
#define __SIZEOF_PTHREAD_ATTR_T 4
#define __SIZEOF_PTHREAD_MUTEX_T 4
#define __SIZEOF_PTHREAD_MUTEXATTR_T 4
#define __SIZEOF_PTHREAD_COND_T 4
#define __SIZEOF_PTHREAD_CONDATTR_T 4
#define __SIZEOF_PTHREAD_RWLOCK_T 4
#define __SIZEOF_PTHREAD_RWLOCKATTR_T 4
#define __SIZEOF_PTHREAD_BARRIER_T 4
#else
#define __PTHREAD_INITIALIZER_BYTE 0
#define __SIZEOF_PTHREAD_ATTR_T 56
#define __SIZEOF_PTHREAD_MUTEX_T 40
#define __SIZEOF_PTHREAD_MUTEXATTR_T 4
#define __SIZEOF_PTHREAD_COND_T 48
#define __SIZEOF_PTHREAD_CONDATTR_T 4
#define __SIZEOF_PTHREAD_RWLOCK_T 56
#define __SIZEOF_PTHREAD_RWLOCKATTR_T 8
#define __SIZEOF_PTHREAD_BARRIER_T 32
#endif

  struct pthread_mutex_t_replica { // Unverified
    uint8_t size[__SIZEOF_PTHREAD_MUTEX_T];
#if (POINTER_SIZE == 4) & (defined (__arm__) | defined (__PPC__) | defined (__mips__))
  } __attribute__ ((aligned (4)));
#else
  } __attribute__ ((aligned (8)));
#endif
  ASSERT_T_EQ(struct pthread_mutex_t_replica, pthread_mutex_t);

  struct pthread_rwlock_t_replica { // Unverified
    uint8_t size[__SIZEOF_PTHREAD_RWLOCK_T];
#if (POINTER_SIZE == 4) & (defined (__arm__) | defined (__PPC__) | defined (__mips__))
  } __attribute__ ((aligned (4)));
#else
  } __attribute__ ((aligned (8)));
#endif
  ASSERT_T_EQ(struct pthread_rwlock_t_replica, pthread_rwlock_t);

  struct pthread_mutexattr_t_replica { // Unverified
    uint8_t size[__SIZEOF_PTHREAD_MUTEXATTR_T];
#if (POINTER_SIZE == 4) | defined (__amd64__) | defined (__PPC64__) | defined (__mips__) | defined (__s390x__) | defined (__sparc__)
  } __attribute__ ((aligned (4)));
#else
  } __attribute__ ((aligned (8)));
#endif
  ASSERT_T_EQ(struct pthread_mutexattr_t_replica, pthread_mutexattr_t);

  struct pthread_cond_t_replica { // Unverified
    uint8_t size[__SIZEOF_PTHREAD_COND_T];
#if defined (__vita__)
  } __attribute__ ((aligned (4)));
#else
  } __attribute__ ((aligned (8)));
#endif
  ASSERT_T_EQ(struct pthread_cond_t_replica, pthread_cond_t);

  struct pthread_condattr_t_replica { // Unverified
    uint8_t size[__SIZEOF_PTHREAD_CONDATTR_T];
  } __attribute__ ((aligned (4)));
  ASSERT_T_EQ(struct pthread_condattr_t_replica, pthread_condattr_t);

  struct pthread_attr_t_replica { // Unverified
    uint8_t __size[__SIZEOF_PTHREAD_ATTR_T];
#if defined (__vita__)
  } __attribute__ ((aligned (4)));
#else
  };
#endif
  ASSERT_T_EQ(struct pthread_attr_t_replica, pthread_attr_t);

  struct pthread_rwlockattr_t_replica { // Unverified
    uint8_t __size[__SIZEOF_PTHREAD_RWLOCKATTR_T];
#if defined (__vita__)
  } __attribute__ ((aligned (4)));
#else
  };
#endif
  ASSERT_T_EQ(struct pthread_rwlockattr_t_replica, pthread_rwlockattr_t);

  // unverified constants
  pthread_mutex_t pthread_mutex_initializer = PTHREAD_MUTEX_INITIALIZER;
  struct pthread_mutex_t_replica pthread_mutex_initializer_replica;
  memset(&pthread_mutex_initializer_replica, __PTHREAD_INITIALIZER_BYTE, sizeof (pthread_mutex_initializer_replica));
  ASSERT_BYTES_EQ(&pthread_mutex_initializer, &pthread_mutex_initializer_replica);

  pthread_cond_t pthread_cond_initializer = PTHREAD_COND_INITIALIZER;
  struct pthread_cond_t_replica pthread_cond_initializer_replica;
  memset(&pthread_cond_initializer_replica, __PTHREAD_INITIALIZER_BYTE, sizeof (pthread_cond_initializer_replica));
  ASSERT_BYTES_EQ(&pthread_cond_initializer, &pthread_cond_initializer_replica);

  pthread_rwlock_t pthread_rwlock_initializer = PTHREAD_RWLOCK_INITIALIZER;
  struct pthread_rwlock_t_replica pthread_rwlock_initializer_replica;
  memset(&pthread_rwlock_initializer_replica, __PTHREAD_INITIALIZER_BYTE, sizeof (pthread_rwlock_initializer_replica));
  ASSERT_BYTES_EQ(&pthread_rwlock_initializer, &pthread_rwlock_initializer_replica);

#define __SIZEOF_PTHREAD_BARRIERATTR_T 4
#define __PTHREAD_MUTEX_HAVE_PREV 1
#define __PTHREAD_RWLOCK_INT_FLAGS_SHARED 1

	ASSERT_EQ(PTHREAD_MUTEX_NORMAL, 0);
	ASSERT_EQ(PTHREAD_MUTEX_RECURSIVE, 1);
	ASSERT_EQ(PTHREAD_MUTEX_ERRORCHECK, 2);
	#if defined(__3DS__) | defined (ESP_PLATFORM)
	ASSERT_EQ(FD_SETSIZE, 64);
	#elif defined(__vita__)
	ASSERT_EQ(FD_SETSIZE, 256);
	#else 
	ASSERT_EQ(FD_SETSIZE, 1024);
	#endif

	ASSERT_EQ(ENOENT, 2);
	ASSERT_EQ(ESRCH, 3);
	ASSERT_EQ(EINTR, 4);
	ASSERT_EQ(EIO, 5);
	ASSERT_EQ(ENXIO, 6);
	ASSERT_EQ(E2BIG, 7);
	ASSERT_EQ(ENOEXEC, 8);
	ASSERT_EQ(EBADF, 9);
	ASSERT_EQ(ECHILD, 10);
	ASSERT_EQ(EAGAIN, 11);
	ASSERT_EQ(ENOMEM, 12);
	ASSERT_EQ(EACCES, 13);
	ASSERT_EQ(EFAULT, 14);
	ASSERT_EQ(EBUSY, 16);
	ASSERT_EQ(EEXIST, 17);
	ASSERT_EQ(EXDEV, 18);
	ASSERT_EQ(ENODEV, 19);
	ASSERT_EQ(ENOTDIR, 20);
	ASSERT_EQ(EISDIR, 21);
	ASSERT_EQ(EINVAL, 22);
	ASSERT_EQ(ENFILE, 23);
	ASSERT_EQ(EMFILE, 24);
	ASSERT_EQ(ENOTTY, 25);
	ASSERT_EQ(ETXTBSY, 26);
	ASSERT_EQ(EFBIG, 27);
	ASSERT_EQ(ENOSPC, 28);
	ASSERT_EQ(ESPIPE, 29);
	ASSERT_EQ(EROFS, 30);
	ASSERT_EQ(EMLINK, 31);
	ASSERT_EQ(EPIPE, 32);
	ASSERT_EQ(EDOM, 33);
	ASSERT_EQ(ERANGE, 34);
	ASSERT_EQ(ENOMSG, 35);
	ASSERT_EQ(EIDRM, 36);
	ASSERT_EQ(EDEADLK, 45);
	ASSERT_EQ(ENOLCK, 46);
	ASSERT_EQ(ENOSTR, 60);
	ASSERT_EQ(ENODATA, 61);
	ASSERT_EQ(ETIME, 62);
	ASSERT_EQ(ENOSR, 63);
	ASSERT_EQ(ENOLINK, 67);
	ASSERT_EQ(EPROTO, 71);
	ASSERT_EQ(EMULTIHOP, 74);
	ASSERT_EQ(EBADMSG, 77);
	ASSERT_EQ(EFTYPE, 79);
	ASSERT_EQ(ENOSYS, 88);
	ASSERT_EQ(ENOTEMPTY, 90);
	ASSERT_EQ(ENAMETOOLONG, 91);
	ASSERT_EQ(ELOOP, 92);
	ASSERT_EQ(EOPNOTSUPP, 95);
	ASSERT_EQ(EPFNOSUPPORT, 96);
	ASSERT_EQ(ECONNRESET, 104);
	ASSERT_EQ(ENOBUFS, 105);
	ASSERT_EQ(EAFNOSUPPORT, 106);
	ASSERT_EQ(EPROTOTYPE, 107);
	ASSERT_EQ(ENOTSOCK, 108);
	ASSERT_EQ(ENOPROTOOPT, 109);
	ASSERT_EQ(ECONNREFUSED, 111);
	ASSERT_EQ(EADDRINUSE, 112);
	ASSERT_EQ(ECONNABORTED, 113);
	ASSERT_EQ(ENETUNREACH, 114);
	ASSERT_EQ(ENETDOWN, 115);
	ASSERT_EQ(ETIMEDOUT, 116);
	ASSERT_EQ(EHOSTDOWN, 117);
	ASSERT_EQ(EHOSTUNREACH, 118);
	ASSERT_EQ(EINPROGRESS, 119);
	ASSERT_EQ(EALREADY, 120);
	ASSERT_EQ(EDESTADDRREQ, 121);
	ASSERT_EQ(EMSGSIZE, 122);
	ASSERT_EQ(EPROTONOSUPPORT, 123);
	ASSERT_EQ(EADDRNOTAVAIL, 125);
	ASSERT_EQ(ENETRESET, 126);
	ASSERT_EQ(EISCONN, 127);
	ASSERT_EQ(ENOTCONN, 128);
	ASSERT_EQ(ETOOMANYREFS, 129);
	ASSERT_EQ(EDQUOT, 132);
	ASSERT_EQ(ESTALE, 133);
	ASSERT_EQ(ENOTSUP, 134);
	ASSERT_EQ(EILSEQ, 138);
	ASSERT_EQ(EOVERFLOW, 139);
	ASSERT_EQ(ECANCELED, 140);
	ASSERT_EQ(ENOTRECOVERABLE, 141);
	ASSERT_EQ(EOWNERDEAD, 142);
	ASSERT_EQ(EWOULDBLOCK, 11);

	ASSERT_EQ(F_DUPFD, 0);
	ASSERT_EQ(F_GETFD, 1);
	ASSERT_EQ(F_SETFD, 2);
	ASSERT_EQ(F_GETFL, 3);
	ASSERT_EQ(F_SETFL, 4);
	ASSERT_EQ(F_GETOWN, 5);
	ASSERT_EQ(F_SETOWN, 6);
	ASSERT_EQ(F_GETLK, 7);
	ASSERT_EQ(F_SETLK, 8);
	ASSERT_EQ(F_SETLKW, 9);
	ASSERT_EQ(F_RGETLK, 10);
	ASSERT_EQ(F_RSETLK, 11);
	ASSERT_EQ(F_CNVT, 12);
	ASSERT_EQ(F_RSETLKW, 13);
	ASSERT_EQ(F_DUPFD_CLOEXEC, 14);

	ASSERT_EQ(O_RDONLY, 0);
	ASSERT_EQ(O_WRONLY, 1);
	ASSERT_EQ(O_RDWR, 2);
	ASSERT_EQ(O_APPEND, 8);
	ASSERT_EQ(O_CREAT, 512);
	ASSERT_EQ(O_TRUNC, 1024);
	ASSERT_EQ(O_EXCL, 2048);
	ASSERT_EQ(O_SYNC, 8192);
	ASSERT_EQ(O_NONBLOCK, 16384);

	ASSERT_EQ(O_ACCMODE, 3);
	ASSERT_EQ(O_CLOEXEC, 0x80000);

	#if !defined (__3DS__)
	ASSERT_EQ(RTLD_LAZY, 0x1);
	#endif

	ASSERT_EQ(STDIN_FILENO, 0);
	ASSERT_EQ(STDOUT_FILENO, 1);
	ASSERT_EQ(STDERR_FILENO, 2);

	ASSERT_EQ(SEEK_SET, 0);
	ASSERT_EQ(SEEK_CUR, 1);
	ASSERT_EQ(SEEK_END, 2);

	#if !defined (__3DS__) & !defined (__vita__)
	ASSERT_EQ(FIOCLEX, 0x20006601);
	ASSERT_EQ(FIONCLEX, 0x20006602);
	#endif

	ASSERT_EQ(S_BLKSIZE, 1024);
	ASSERT_EQ(S_IREAD, 256);
	ASSERT_EQ(S_IWRITE, 128);
	ASSERT_EQ(S_IEXEC, 64);
	ASSERT_EQ(S_ENFMT, 1024);
	ASSERT_EQ(S_IFMT, 61440);
	ASSERT_EQ(S_IFDIR, 16384);
	ASSERT_EQ(S_IFCHR, 8192);
	ASSERT_EQ(S_IFBLK, 24576);
	ASSERT_EQ(S_IFREG, 32768);
	ASSERT_EQ(S_IFLNK, 40960);
	ASSERT_EQ(S_IFSOCK, 49152);
	ASSERT_EQ(S_IFIFO, 4096);
	ASSERT_EQ(S_IRUSR, 256);
	ASSERT_EQ(S_IWUSR, 128);
	ASSERT_EQ(S_IXUSR, 64);
	ASSERT_EQ(S_IRGRP, 32);
	ASSERT_EQ(S_IWGRP, 16);
	ASSERT_EQ(S_IXGRP, 8);
	ASSERT_EQ(S_IROTH, 4);
	ASSERT_EQ(S_IWOTH, 2);
	ASSERT_EQ(S_IXOTH, 1);

	#if !defined (__3DS__) & !defined (__vita__)
	ASSERT_EQ(SOL_TCP, 6);
	#endif

	ASSERT_EQ(PF_UNSPEC, 0);
	ASSERT_EQ(PF_INET, 2);
	ASSERT_EQ(PF_INET6, 23);

	ASSERT_EQ(AF_UNSPEC, 0);
	ASSERT_EQ(AF_INET, 2);

	ASSERT_EQ(CLOCK_REALTIME, 1);
	ASSERT_EQ(CLOCK_MONOTONIC, 4);
	#if !defined (__3DS__) & !defined (__vita__)
	ASSERT_EQ(CLOCK_BOOTTIME, 4); // 7
	#endif

	ASSERT_EQ(SOCK_STREAM, 1);
	ASSERT_EQ(SOCK_DGRAM, 2);

	ASSERT_EQ(SHUT_RD, 0);
	ASSERT_EQ(SHUT_WR, 1);
	ASSERT_EQ(SHUT_RDWR, 2);

	#if !defined (__3DS__) & !defined (__vita__)
	ASSERT_EQ(SO_BINTIME, 0x2000);
	ASSERT_EQ(SO_NO_OFFLOAD, 0x4000);
	ASSERT_EQ(SO_NO_DDP, 0x8000);
	ASSERT_EQ(SO_REUSEPORT_LB, 0x10000);
	ASSERT_EQ(SO_LABEL, 0x1009);
	ASSERT_EQ(SO_PEERLABEL, 0x1010);
	ASSERT_EQ(SO_LISTENQLIMIT, 0x1011);
	ASSERT_EQ(SO_LISTENQLEN, 0x1012);
	ASSERT_EQ(SO_LISTENINCQLEN, 0x1013);
	ASSERT_EQ(SO_SETFIB, 0x1014);
	ASSERT_EQ(SO_USER_COOKIE, 0x1015);
	ASSERT_EQ(SO_PROTOCOL, 0x1016);
	ASSERT_EQ(SO_PROTOTYPE, SO_PROTOCOL);
	ASSERT_EQ(SO_VENDOR, 0x80000000);
	#endif
	#if !defined (__3DS__)
	ASSERT_EQ(SO_DEBUG, 0x01);
	ASSERT_EQ(SO_ACCEPTCONN, 0x0002);
	#endif
	ASSERT_EQ(SO_REUSEADDR, 0x0004);
	#if !defined (__3DS__)
	ASSERT_EQ(SO_KEEPALIVE, 0x0008);
	ASSERT_EQ(SO_DONTROUTE, 0x0010);
	#endif
	ASSERT_EQ(SO_BROADCAST, 0x0020);
	#if !defined (__3DS__)
	ASSERT_EQ(SO_USELOOPBACK, 0x0040);
	#endif
	ASSERT_EQ(SO_LINGER, 0x0080);
	ASSERT_EQ(SO_OOBINLINE, 0x0100);
	#if !defined (__3DS__)
	ASSERT_EQ(SO_REUSEPORT, 0x0200);
	ASSERT_EQ(SO_TIMESTAMP, 0x0400);
	#endif
	#if !defined (__3DS__) & !defined (__vita__)
	ASSERT_EQ(SO_NOSIGPIPE, 0x0800);
	ASSERT_EQ(SO_ACCEPTFILTER, 0x1000);
	#endif
	ASSERT_EQ(SO_SNDBUF, 0x1001);
	ASSERT_EQ(SO_RCVBUF, 0x1002);
	ASSERT_EQ(SO_SNDLOWAT, 0x1003);
	ASSERT_EQ(SO_RCVLOWAT, 0x1004);
	#if !defined (__3DS__)
	ASSERT_EQ(SO_SNDTIMEO, 0x1005);
	ASSERT_EQ(SO_RCVTIMEO, 0x1006);
	#endif
	#if defined (__3DS__)
  ASSERT_EQ(SO_ERROR, 0x1009);
	#else
  ASSERT_EQ(SO_ERROR, 0x1007);
	#endif
	ASSERT_EQ(SO_TYPE, 0x1008);

	#if !defined (__3DS__) & !defined(__vita__)
	ASSERT_EQ(SOCK_CLOEXEC, O_CLOEXEC);
	#endif

	ASSERT_EQ(INET_ADDRSTRLEN, 16);

	#if !defined (__3DS__) & !defined (__vita__)
	ASSERT_EQ(IFF_UP, 0x1);
	ASSERT_EQ(IFF_BROADCAST, 0x2);
	ASSERT_EQ(IFF_DEBUG, 0x4);
	ASSERT_EQ(IFF_LOOPBACK, 0x8);
	ASSERT_EQ(IFF_POINTOPOINT, 0x10);
	ASSERT_EQ(IFF_NOTRAILERS, 0x20);
	ASSERT_EQ(IFF_RUNNING, 0x40);
	ASSERT_EQ(IFF_NOARP, 0x80);
	ASSERT_EQ(IFF_PROMISC, 0x100);
	ASSERT_EQ(IFF_ALLMULTI, 0x200);
	ASSERT_EQ(IFF_OACTIVE, 0x400);
	ASSERT_EQ(IFF_SIMPLEX, 0x800);
	ASSERT_EQ(IFF_LINK0, 0x1000);
	ASSERT_EQ(IFF_LINK1, 0x2000);
	ASSERT_EQ(IFF_LINK2, 0x4000);
	ASSERT_EQ(IFF_ALTPHYS, IFF_LINK2);
	ASSERT_EQ(IFF_MULTICAST, 0x8000);
	#endif

	ASSERT_EQ(TCP_NODELAY, 1);
	ASSERT_EQ(TCP_MAXSEG, 2);
	#if !defined (__3DS__) & !defined(__vita__)
	ASSERT_EQ(TCP_NOPUSH, 4);
	ASSERT_EQ(TCP_NOOPT, 8);
	ASSERT_EQ(TCP_KEEPIDLE, 256);
	ASSERT_EQ(TCP_KEEPINTVL, 512);
	ASSERT_EQ(TCP_KEEPCNT, 1024);
	#endif

	#if defined (__3DS__)
  ASSERT_EQ(IP_TOS, 7);
  ASSERT_EQ(IP_TTL, 8);
  ASSERT_EQ(IP_MULTICAST_LOOP, 9);
  ASSERT_EQ(IP_MULTICAST_TTL, 10);
  ASSERT_EQ(IP_ADD_MEMBERSHIP, 11);
  ASSERT_EQ(IP_DROP_MEMBERSHIP, 12);
  #else
  ASSERT_EQ(IP_TOS, 3);
  ASSERT_EQ(IP_TTL, 4);
  ASSERT_EQ(IP_MULTICAST_IF, 9);
  ASSERT_EQ(IP_MULTICAST_TTL, 10);
  ASSERT_EQ(IP_MULTICAST_LOOP, 11);
  ASSERT_EQ(IP_ADD_MEMBERSHIP, 12);
  ASSERT_EQ(IP_DROP_MEMBERSHIP, 13);
	#endif

	#if !defined (__3DS__) & !defined (__vita__)
	ASSERT_EQ(IPV6_UNICAST_HOPS, 4);
	ASSERT_EQ(IPV6_MULTICAST_IF, 9);
	ASSERT_EQ(IPV6_MULTICAST_HOPS, 10);
	ASSERT_EQ(IPV6_MULTICAST_LOOP, 11);
	ASSERT_EQ(IPV6_V6ONLY, 27);
	ASSERT_EQ(IPV6_JOIN_GROUP, 12);
	ASSERT_EQ(IPV6_LEAVE_GROUP, 13);
	ASSERT_EQ(IPV6_ADD_MEMBERSHIP, 12);
	ASSERT_EQ(IPV6_DROP_MEMBERSHIP, 13);
	#endif

	#if !defined (__vita__)
	ASSERT_EQ(HOST_NOT_FOUND, 1);
	ASSERT_EQ(NO_DATA, 2);
	ASSERT_EQ(NO_ADDRESS, 2);
	ASSERT_EQ(NO_RECOVERY, 3);
	ASSERT_EQ(TRY_AGAIN, 4);
	#endif

	ASSERT_EQ(AI_PASSIVE, 1);
	ASSERT_EQ(AI_CANONNAME, 2);
	ASSERT_EQ(AI_NUMERICHOST, 4);
	ASSERT_EQ(AI_NUMERICSERV, 0);
	ASSERT_EQ(AI_ADDRCONFIG, 0);

	ASSERT_EQ(NI_MAXHOST, 1025);
	ASSERT_EQ(NI_MAXSERV, 32);
	ASSERT_EQ(NI_NOFQDN, 1);
	ASSERT_EQ(NI_NUMERICHOST, 2);
	ASSERT_EQ(NI_NAMEREQD, 4);
	ASSERT_EQ(NI_NUMERICSERV, 0);
	ASSERT_EQ(NI_DGRAM, 0);

	#if !defined (__3DS__)
	ASSERT_EQ(EAI_AGAIN, 2);
	ASSERT_EQ(EAI_BADFLAGS, 3);
	ASSERT_EQ(EAI_FAIL, 4);
	#endif
	ASSERT_EQ(EAI_FAMILY, 5);
	ASSERT_EQ(EAI_MEMORY, 6);
	ASSERT_EQ(EAI_NONAME, 8);
	#if !defined (__3DS__)
	ASSERT_EQ(EAI_SERVICE, 9);
	#endif
	ASSERT_EQ(EAI_SOCKTYPE, 10);
	#if !defined (__3DS__)
	ASSERT_EQ(EAI_SYSTEM, 11);
	#endif
	#if !defined (__3DS__) & !defined (__vita__)
	ASSERT_EQ(EAI_BADHINTS, 12);
	ASSERT_EQ(EAI_PROTOCOL, 13);
	#endif
	#if !defined (__3DS__)
	ASSERT_EQ(EAI_OVERFLOW, 14);
	#endif

	ASSERT_EQ(EXIT_SUCCESS, 0);
	ASSERT_EQ(EXIT_FAILURE, 1);

	#if !defined (__3DS__)
	ASSERT_EQ(PRIO_PROCESS, 0);
	ASSERT_EQ(PRIO_PGRP, 1);
	ASSERT_EQ(PRIO_USER, 2);
	#endif
}

EDIT: I think i've modified the script code so some test maybe false positives/negatives on all/some platforms

Jan 04 '24 10:01 zetanumbers

Could anyone give me a summary of the discussion here? I'd like to help merge this but don't have enough context.

A lot of wrong definitions, particularly for newlib targets. The main reason, I would say, is that it has definitions from libctru for nintendo 3DS, which are super different from the most og newlib. This is because newlib code was introduced precisely for nintendo 3DS.

I no longer have time to untangle this problem to check field offests, struct alignments and sizes, etc. I would be happy to pass this torch to anyone interested. I thought of making some instrument to generate test C code, but it bacame too unwieldy to me. I also had hard time configuring ESP32 toolchain, it being a bash shell you have to dive into before getting access to compiler, so I haven't managed to check it either.

I can try to run your script on top of ESP IDF, but it would take a few days. Will report here once I have some progress.

Jan 05 '24 19:01 ivmarkov

Keep in mind horizon os of 3DS as to preserve these libctru definitions.

As long as all of these are marked with target_os = "horizon", sure. Otherwise how would you suggest we can figure out what is horizon-specific vs what is plain wrong. Or shall we treat all "wrong" definitions as horizon-specific?

Jan 05 '24 19:01 ivmarkov

Keep in mind horizon os of 3DS as to preserve these libctru definitions.

As long as all of these are marked with target_os = "horizon", sure. Otherwise how would you suggest we can figure out what is horizon-specific vs what is plain wrong. Or shall we treat all "wrong" definitions as horizon-specific?

Nothing you can't think of. I've roughtly translated newlib.rs into this script, but ideally there should be an automatic tool for every target.

EDIT: Sorry, didn't answer your question. For now wrong horizon os (or more correctly for exact armv6k-nintendo-3ds target) should probably stay the same.

Jan 06 '24 09:01 zetanumbers

Hi all, I meant to mention this sooner, but a long time ago I had been trying to add test support in libc-test/build.rs for the 3DS target, which might help avoid regressions here...

The changes were originally in https://github.com/Meziu/libc/pull/8 and are kinda hacky and not very portable since they require the 3ds toolchain to compile and a device/emulator to actually run the tests. I started trying to revive that test changes and so far have had a bunch of trouble compiling, but I can post back here if I get something working.

At the time, we didn't have a way to run tests automatically for the 3DS but have since created https://github.com/rust3ds/test-runner/. Maybe a potential path forward would be to add the 3DS to this repo's CI? It would probably take a fair amount of work to get set up, so I wouldn't want it to block this PR but just thinking about the long term

For now, this is probably the easiest option:

Or shall we treat all "wrong" definitions as horizon-specific?

I think this is basically what we did whenever we found a conflict between different definitions, and assumed that the 3DS definition was simply unusual. If it's not too much trouble to preserve all the current definitions with #[cfg(target_os = "horizon")] that's probably easiest, and in the long term we can go back and unify the definitions if they turn out to be wrong for the 3DS too (via the testing I mentioned above).

Jan 06 '24 20:01 ian-h-chamberlain

Hi! Are there any updates regarding this PR? @zetanumbers

Thanks in advance!

Sep 10 '24 07:09 SergioGasquez

Could anyone give me a summary of the discussion here? I'd like to help merge this but don't have enough context.

A lot of wrong definitions, particularly for newlib targets. The main reason, I would say, is that it has definitions from libctru for nintendo 3DS, which are super different from the most og newlib. This is because newlib code was introduced precisely for nintendo 3DS. I no longer have time to untangle this problem to check field offests, struct alignments and sizes, etc. I would be happy to pass this torch to anyone interested. I thought of making some instrument to generate test C code, but it bacame too unwieldy to me. I also had hard time configuring ESP32 toolchain, it being a bash shell you have to dive into before getting access to compiler, so I haven't managed to check it either.

I can try to run your script on top of ESP IDF, but it would take a few days. Will report here once I have some progress.

@SergioGasquez specifically for ESP IDF, this ^^^ is pending. On our side. It is just that several months ago, I was asking for help in the esp-rs community because I don't have cycles to do this (and other libc related stuff from "the list") myself.

Sep 10 '24 07:09 ivmarkov