Contiki: Update to Contiki-NG

[kkrentz]

This PR updates libcoap to the latest version of Contiki: Contiki-NG. I tested some basic scenarios without DTLS. Three remaining issues I am aware of are:

• ~~It is not currently possible to use libcoap and Contiki-NG's logging API within the same C file due to name collisions.~~ (in C files using Contiki's logging API, include log.h after coap.h; don't mix them in a single C file)
• ~~libcoap's Contiki example quickly runs out of RAM when I query, e.g., .well-known/core. This may be normal or a memory leak in the Contiki-specific code or in libcoap.~~ (to support multiple clients, call coap_context_set_max_idle_sessions)
• A few commits need to be merged into Contiki-NG for libcoap to work, in particular contiki-ng/contiki-ng#522. I created a branch that entails those commits: git clone --branch libcoap-pr https://github.com/kkrentz/contiki-ng.git.

[obgm]

Nice, thank you!

[kkrentz]

I found the reason why Contiki-NG runs out of RAM so quickly. When a new request comes in, libcoap allocates a new coap_session_t of type COAP_SESSION_TYPE_SERVER. There seem to no procedure that frees "server sessions", is there?

[mrdeep1]

In coap_endpoint_get_session() there is the checking for whether there is a match on an existing session, an old session is no longer in use (idle) so can be reused for a new session, or a new session is created.

There is a function coap_context_set_max_idle_sessions() that can be used to set the maximum number of idle sessions so that sessions get re-used more rapidly. See coap_context(3) for more information.

Then there is coap_io_prepare_io() (usually called by coap_io_process()) (posix variants) which will remove an idle session based on COAP_DEFAULT_SESSION_TIMEOUT, a or value defined by coap_context_set_session_timeout(). However, the coap_io_process() for CONTIKI does not call coap_io_prepare_io() which is where I think you are seeing the issue. Note that CONTIKI coap_io_prepare() appears not to be used and I think should be named coap_io_prepare_io() even though it is a stub function.

[I have not checked the above against your specific patch]

[mrdeep1]

You will need to have code that looks something like (in coap_io_process()):-

  coap_session_t *s, *rtmp;
  coap_endpoint_t *ep;
  coap_tick_t session_timeout;

  if (ctx->session_timeout > 0)
    session_timeout = ctx->session_timeout * COAP_TICKS_PER_SECOND;
  else
    session_timeout = COAP_DEFAULT_SESSION_TIMEOUT * COAP_TICKS_PER_SECOND;

  LL_FOREACH(ctx->endpoint, ep) {
    SESSIONS_ITER_SAFE(ep->sessions, s, rtmp) {
      if (s->type == COAP_SESSION_TYPE_SERVER && s->ref == 0 &&
          s->delayqueue == NULL &&
          (s->last_rx_tx + session_timeout <= now ||
           s->state == COAP_SESSION_STATE_NONE)) {
        coap_handle_event(ctx, COAP_EVENT_SERVER_SESSION_DEL, s);
        coap_session_free(s);
      }
    }
  }

What does concern me is that the posix coap_io_prepare_io() does other checks as well which may be stumbled into when doing CONTIKI testing.

[kkrentz]

coap_context_set_max_idle_sessions did the trick. I also integrated with coap_io_prepare_io, which obviates most of the retransmission business. Thank you!

[mrdeep1]

A few commits need to be merged into Contiki-NG for libcoap to work, in particular https://github.com/contiki-ng/contiki-ng/pull/522. I created a branch that entails those commits: git clone --branch libcoap-pr https://github.com/kkrentz/contiki-ng.git.

Until the CSPRNG work has been merged into Contiki-NG, libcoap must work with the vanilla Contiki-NG, which means that random_rand() must be used for this PR to be merged. There is nothing stopping you having your own additional PR to support CSPRNG that is separate from this PR.

Can you please remove non-vanilla Contiki-NG usage from this PR so that I can review changes and do some testing against the latest Contiki-NG code. Thanks.

[mrdeep1]

@kkrentz As I would like to see this PR merged, I am thinking about taking all your changes, backing out the CSPRNG specific changes as they have not hit mainstream Contiki-NG yet and merging the remainder into the develop branch after doing a review and testing. What do you think?

[kkrentz]

@mrdeep1 Exchanging csprng_rand with random_rand is not a big deal. Nevertheless, some further tweaks to Contiki-NG would be necessary. I created another PR over there contiki-ng/contiki-ng#2400.

[mrdeep1]

Actually, https://github.com/contiki-ng/contiki-ng/pull/2400 is not needed. Making this change to your code

diff --git a/Makefile.libcoap b/Makefile.libcoap
index 5e8dccab..c181ed2e 100644
--- a/Makefile.libcoap
+++ b/Makefile.libcoap
@@ -1,5 +1,5 @@
 LIBCOAP_DIR = os/net/app-layer/libcoap
 MODULES += $(LIBCOAP_DIR)/src
 MODULES += $(LIBCOAP_DIR)/include
-MODULES_EXCLUSIONS += $(LIBCOAP_DIR)/src/coap_io_riot.c
-MODULES_EXCLUSIONS += $(LIBCOAP_DIR)/src/coap_io_lwip.c
+MODULES_SOURCES_EXCLUDES += coap_io_riot.c
+MODULES_SOURCES_EXCLUDES += coap_io_lwip.c

fixes your issue.

[kkrentz]

Indeed, it seems to work with MODULES_SOURCES_EXCLUDES . MODULES_EXCLUSIONS is only required by my OSCORE implementation and I eventually used it for excluding all unwanted source files.

[kkrentz]

Am now using random_rand and MODULES_SOURCES_EXCLUDES.

[mrdeep1]

Thanks - will check it out tomorrow

[mrdeep1]

Thanks for making the changes. It would be good to also see the minor nits fixed as in contiki.patch.txt

It would be good to do an interoperability test between your OSCORE implementation and libcoap's version (I appreciate that they started from the same code base way back) to check there are no issues. examples/oscore_testcases.sh would be a good starting pint.

[mrdeep1]

All looks good to me - thanks.