Way to avoid the (LENGTH) call?

[phmarek]

I'd like to use one kernel for multiple tasks (to have a fixed amount of threads active); these need some special variable set, to different values depending on callsite.

Now, :BINDINGS in MAKE-KERNEL (if I read that correctly) takes the value of the thread running MAKE-KERNEL; I need to have different values, so that's out.

Next idea was to pass the needed value to the thread:

(lparallel:pmap
  nil
  (lambda (a b)
    (let ((*special* a))
      (... b)))
  (alexandria:make-circular-list 1 :initial-element *special*)
  (list :a :b :c :d))

But that just hangs, because PMAP* runs LENGTH on the input, which won't work with the circular list. Of course I could build a list with the same length as the other input sequence... but that's quite CONSing, isn't it? ;)

I found :SIZE, but that should be set to a smaller value than the list length - and then not all elements will be processed. (Otherwise, I could use MOST-POSITIVE-FIXNUM ;)

:CONTEXT is for the whole length of a worker thread, so I can't rebind the special if the same kernel is used for another task.

Is there some easy way to get a call-size specific special into the work items?

Thank you for any help!

[lmj]

Well :bindings can use the values from the thread calling make-kernel, but it's more general than that since the values for the bindings are obtained from eval in each worker thread. This isn't lparallel-specific; it's just the behavior of bordeaux-threads. Conceptually, :bindings is just a convenience wrapper for :context that is tailored for initializing specials. Both are one-time triggers called at the start of each worker thread.

But you want to rebind a dynamic variable inside tasks, so, right, neither :bindings nor :context are going to help in this case.

It's not clear to me why :size wouldn't work here. It doesn't call length and thus works fine on circular lists which are effectively treated as sequences of infinite length. The value of :size is not restricted.

(lparallel:pmap 'vector
                #'+
                :size 3
                (alexandria:make-circular-list 1 :initial-element 4)
                (alexandria:make-circular-list 1 :initial-element 5))
;; => #(9 9 9)

But if you just want to carry dynamic bindings into tasks then that is more easily done with a macro.

(defmacro dynamic-lambda (lambda-list vars &body body)
  "Capture the current values of the dynamic variables in `vars' and
return a lambda inside which those dynamic variables are bound to
those values, respectively."
  (let ((syms (loop repeat (length vars) collect (gensym))))
    `(let ,(mapcar #'list syms vars)
       (lambda ,lambda-list
         (let ,(mapcar #'list vars syms)
           ,@body)))))

(defvar *foo*)

(defun use-foo (x)
  (+ x *foo*))

(let ((*foo* 99))
  (lparallel:pmap 'vector
                  (dynamic-lambda (x) (*foo*) 
                    (use-foo x))
                  #(1 2 3)))
;; => #(100 101 102)