# Could I have concurrency problem here?



## Nowhk (Nov 26, 2015)

Hi all,

I have multiple wait that are running in parallel using the "trick" of triggers note for making a new "thread" for each note


```
on listener
  select (NI_SIGNAL_TYPE)
    case NI_SIGNAL_TRANSP_START
      status := 1
      lfo_trigger_1 := play_note(30, 1, 0, 1) 
      lfo_trigger_2 := play_note(30, 1, 0, 1)
    case NI_SIGNAL_TRANSP_STOP
      status := 0
  end select
end on

on release
  if ($EVENT_ID=$lfo_trigger_1)
    while ($status=1)
      $index := random(0, 100)
      message("thread 1: " & $index)
      wait(100)
    end while
  end if
  if ($EVENT_ID=$lfo_trigger_2)
    while ($status=1)
      $index := random(0, 100)
      message("thread 2: " & $index)
      wait(1000)
    end while
  end if
end on
```
I'm correct if I say that $index here could have concurrency problems? I mean, that I print "thread 1: value" with the value of the $lfo_trigger_2 instead of the above/just generated $lfo_trigger_1? Or am I wrong?

How would you fix this if so? Thanks.


----------



## EvilDragon (Nov 26, 2015)

Yeah you should probably use a different index variable for the second trigger.


----------



## Nowhk (Nov 26, 2015)

Damn, that's a panic! Since I use those variable inside Nil's functions, that are called (so copied and pasted) many times across different threads :O


----------



## EvilDragon (Nov 26, 2015)

That's why we love find&replace tool, isn't it?


----------



## Nowhk (Nov 26, 2015)

Uhm no  Because I can't do find&replace in Nil's function: using that, it will copy and propagate the same variable name in all compiled functions, but later. So I should, every time I compile, watch my whole code and for each shared variable doing a replace. This is the pain...


----------



## Nowhk (Nov 26, 2015)

Here for example:


```
function SetSingleLFOValue(led_index, lfo_shape_index, lfo_shape)
    set_control_par(engine_knob_array[led_values[led_index]], CONTROL_PAR_VALUE, lfo_shape[lfo_shape_index])
    engine_row := led_values[led_index] / 3
    engine_column := led_values[led_index] mod 3
    select(engine_column)
          case 0
            ChangeEngineTune(engine_knob_array[led_values[led_index]], engine_row)       
          case 1
           ChangeEngineEQ(engine_knob_array[led_values[led_index]], engine_row)
          case 2
              ChangeEngineDist(engine_knob_array[led_values[led_index]], engine_row)
    end select
end function

function ChangeEngineTune(knob, group)
  knob_value := get_control_par(knob, CONTROL_PAR_VALUE)
  set_control_par_str(knob, CONTROL_PAR_LABEL, knob_value & " %")
  set_engine_par(ENGINE_PAR_TUNE, knob_value * 10000, group, 0, 1)  
end function
```
*engine_row*, *engine_column* and *knob_value* will suffer the problem, if I call *SetSingleLFOValue* in two different note-threads. And I can't use find&replace here...


----------



## EvilDragon (Nov 26, 2015)

OK then an easier solution would be to make those values an array instead. Then your first thread is, say, knob_value[0], your second thread is knob_value[1]. That's easier to fix.


----------



## andreasOL (Nov 26, 2015)

Is there really a concurrency problem with $index? To my understanding a script can have more than one thread (here start with the bleep note mechanism) but there no preemtive multitasking in one script, i.e. a variable is safe between points like "start of callback", "end of callback" and wait statements. After a wait it could have been altered in another thread but here is it set and used (in the message) without a point where the script can switch to another thread. After the wait, of course, it can be altered by the other while loop

Andreas


----------



## Nowhk (Nov 26, 2015)

andreasOL said:


> Is there really a concurrency problem with $index? To my understanding a script can have more than one thread (here start with the bleep note mechanism) but there no preemtive multitasking in one script, i.e. a variable is safe between points like "start of callback", "end of callback" and wait statements. After a wait it could have been altered in another thread but here is it set and used (in the message) without a point where the script can switch to another thread. After the wait, of course, it can be altered by the other while loop
> 
> Andreas


So you are saying that variables before wait are safe? I.e. variable between a starting while iteration and wait are safe?


----------



## andreasOL (Nov 26, 2015)

Nowhk said:


> So you are saying that variables before wait are safe? I.e. variable between a starting while iteration and wait are safe?


Yes.

To turn your question into a more general statement: variables are safe between points where Kontakt can switch to another thread (of the same script).


----------



## Nowhk (Nov 26, 2015)

So even if I'm using more "note bleep" thread, the process that is executed is always one at time, when the first wait stop to "wait". In this case I should be ok, since all variable I manage are between the init of a while the start of the wait...

EvilDragon can you confirm this? A sort of double check would be even better


----------



## willbedford (Nov 26, 2015)

Yes, KSP runs in a single thread, so your variables will be safe until the first wait statement.
This is why we use the word "thread" in quotation marks - using the note release trick isn't really creating a new thread. 

If you want the variable to stay safe throughout the callback even after wait statements, make it a polyphonic variable.


----------



## Nowhk (Nov 26, 2015)

willbedford said:


> Yes, KSP runs in a single thread, so your variables will be safe until the first wait statement.


What do you mean with "first wait statement"? That then all variable all not safe or that it will treat the new variables "safe" until the next wait, and so till the end of process? Having variable that are re-init at every while iteration (and not used after the wait, since after the wait I've only the re-start of iteration) I should suffer this problem.

In few words, even if I "split" 4 different notes/process, Kontakt still process only 1 of this at time?


----------



## andreasOL (Nov 26, 2015)

I'm a fan of general answers...

So, a variable *is* safe during the execution path of a script *until* the end of the callback is reached or until a wait statement is executed (in that callback). Both of these conditions (end of cb or wait) allow Kontakt to continue operation at a different location in your script which can be another callback (because a e.g. midi or a listener event is waiting for being handled) or another wait of which the time has elapsed. If there's no new event to be processed or no wait has reached its end Kontakt continues with its background work, i.e. playing sounds according to the group programming etc. reacting on user input, updating a gui, whatever.


----------



## willbedford (Nov 26, 2015)

Kontakt will only process one callback at a time. When it reaches a wait statement, it will leave the current callback and process any other callbacks in the queue, then return to the original callback after the 'wait' is complete. During the wait, if any other callback has changed a variable you were using in the first callback, the value will also change when the first callback has resumed. If the variable is polyphonic, Kontakt creates a unique copy of that variable for each callback instance. This means more memory will be used, but you can be sure that the value can't be changed outside the current callback.

Edit: Andreas posted his answer while I was writing mine, and explained things better than me


----------



## andreasOL (Nov 26, 2015)

...and yes to your last question. Apart from multiple cores running truely parallel (which is nothing to take into account) all is done sequentially but so fast that is appears as if multiple things are done simultaneously.


----------



## Nowhk (Nov 27, 2015)

Perfect! So in this case I'm ok, since at every while iteration I re-init all variables with new values! Thank you for clarify this!


----------



## RustyPine (Apr 8, 2016)

This thread has been very useful! I have a question regarding the reentry of a callback that's done waiting while another is still running. Say I have 2 separate callbacks.

- Callback* A* is served first, and it runs into a wait(50) where it waits for 50 uSec.
- Callback *B *is next in the queue, and while *A *is waiting it's served. It has a lot of work to do and takes much longer then 50 uSec. Lets say it takes 100uSec to finish (and never runs into a wait() statement).

Will callback *A
1) *"interrupt" & "freeze" *B *and resume after 50uSec? Or
*2) *wait until *B *is finished

In other words, does switching between callbacks only ever happen after either a _Callback Ends_ or _Callback runs into a wait() statement _? 

If *2)* is true, then the wait() statement would be faulty, correct? It wouldn't be waiting for that exact amount of time
If *1) *is true, then I have questions about what kind of statements are atomic. For eg, is the simple assignment operation _$var := 0 _atomic? This is in tandem with a question I posted on another KSP forum (but I can't post the link because it's being flagged as spam. It's the recent "Thread Safety" message thread on the NI Kontakt Workshop forum)

Many thanks for all the help in advance!!


----------



## andreasOL (Apr 24, 2016)

RustyPine said:


> This thread has been very useful! I have a question regarding the reentry of a callback that's done waiting while another is still running. Say I have 2 separate callbacks.
> 
> - Callback* A* is served first, and it runs into a wait(50) where it waits for 50 uSec.
> - Callback *B *is next in the queue, and while *A *is waiting it's served. It has a lot of work to do and takes much longer then 50 uSec. Lets say it takes 100uSec to finish (and never runs into a wait() statement).
> ...



Hi, I'd say "definitely 2".

You cannot rely on waits from a real-time perspective. In fact, if you say wait(50) and Kontakt has nothing else to do, it can be, that it even continues the thread immediately and that whatever you do after the wait that affects e.g. playings voice will affect the chunks (buffer size) in which audio data is transferred to the host *as if* it is done exactly 50us after what has been done before the wait. I can elaborate more about this if you want.

Andreas


----------



## Lindon (Apr 25, 2016)

Not so sure about this:


> In fact, if you say wait(50) and Kontakt has nothing else to do, it can be, that it even continues the thread immediately


I used to use wait statements to manage a sequencer,(in the days before on listener) and it had nothing else to do, but still stayed in time.. I think it always waits at least the wait statement. But I've been wrong more than once here...


----------



## andreasOL (Apr 25, 2016)

Lindon said:


> Not so sure about this:
> 
> I used to use wait statements to manage a sequencer,(in the days before on listener) and it had nothing else to do, but still stayed in time.. I think it always waits at least the wait statement. But I've been wrong more than once here...



Yep...this would be an interesting topic for further investigation. You can be right here...and I think what you write and what I wrote is no contradiction. Image you export a final audio file in your host and have unchecked "[ ] real-time export" (Cubase has this IIRC) then you can export a song that runs for e.g. 5 minutes in under a minute if your computer has the power. Of course, if you have embedded loops and sequences in one of the used Kontakt instruments and they are done with waits everything stays in time but, it's not real-time, but for the period of time the export takes, a faster time, which e.g. $ENGINE_UPTIME correctly reflects (contrary to $KSP_TIMER which always give useconds of our "real-time"). If you don't do an "offline bounce" but normal playback in your host then this can also happen during the generation of the audio chunks that are delivered from a VSTi to the host as these chunks must be prepared fast (and early) enough to be ready for playback in "our real-time".


----------

