Intermixed info in log file when processing multiple files

[HolterPhylo]

Describe the bug Intermixed info in log file when processing multiple files

Expected behavior Each processed file info in its block of text

Steps To Reproduce Process a large amount of files that generate a mix of WARN and/or ERR as well as no WARN/ERR (just INFO lines)

Specs (please complete the following information, if applicable):

• libMBIN (MBINCompiler) Version: 4.37.0.1
• NMS Game Version or Steam Build Number: irrelevant
• MBIN File: irrelevant
• MBIN File Version: irrelevant

Attachments: see example log output MBINCompiler.log  ://github.com/monkeyman192/MBINCompiler/assets/3037251/103d1393-faf7-4210-b49b-d13c154f437e)

[HolterPhylo]

Also, this happens: missing 2 spaces in front of lines 18-19 on the left that are normally like the ones on the right. This with what happens above makes parsing for the right filename of the reported ERROR/WARNING very hit and miss.

[cmkushnir]

If mbinc is processing mult mbin in parallel then messages written to single log will be mixed. The easiest way to prevent this is to have each mbin instance maintain its own log (list) of messages, then write them all to the log once the mbin finishes processing. The write to the log would be serialized, so although all messages for a given mbin would be together the order that the mbin's are reported would be random (depends when a thread finishes a given mbin). If you want mbin's in order as well then wait until all mbin's finish then write to log, however, then carrying state around for each mbin after it's finished which won't scale well ... though for this case would likely be ok. Other issue is if crash, then won't have messages in log upto crash point.

[HolterPhylo]

Thanks for comment. Keeping the order of the files processing is not important. Keeping the information of each file process together is what counts, not all intermixed with other files output

[HolterPhylo]

From Discord:

• One idea I have would be to assign an index to each read file on input and use that index in all logging for that file. The log could then be all mixed up but it would be easy to related the index to the right file when reading the log.

• AMUMSS does not run multiple copy of MBINCompiler at the same time, it just ask MBINCompiler to process all MBIN or EXML in a given folder. Almost the same as dropping the folder content on MBINCompiler.

• The script CompanionPetUnlocker_1.lua.txt (remove .txt extension) put in AMUMSS ModScript folder and executing BUILDMOD.bat will produce a MBINCompiler.log MBINCompiler_1.log. One MBIN to process to a modded EXML that cannot be compiled (see WARN)

• The script CompanionPetUnlocker_2.lua.txt (remove .txt extension) put in AMUMSS ModScript folder and executing BUILDMOD.bat will produce a MBINCompiler.log MBINCompiler_2.log where the WARN is anonymous/not related to the preceding processed EXML file. It is impossible to connect the WARN to the right file.

[HolterPhylo]

MBINCompiler is called like this:

  -- action = Compile
  -- sourcePath = where the EXML files are relative to MODBUILDER folder
  -- IsWithThreads = bool, optional
--
-- returns: string: success
function H.MBINCompiler_C(sourcePath,IsWithThreads)
  if IsWithThreads == nil then IsWithThreads = true end
  local threadInfo = "(multi-thread)"
  local withThreads = ""
  if not IsWithThreads then
    withThreads = "--no-threads"
    threadInfo = "(single-thread, due to MBINCompiler buggy log in multi-thread)"
  end
  
  local success = ""
  
   --clear .log
   local cmd = [[del MBINCompiler.log 1>NUL 2>NUL]]
   os.execute(cmd)
 
   local start = os.clock()
   print(H._zBRIGHTGREEN.."     @@@ creating MBIN files "..threadInfo.."..."..H._zDEFAULT)

  --  ===========
   local cmd = [[MBINCompiler.exe -q -y -f -iEXML --exclude=";" ]]..withThreads..[[ "]]..sourcePath --..[["]]
  --  ===========

   local state,str,num = os.execute(cmd) --fast and same output as batch
   
   local delta = os.clock() - start
   print(H._zBRIGHTGREEN.."      - done in "..H.dClock(delta)..H._zDEFAULT)
   
   if state then
     success = "OK"
   else
     -- print("@@@ MBINCompiler returned: "..str..", "..tostring(num))
     success = "ERROR"
   end 

  return success
end