Agfdhyk

I have a bash script which processes several years of data, therefore the script can take a week to finish.
To speed up the process, I use multithreading, by running several instances in parallel (each instance = 1 day of data). Each instance occupies 1 CPU, so I can run as many instances as CPUs available. As I am running the process in a powerful server which I am sharing with others, at some time I may have more or less CPUs available.
My current script is:

#!/bin/bash

function waitpid {

   #Gather the gLABs PID background processes (Maximum processes in 

   #background as number of CPUs)

   NUMPIDS=`jobs -p|awk 'END {print NR}'`

   #A while is set because there seems to be a bug in bash that makes 

   #sometimes the "wait -n" command

   #exit even if none of provided PIDs have finished. If this happens, 

   #the while loops forces the 

   #script to wait until one of the processes is truly finished

   while [ ${NUMPIDS} -ge ${NUMCPUS} ]

   do

     #Wait for gLAB processes to finish

     PIDS="`jobs -p|awk -v ORS=" " '{print}'`"

     wait -n ${PIDS} >/dev/null 2>/dev/null

     NUMPIDS=`jobs -p|awk 'END {print NR}'`

   done

}

NUMPCUS=10

for(...) #Loop for each day

do

   day=... #Set current day variable

   #Command to execute, put in background

   gLAB_linux -input ${day}folder/${day}.input -output ${day)outfolder/${day}.output &        

   #Wait for any process to finish if NUMCPUS number of processes are running in background

   waitpid 

done

Therefore, my question is: if this script is running, is there any way I can change the variable NUMCPUS to any value (for instance NUMCPUS=23) without stopping the script?.
If possible, I would prefer a method that does not involve reading or writing to file (I like to reduce temporary files to 0 if possible).
I don't mind if it is a "hackish" process, such as the method described in this answer. Actually, I tried in gdb similar commands as in that answer but it didn't work, I had the following errors in gdb (and also made the process to crash):

(gdb) attach 23865

(gdb) call bind_variable("NUMCPUS",11,0)

'bind_variable' has unknown return type; cast the call to its declared return type

(gdb) call (int)bind_variable("NUMCPUS",11,0)

Program received signal SIGSEGV, Segmentation fault

EDIT1: Some comments to the script:

• gLAB_linux is a single core processing program and is not aware of NUMCPUS variable


• Each gLAB_linux execution takes around 5 hours to finish, therefore the bash script is most of the time sleeping inside the wait -n.


• NUMCPUS must be a local variable for the script, as there may be another script like this one running in parallel (changing only the parameters given to gLAB_linux). Therefore NUMCPUS can't be an environment variable.


• The only process accessing NUMCPUS is the bash script

EDIT2: After @Kamil answer, I add my proposal for reading from a file the number of CPUs

function waitpid {

   #Look if there is a file with new number of CPUs

   if [ -s "/tmp/numCPUs_$$.txt" ]

   then

     TMPVAR=$(awk '$1>0 {print "%d",$1} {exit}' "/tmp/numCPUs_$$.txt")

     if [ -n "${TMPVAR}" ]

     then

       NUMCPUS=${TMPVAR}

       echo "NUMCPUS=${TMPVAR}"

     fi

     rm -f "/tmp/numCPUs_$$.txt"

   fi



   #Gather the gLABs PID background processes (Maximum processes in 

   #background as number of CPUs)

   NUMPIDS=`jobs -p|awk 'END {print NR}'`

   #A while is set because there seems to be a bug in bash that makes 

   #sometimes the "wait -n" command

   #exit even if none of provided PIDs have finished. If this happens, 

   #the while loops forces the 

   #script to wait until one of the processes is truly finished

   while [ ${NUMPIDS} -ge ${NUMCPUS} ]

   do

     #Wait for gLAB processes to finish

     PIDS="`jobs -p|awk -v ORS=" " '{print}'`"

     wait -n ${PIDS} >/dev/null 2>/dev/null

     NUMPIDS=`jobs -p|awk 'END {print NR}'`

   done

}

The best would be to modify the bash script so it is aware that you change the value. Modifying an environment variable from inside a gdb session - that's just intrusive and mostly discards other developers work.

Below I use a file named /tmp/signal_num_cpus. If the file doesn't exist, the script uses the NUMCPUS value. If the file does exists, it reads it content and updates the number of NUMCPUS accordingly and then prints some notification that the numcpus was changed to the file. If the file does exists and does not contain a valid number (for example in predefined range or smth) it prints some error message into the file. The other side is notified that everything is ok or something bad happened

#!/bin/bash



is_not_number() { 

    (( $1 != $1 )) 2>/dev/null

}



# global variable to hold the number of cpus with a default value

NUMCPUS=${NUMCPUS:=5}

# this will ideally execute on each access to NUMCPUS variable

# depending on content

get_num_cpus() { 

   # I tell others that NUMCPUS is a global variable and i expect it here

   declare -g NUMCPUS

   # I will use this filename to communicate

   declare -r file="/tmp/signal_num_cpus"

   # If the file exists and is a fifo...

   if [ -p "$file" ]; then

       local tmp

       # get file contents

       tmp=$(<"$file")

       if [ -z "$tmp" ]; then

           #empty is ignored

           :;

       elif is_not_number "$tmp"; then

           echo "Error reading a number from $file" >&2

           echo "error: not a number, please give me a number!" > "$file"

       else

           # If it is ok, update the NUMCPUS value

           NUMCPUS=$tmp

           echo "ok $NUMCPUS" > "$file"  # this will block until other side starts reading

       fi

   fi

   # last but not least, let's output it

   echo "$NUMCPUS"

}



# code duplication is the worst (ok, sometimes except for databases frameworks)

get_num_bg_jobs() {

    jobs -p | wc -l

}



waitpid() {

   while 

         (( $(get_num_bg_jobs) >= $(get_num_cpus) ))

   do

         wait -n

   done

}



# rest of the script



NUMPCUS=10

for(...) #Loop for each day

do

   day=... #Set current day variable

   #Command to execute, put in background

   gLAB_linux -input "${day}folder/${day}.input" -output "${day)outfolder/${day}.output" &        

   #Wait for any process to finish if NUMCPUS number of processes are running in background

   waitpid 

done

And changing the value script could look like this:

#!/bin/bash



# shared context between scripts

declare -r file="/tmp/signal_num_cpus"



mkfifo "$file"



echo 1 > "$file" # this will block until other side will start reading



IFS= read -r line < "$file"



case "$line" in

ok*) 

     read _ numcpus <<<"$line"

     echo "the script changed the number of numcpus to $numcpus"

     ;;

*)

     echo "the script errored with $error"

     ;;

esac



rm "$file"

Marks:

• the correct way to define a function is func() { :; } Using function func { } is somthing taken from ksh and is supported as an extension. Use func() {}
  


• It is nice to use arithmetic expansion (( ... )) for number comparisons nad handling.


• Using backticks ` for command substitution $( ... ) is deprecated.

Chapter 7.1 of GNU Parallel 2018 covers how to change the number of threads to use while running https://zenodo.org/record/1146014

echo 50% > my_jobs

/usr/bin/time parallel -N0 --jobs my_jobs sleep 1 :::: num128 &

sleep 1

echo 0 > my_jobs

wait

So you simply put the argument for --jobs into my_jobs and GNU Parallel will read this after each completed job.