Multi-threaded scheduling has always been in the interest of developers to increase application performance and maximize resource usage.. This guide will introduce you to the basics of Bash multithreaded encoding.
What is it multiprocess programming?
A picture is worth a thousand words, and this is valid when it comes to showing the difference between programming a single (1) thread and multi-thread scheduling (> 1) in bash:
sleep 1 sleep 1 & sleep 1
Our first single line mini script or multi-threaded scheduling setup couldn't have been simpler; on the front line, we slept a second using the sleep 1 command. Regarding the user, a single thread was executing a single one-second dream.
On the second line, we have two one-second suspend commands. We join them using a & separator, which not only acts as a separator between the two sleep commands, but also as a Bash prompt to start the first command in a background thread.
Regularly, one would end a command using a semicolon (;). In doing so, the command would be executed and only then would it be passed to the next command that appears after the semicolon. As an example, running sleep 1; sleep 1 it would take just over two seconds: exactly one second for the first command, one second for the second and a small amount of system overhead for each of the two commands.
Despite this, instead of ending a command with a semicolon, you can use other command terminators that Bash recognizes as &, && and ||. the && the syntax has nothing to do with multithreaded programming, just do this; proceed with the execution of the second command only if the first command was successful. the || It is the opposite of && and it will execute the second command only if the first command failed.
Going back to multithreaded programming, using & since our command terminator will start a background procedure by executing the command that precedes it. It then immediately proceeds to run the following command in the current shell while letting the background procedure (hilo) run by itself.
In the command output we can see that a procedure is starting in the background (as indicated [1] 445317 where 445317 is the procedure ID or PID of the newly started background procedure, and [1] it is stated that this is our first background procedure) and subsequently terminated (as indicated [1]+ Done sleep 1).
If you want to see an additional example of background process handling, see our post Bash Automation and Scripting Basics (Part 3). At the same time, Bash procedure termination tricks may be of interest.
Now let's show that we are indeed running two sleep processes at the same time:
time sleep 1; echo 'done'
time $(sleep 1 & sleep 1); echo 'done'
Here we start our sleep low procedure time and we can see how our single threaded command ran for exactly 1.003 seconds before the command line prompt was returned.
Despite this, in the second example, took about the same time (1,005 seconds) even though we were running two periods (and processes) suspension, even if not consecutively. Again we use a background procedure for the first suspend command, leading to an execution (semi) parallel, In other words, multiproceso.
We also use a sublayer container ($(...)) around our two dream commands to combine them under time. How can we see our done the output is shown in 1.005 seconds and, because, both sleep 1 the commands must have been executed simultaneously. Interesting is the very small increase in total processing time (0,002 seconds) which can be easily explained by the time it takes to start a sublayer and the time it takes to start a background procedure.
Managing multi-threaded processes (and in the background)
In bash, multi-threaded encoding will regularly involve background threads from a one-line main script or a full bash script. In essence, one can think of multi-threaded encoding in Bash as starting multiple background threads. When one starts coding using multiple threads, it quickly becomes clear that such threads will generally require some handling. As an example, take the fictitious example where we start five concurrent periods (and processes) of sleep in a Bash script;
#!/bin/bash
sleep 10 &
sleep 600 &
sleep 1200 &
sleep 1800 &
sleep 3600 &
When we start the script (after making it executable using chmod +x rest.sh), we don't see any way out!! Even if we run jobs (the command that displays the background jobs in progress), no way out. Why?
The reason is that the shell that was used to start this script (In other words, the current shell) is not the same shell (nor the same thread; to begin to think in terms of sublayers as threads in and of themselves) who executed the royal dream. commands or placed them in the background. It was rather the (sub) shell that started when ./rest.sh was executed.
Let's change our script by adding jobs inside the script. This will ensure that jobs runs from within the (sub) shell where relevant, the same in which the periods began (and processes) sleep.
This time we can see the list of background processes that are starting thanks to the jobs command at the end of the script. We can also see their PIDs (procedure identifiers). These PIDs are very important when it comes to handling and managing background processes.
Another way to get the Background Procedure Handle is to query it immediately after placing a program / background procedure:
#!/bin/bash
sleep 10 &
echo ${!}
sleep 600 &
echo ${!}
sleep 1200 &
echo ${!}
sleep 1800 &
echo ${!}
sleep 3600 &
echo ${!}
Similar to our jobs command (with new PIDs now when we restart our rest.sh script), thanks to the bash ${!} When the variable is repeated, now we will see that all five PIDs are displayed almost immediately after the script starts: multiple suspend processes were put into background threads one after another.
The wait Command
Once we have started our background processes, we have nothing else to do but wait for them to finish. Despite this, when each background procedure is executing a complex subtask, and we need the main script (which started the background processes) resume execution when one or more of the background processes terminate, we need additional code to handle this.
Let's expand our script now with the wait command to handle our background threads:
#!/bin/bash
sleep 10 &
T1=${!}
sleep 600 &
T2=${!}
sleep 1200 &
T3 = ${!}
sleep 1800 &
T4 = ${!}
sleep 3600 &
T5 = ${!}
echo "This script started 5 background threads which are currently executing with PID's ${T1}, ${T2}, ${T3}, ${T4}, ${T5}."
wait ${T1}
echo "Thread 1 (sleep 10) with PID ${T1} has finished!"
wait ${T2}
echo "Thread 2 (sleep 600) with PID ${T2} has finished!"
Here we extend our script with two wait commands that wait for the PID attached to the first and second threads to finish. After 10 seconds, our first thread exists and we are notified of it. Step by Step, this script will do the following: start five threads almost at the same time (even though the start of the threads itself is still sequential and not parallel) where each of the five sleepwill run in parallel.
After, the main script reports (sequentially) over the created thread and, subsequently, wait for the procedure id of the first thread to finish. When that happens, will report sequentially on the end of the first thread and start waiting for the second thread to finish, etc.
Using bash idioms &, ${!} and the wait The command gives us great flexibility when it comes to running multiple threads in parallel (as background threads) in bash.
Ending
In this post, We explore the basics of Bash multithreaded scripting. Introduced the background procedure operator (&) using some easy to follow examples showing both single threaded and multi threaded sleep commands. Next, we discuss how to handle background processes through commonly used bash idioms. ${!} and wait. We also explore the jobs command to view thread execution / background processes.
If you enjoyed reading this post, take a look at our post Bash Procedure Termination Hacks.
