HOW-TO use rt-app and workgen to emulate a workload ?

**** rt-app and workgen ****
rt-app/workgen is a tool that can be used to emulate a use case. Not only the
sleep and run pattern can be emulated but also the dependency between tasks
like accessing same critical resources, creating sequential wake up or syncing
the wake up of threads. The use case is described in a json like file which is
first parsed by workgen then rt-app.

workgen is a python script that will parse, check and update the json like file
in order to strictly match the json grammar before calling rt-app that will
execute the sequence described in it.

Even if the json protocol is used in rt-app, workgen enables some freedom
compared to stricter json grammar rules. For examples,
- You don't have to ensure uniqueness of key in json object, workgen will check
the file and ensure that each key is unique by appending an index if necessary.
The feature is quite useful when you describe a sequence made of same kind of
events e.g. a sequence of multiple sleep and run events.
- Some values can be omitted and workgen will add them when it parses the file
before starting the use case.

**** json file skeleton ****

The json file that describes a workload is made on 3 main objects: tasks,
resources and global objects. Only tasks object is mandatory; default value will
be used if global object is not defined and resources object is kept for
backward compatibility with old version of rt-app but it doesn't give any
benefit to declare it as the resources are now dynamically created by rt-app
when it parses the file.

**** global object ****

The global object defines parameters for the whole use case:

* duration : Integer. Duration of the use case in seconds. All the threads will
be killed once the duration has elapsed. if -1 has been set, the use case will
run indefinitly until all threads kill themselves (as an example if a finite
number of loop has been defined in their running pattern) or if a signal
is received to stop the use case.

* calibration : String or Integer: A String defines the CPU that will be used
to calibrate the ns per loop value. "CPU0" is the default value (see run event
section for details about ns per loop value). A integer skips the calibration
step and uses the integer value as ns per loop.

* default_policy : String. Default scheduling policy of threads. Default
value is SCHED_OTHER.

* pi_enabled: Boolean. Enable the priority inheritance of mutex. Default value
is False.

* lock_pages : Boolean. Lock the mem page in RAM. Locking the page in RAM
ensures that your RT thread will not be stalled until a page is moved from swap
to RAM.  The lock of the page is only possible for non CFS tasks. Default value
is True.

* logdir : String. Path to store the various log files. The default path is
the current directory (./). Directory must be already existing.

* log_basename : String. Prefix used for all log files of the use case.
"rt-app-" is used by default.

* log_size : String or Integer. A Integer defines a fix size in MB of the
temporary buffer (size per thread) that will be used to store the log data
before saving them in a file. This temporary buffer is used as a circular
buffer so the oldest data will be lost in case of overflow. A string is used
to set a predefined behavior:
  - "file" will be used to store the log data directly in the file without
	using a temporary buffer.
  - "Disable" will disable the log mechanism.
  - "Auto" will let rt-app compute the buffer size to not overflow the latter
	during the use case.
The use of a temporary buffer prevents the threads of unexpected wait during
io access. The "Auto" mode is not implemented yet and fallback to "file" mode
for the moment.

* ftrace: String. If not "none", rt-app logs in ftrace the events corresponding
to the requested categories, which can be specified as a comma separated list of
names.

NOTE: the usage of a "boolean" value for this attribute is DEPRECATED and will
be removed soon. For the time being we map "True/False" to enable/disable all
the supported category names.

The category names currently supported are:
 - main  : events generated by the main rt-app task
 - task  : events generated by a task
 - run   : events generated by the run of a task
 - loop  : events generated by each loop of a task
 - stats : events reporting statistics on task's activation

Default value is "none".

* gnuplot : Boolean. If True, it will create a gnu plot compatible file for
each threads (see gnuplot section for more details). Default value is False.

"io_device" : Text. Path to the file which will be written to create IO-bounded
busy loop. Specify it carefully since it might damage the specified file.
Default value is "/dev/null".

"mem_buffer_size" : Integer. The size of per-thread memory buffer in byte being
used to create IO-bounded and memory-bounded busy loop. Default value is
4194304(4MB).

* cumulative_slack : Boolean. Accumulate slack (see below) measured during
  successive timer events in a phase. Default value is False (time between the
  end of last event and the end of the phase).

*** default global object:
	"global" : {
		"duration" : -1,
		"calibration" : "CPU0",
		"default_policy" : "SCHED_OTHER",
		"pi_enabled" : false,
		"lock_pages" : false,
		"logdir" : "./",
		"log_size" : "file",
		"log_basename" : "rt-app",
		"ftrace" : "none",
		"gnuplot" : false,
		"io_device" : "/dev/null"
		"mem_buffer_size" : 4194304,
		"cumulative_slack" : false
	}

**** tasks object ****

 The tasks object is made of sub-objects that describe threads. No other kind
 of object than thread object is allowed. The key value of each object will be
 used as thread's name.

"tasks" : {
	"thread_name1" : {
		...
	},
	"thread_name2" : {
		...
	},
	"thread_name3" : {
		...
	},
	...
}

*** thread object ***

Thread object describes the behavior of one kind of thread which means that
several threads can be created with the same object (see instance parameter
below).

* instance : Integer. Define the number of threads that will be created with
the properties of this thread object. Default value is 1. A value of 0 will
create the structures of the task but will not create a pthread.

* delay: Integer. Initial delay before a thread starts execution. The unit
is usec.

* phases: Object. The phases object describes the behavior of the thread. This
  behavior can be split in several distinct phases with their own events and
  properties. See phase object parameter below.

*** phase object ***

If there is only 1 phase, the sequence of events can be directly put in the
thread object instead of creating a phases object. As an example, the 2 objects
below are equals:

"task" : {
	"thread1" : {
		"phases" : {
			"phase1" : {
				"run" : 10,
				"sleep" : 10
			}
		}
	}
}

"task" : {
	"thread1" : {
		"run" : 10,
		"sleep" : 10
	}
}

*** thread and phase object properties ***

Several properties can be set at thread and/or phase levels. All these
properties are optional and default value will be used if nothing is defined.

* loop: Integer. Define the number of times the parent object must be run.  The
  parent object can be a phase or a thread. For phase object, the loop defines
the number of time the phase will be executed before starting the next phase.
For thread object, the loop defines the number of times that the complete
"phases" object will be executed. The default value is -1 for thread object and
1 for phases.

*** scheduling policy

The scheduling policy can be set at thread and phase levels. A default policy
is set after the thread creation if nothing has been defined. Then, the
scheduling properties can be changed at the beginning of each phase if it has
been defined in the phase object. The behavior is quite similar to an event as
a update happens only if one of the scheduling parameters has been set
otherwise the scheduling parameters will stay unchanged

* policy : String. Define the scheduling policy of the thread. default_policy
is used if not defined. Accepted value are:
  - "SCHED_OTHER"
  - "SCHED_IDLE"
  - "SCHED_RR"
  - "SCHED_FIFO"
  - "SCHED_DEADLINE"

* priority : Integer. Define the scheduling priority of the thread. The value
must be aligned with the range allowed by the policy. The default priority is
0 for sched_other and sched_deadline class and 10 for rt classes

* dl-runtime : Integer: Define the runtime budget for deadline scheduling class.
Default value is 0. The unit is usec.  For backward compatibility, the
key "runtime" will also be checked in thread object but special must be taken
as this can conflict with runtime event.

Since Linux 6.12 this parameter also applies to the SCHED_OTHER scheduling
class. It is used to request a custom slice length for the thread.

* dl-period : Integer. Define the period duration for deadline scheduling class.
Default value is runtime. The unit is usec. For backward compatibility, the
key "period" will also be checked in thread object.

* dl-deadline : Integer. Define the deadline parameter for deadline scheduling
class. Default value is period. The unit is usec.  For backward compatibility,
the key "deadline" will also be checked in thread object.

*** CPUs affinity

* cpus: Array of Integer. Define the CPU affinity of the thread. Default
value is all CPUs of the system. An example :  "cpus" : [0, 2, 3]

"cpus" can be specified at task level or phase level. As an example, below
creates two threads. One thread will run with affinity of CPU 2 and 3. The
second task will run first phase with affinity to CPU 0 and 1, second phase with
affinity to CPU 2, and the third phase with affinity to CPU 4, 5, and 6 (it will
inherit the affinity from the task level):

"tasks" : {
	"thread1" : {
		"cpus" : [2,3],
		"phases" : {
			"phase1" : {
				"run" : 10,
				"sleep" : 10
			}
		}
	}
	"thread2" : {
		"cpus" : [4,5,6],
		"phases" : {
			"phase1" : {
				"cpus" : [0,1],
				"run" : 10,
				"sleep" : 10
			}
			"phase2" : {
				"cpus" : [2],
				"run" : 10,
				"sleep" : 10
			}
			"phase3" : {
				"run" : 10,
				"sleep" : 10
			}
		}
	}
}

*** Utilization Clamping

util_min: Integer. Can be specified at task level or phase level. Sets the min
utilization value the task can have; which helps boosting the task from
scheduler point of view.

util_max: Integer. Can be specified at task level or phase level. Sets the max
utilization value the task can have; which helps capping the task from
scheduler point of view.

*** NUMA memory binding

* nodes_membind: Array of Integer. Define the NUMA binding of the thread.
Default value is all NUMA nodes of the system. 
An example : "nodes_membind" : [0, 2, 3]

"nodes_membind" Can be specified at task level or phase level. 
As an example, below creates two threads.
One thread will run with memory binding to nodes 2 and 3. 
The second task will run first phase with memory binding to nodes 0 and 1, 
second phase with memory binding to node 2.
Please note, that we follow an "event based policy", which means that 
rt-app changes memory binding when there is a "nodes_membind" event 
and don't do anything otherwise.

"tasks" : {
    "thread1" : {
        "nodes_membind" : [2,3],
        "phases" : {
            "phase1" : {
                "run" : 10,
                "sleep" : 10
            }
        }
    }
    "thread2" : {
        "phases" : {
            "phase1" : {
                "nodes_membind" : [0,1],
                "run" : 10,
                "sleep" : 10
            }
            "phase2" : {
                "nodes_membind" : [2],
                "run" : 10,
                "sleep" : 10
            }
        }
    }
}

*** taskgroups

* taskgroup: String. Can be specified at task level or phase level. Currently
only SCHED_OTHER and SCHED_IDLE tasks are supported to contain taskgroup data.
Tasks with a different policy and without taskgroup data can coexist in the
setup. An empty taskgroup ("") is allowed and is interpreted as if there is
no taskgroup data given. Pre-existing parts of a taskgroup are preserved.
They are not removed during rt-app teardown.
The default build supports up to 32 different taskgroups. In case a
configuration exceeds this limit the error message '[rt-app] <error> [tg] #
taskgroups exceeds max # taskgroups [32]' is issued before rt-app exits. The
number of supported taskgroups can be increased by changing 'const static
unsigned int max_nbr_tgs = 32' at compile time.

*** events ***

events are simple action that will be performed by the thread or on the
thread. They have to be listed by execution order.

* run : Integer. Emulate the execution of a load. The duration is defined in
usec but the run event will effectively run a number of time a loop that waste
cpu cycles. When the run event is executed with parameter t, the workload
executed will be n loops of the calibration workload (pLoad in ns):
	n := floor (t * 1000 / pLoad)
so, it might be running for a shorter amount of time.

This way of working enables to emulate a fixed workload with a duration that
will vary with the frequency or the compute capacity of the CPU.


* runtime : Integer.  The duration is define in usec.  Similar to the
run event, it emulates the execution of a load.  Unlike run, runtime
runs for a specific amount of time irrespective of the compute
capacity of the CPU or the frequency.

* sleep : Integer. Emulate the sleep of a task. The duration is defined in
usec.

* mem : Integer. Emulate the memory write operation. The value defines the size
in byte to be written into the memory buffer. The size of the memory buffer is
defined by "mem_buffer_size" in "global" object.

* iorun : Integer. Emulate the IO write operation. The value defined the size
in byte to be write into the IO device specified by "io_device" in "global"
object.

* timer : Object. Emulate the wake up of the thread by a timer. Timer differs
from sleep event by the start time of the timer duration. Sleep duration starts
at the beginning of the sleep event whereas timer duration starts at the end of
the last use of the timer. So Timer event are immunized against preemption,
frequency scaling and computing capacity of a CPU. The initial starting time of
the timer is set during the 1st use of the latter.

          |<------19---------->|<------19---------->|<------19---------->|
task A ...|run 5|timer 19      |------run 5|timer 19|run 5|timer 19      |
task B                      |run 10  |


As an example to point out the difference between sleep and timer, let consider
a task A that run 5 and then sleep 10. The period of task A should be 15.
Let's add a task B that runs 5 and use a timer to wakes up every 19.

         |<------15------>|<------15------>|<------19---------->|<------15------>|<------16------->|<------19---------->|<------19---------->|<------19---------->|<------19---------->|<------19---------->|
taskA ...|run 5|sleep 10  |run 5|sleep 10  |----run 5|sleep 10  |run 5|sleep 10  |-run 5|sleep 10  |----run 5|sleep 10  |----run 5|sleep 10  |----run 5|sleep 10  |----run 5|sleep 10  |----run 5|sleep 10  |
taskB ...|------run 5|timer 19|--run 5|timer 19    |run 5|timer 19      |run 5|timer 19      |run 5|timer 19      |run 5|timer 19      |run 5|timer 19      |run 5|timer 19      |run 5|timer 19      |
         |<------19---------->|<------19---------->|<------19---------->|<------19---------->|<------19---------->|<------19---------->|<------19---------->|<------19---------->|<------19---------->|

We can see that task B period stays to 19 even if the run event is delayed
because of scheduling preemption whereas the period of task A starts at 15 but
increases to 19 because of the scheduling delay.

"unique" timer device: When a thread that uses a timer is instantiated, the
timer will be shared across the thread instance which disturbs the original
sequence. In order to have 1 timer per instance, you can use the "unique"
prefix so a new timer will be created for each instance. Uniqueness
applies to the thread boundary which means that using the same unique
name in the sequence of a thread will refer to the same timer like the example
below:
	"phases" : {
		"light" : {
			"loop" : 10,
			"run" :   1000,
			"timer" : { "ref" : "unique", "period" : 30000 }
		},
		"heavy" : {
			"loop" : 10,
			"run" :   4000,
			"timer" : { "ref" : "unique", "period" : 30000 }
		}
	}

If you want to refer to different timer you must use different name like
below:
	"phases" : {
		"light" : {
			"loop" : 10,
			"run" :   1000,
			"timer" : { "ref" : "uniqueA", "period" : 30000 }
		},
		"heavy" : {
			"loop" : 10,
			"run" :   4000,
			"timer" : { "ref" : "uniqueB", "period" : 400000 }
		}
	}

Timers can work with a "relative" or an "absolute" reference. By default they
work in "relative" mode, but this mode can also be explicitly specified as the
following:

	"phases" : {
		"phase0" : {
			"loop" : 10,
			"run0" :  10000,
			"timer0" : { "ref" : "unique", "period" : 20000, "mode" : "relative" },
	}

"relative" mode means that the reference for setting the next timer event is
relative to the end of the current phase. This in turn means that if, for some
reason (i.e., clock frequency was too low), events in a certain phase took too
long to execute and the timer of that phase couldn't actually fire at all, the
next phase won't be affected. For example:

  +---- +     +-----+     +-------------------+-----+     +---
  |r0   |     |r0   |     |r0                 |r0   |     |r0
  |     |     |     |     |                   |     |     |
  o-----------o-----------o-------------------o-----------o------->
  0    10    20    30    40    50    60    70    80   100   120
              ^           ^           ^                   ^
              |           |           | MISS!             |
              +           +           +                   +
            Timer0      Timer0       Timer0              Timer0

In this example character "o" denotes when phases finish/start. Third
activation of Timer0 is missed, since r0 executed for more that 20ms. However
the next phase is not affected as Timer0 was set considering the instant of
time when the misbehaving r0 finished executing.

"absolute" mode is specified as the following:

	"phases" : {
		"phase0" : {
			"loop" : 10,
			"run0" :  10000,
			"timer0" : { "ref" : "unique", "period" : 20000, "mode" : "absolute" },
	}

"absolute" mode means that the reference for setting the next timer event is
fixed and always consider the starting time of the first phase. This means that
if, for some reason (i.e., clock frequency was too low), events in a certain
phase took too long to execute and the timer of that phase couldn't actually
fire at all, the next phase (and potentially other subsequent phases) _will_ be
affected. For example, considering again the example above:

  +---- +     +-----+     +-------------------+-----+-----+   +---
  |r0   |     |r0   |     |r0                 |r0   |r0   |   |r0
  |     |     |     |     |                   |     |     |   |
  o-----------o-----------o-------------------o-----o---------o---->
  0    10    20    30    40    50    60    70    80   100   120
              ^           ^           ^           ^           ^
              |           |           | MISS!     | MISS!     |
              +           +           +           +           +
            Timer0      Timer0       Timer0      Timer0      Timer0

Third activation of Timer0 is missed, since r0 executed for more that 20ms.
Even if 4th activation of r0 executes for 10ms (as specified in the
configuration), 4th Timer0 is still missed because the reference didn't change.
In this example 5th activation of r0 then managed to recover, but in general it
depends on how badly a certain phase misbehaves.

* lock : String. Lock the mutex defined by the string value.

* unlock : String. Unlock the mutex defined by the string value.

* wait : Object {"ref" : String, "mutex" : String }. Block the calling thread
until another thread sends a wake up signal to the resource defined by "ref".
The mutex defined by "mutex" is used during the block sequence: See
pthread_cond_wait() for more details about the sequence (especially regarding
the use of the mutex).

* signal : String. Send a wake up signal to the resource defined by the
string. The 1st thread in the wait list will be wokn up. See
pthread_cond_signal() for more details.

* broad : String. Send a wake up signal to the resource defined by the
string. All threads that are blocked on the resource wil wake up. See
pthread_cond_broadcast() for more details.

* sync : Object {"ref" : String, "mutex" : String }. Atomically wakes
up a blocked thread and then blocks the calling thread on the condition.
taskA ...|run 5|wait|------------|run 5| wait |----------------
taskB ...-------------------|sync|--------

The sync event "sync" : {"ref" : "CondA", "mutex" : "mutexA" } generates the
following sequence:
{
	"lock" : "mutexA",
	"signal" : "CondA",
	"wait" : { "ref" : "condA", "mutex" : "mutexA" },
	"unlock" : "mutexA"
}

* barrier : String. Used as at least a pair where the name must match.
Any number of matching uses will cause all threads hitting the barrier event
to wait for a signal. The number of users is recorded, so that when the last user
hits the barrier event, that thread will broadcast and continue to the next
step. This is conceptually exactly the same as a pthread_barrier_wait operation
however, using a pthread_barrier would impose some strict conditions on usage
around thread cleanups - primarily that you cannot cancel an in-progress barrier
operation which would mean that we have to restrict cleanup to only be possible
at the end of a loop cycle (i.e. all phases are complete). This would be too
restrictive for most uses so here we use an alternative. Finally, barrier can't
be used with fork event because we must know how many threads will wait which
is not possible with fork events.

In this implementation, the barrier event manages its own mutex and uses a
variable shared between users to track waiting tasks, protected by the mutex.

You must use a unique name for each sync event since the number of users is
taken from the number of references to the name in the input json. i.e. each
name must represent a single sync point, and be shared amongst all threads
which wish to synchronize at that point.

The barrier event "barrier" : "SyncPointA"
generates the following sequence:
{
    "lock" : "SyncPointA" (internal mutex),
    If the shared variable is 0:
      "signal" : "SyncPointA" (internal condvar),
    Else:
      decrement the shared variable,
      "wait" : { "ref" : "SyncPointA" (internal condvar),
                 "mutex" : "SyncPointA" (internal mutex) },
      increment the shared variable,
    "unlock" : "SyncPointA" (internal mutex)
}

* suspend : String. Block the calling thread until another thread wakes it up
with resume. String is ignored.

* resume : String. Wake up the thread defined by the string.

taskA ...|run 5|suspend |----------------|run 5|suspend |----------------
taskB ...-------------------|resume taskA|run 10    |--------------------

* yield: String. Calls pthread_yield(), freeing the CPU for other tasks. This has a
special meaning for SCHED_DEADLINE tasks. String can be empty.

* fork: String. Instead of creating all tasks at start time, you can fork one
at any point of time using this event. The name of the forked task must match
one of the defined tasks.

**** Trace and Log ****

Some traces and log hooks have been added to ease the debug and monitor various
metrics of the use cases

*** Trace ***

A trace can be inserted into ftrace buffer for each event in order to sync
kernel events like sched_switch with the use case's sequence.

A simple example of ftrace log:

          rt-app-22165 [002] 105512.816244: print:    tracking_mark_write: rtapp_main: event=start
    small_task-0-22166 [005] 105512.839499: print:    tracking_mark_write: rtapp_main: event=clock_ref data=105512715874
    small_task-0-22166 [005] 105512.851733: print:    tracking_mark_write: rtapp_task: event=start
    small_task-0-22166 [001] 105512.852153: print:    tracking_mark_write: rtapp_loop: event=start thread_loop=0 phase=0 phase_loop=0
    small_task-0-22166 [001] 105512.852164: print:    tracking_mark_write: rtapp_event: id=0 type=6 desc=run
    small_task-0-22166 [001] 105512.853565: print:    tracking_mark_write: rtapp_event: id=1 type=10 desc=timer:small_task
    small_task-0-22166 [001] 105512.853585: print:    tracking_mark_write: rtapp_loop: event=end thread_loop=0 phase=0 phase_loop=0
    small_task-0-22166 [001] 105512.853592: print:    tracking_mark_write: rtapp_loop: event=start thread_loop=0 phase=0 phase_loop=1
    small_task-0-22166 [001] 105512.853600: print:    tracking_mark_write: rtapp_event: id=0 type=6 desc=run
    small_task-0-22166 [001] 105512.854999: print:    tracking_mark_write: rtapp_event: id=1 type=10 desc=timer:small_task
      big_task-1-22167 [000] 105512.855414: print:    tracking_mark_write: rtapp_task: event=start
      big_task-1-22167 [001] 105512.855723: print:    tracking_mark_write: rtapp_loop: event=start thread_loop=0 phase=0 phase_loop=0
      big_task-1-22167 [001] 105512.855732: print:    tracking_mark_write: rtapp_event: id=0 type=6 desc=run
      big_task-1-22167 [001] 105512.862754: print:    tracking_mark_write: rtapp_event: id=1 type=10 desc=timer:big_task
      big_task-1-22167 [001] 105512.862772: print:    tracking_mark_write: rtapp_loop: event=end thread_loop=0 phase=0 phase_loop=0
      big_task-1-22167 [001] 105512.862779: print:    tracking_mark_write: rtapp_loop: event=start thread_loop=0 phase=0 phase_loop=1
      big_task-1-22167 [001] 105512.862786: print:    tracking_mark_write: rtapp_event: id=0 type=6 desc=run
    small_task-0-22166 [001] 105512.869674: print:    tracking_mark_write: rtapp_loop: event=end thread_loop=0 phase=0 phase_loop=1
    [...]
    small_task-0-22166 [001] 105515.829685: print:    tracking_mark_write: rtapp_task: event=end
    [...]
      big_task-1-22167 [001] 105515.838870: print:    tracking_mark_write: rtapp_task: event=end
          rt-app-22165 [002] 105515.839357: print:    tracking_mark_write: rtapp_main: event=end


The rt-app task index is now encoded in the task name, which is reported at
the beginning of each trace event following this template:

    <task_name>-<task_id>[-<fork_id>]-<pid>

Where:
- task_name : the name of a task as specified in the task definition
- task_id   : the ID of the task created by rt-app and matching its
              position into the JSON file
- fork_id   : the ID of the nth forked version of a task, this filed is optional
              and it's there only for forked tasks.
- pid       : the Linux assigned PID for this task

The generated events are of these main categories:

* Main task events, for example:

  - rtapp_main: event=start
  - rtapp_main: event=clock_ref data=105512715874
  - rtapp_main: event=end

  Reporting the start and end of the main thread.

  The additional "clock_ref" event back annotates in the trace the time
  reference used by the performance log entries generated by each task.

* Workload tasks events, for example:

  - rtapp_task: event=start
  - rtapp_task: event=end

  Reporting the start end end of worker tasks.

* Workload's loops events, for example:

  - rtapp_loop: event=start thread_loop=0 phase=0 phase_loop=0
  - rtapp_loop: event=end thread_loop=0 phase=0 phase_loop=0
  
  Reporting the start and end of workload's phases and loops.

* Workload's events, for example:

  - rtapp_event: id=0 type=6 desc=run
  - rtapp_event: id=1 type=10 desc=timer:small_task

  Reporting the execution of the specified event type in the current phase/loop.

* Workload's stats, for example:

  - rtapp_stats: period=3948 run=1378 wu_lat=95 slack=2454 c_run=16000 c_period=1600

  Reporting the major performance metrics measured after each workload
  activation.

*** Log and gnuplot ***

You can log some metrics for each phase that is executed by a thread. These
metrics are:
- perf: fixed amount of work performed during a phase (c_duration/calibration)
  (see events/run above) [number of 1000 loops, each pLoad]
- run: time spent by the thread to execute the run events [us]
- period: duration to execute the complete phase [us]
- start/end : absolute start and end time of a phase. Same time base is used in
  ftrace
- rel_st: start time of a phase relatively to the beg of the use case
- slack: if global option "cumulative_slack" (see above) is false, time between
  the end of last event and the end of the phase, e.g.

  taskA ...|-- run5 --|- sleep5 -|-- run5--|..timer20.|-- run5 --|- sleep5 -|-- run6 --|.timer20.|
            <--------------- period 20 --------------> <--------------- period 20 -------------->
                                            <-slack5->                                  <slack4->

  it can also be negative if the execution of a phases' events overshoots the
  current period, e.g.

  taskA ...|-- run5 --|- sleep5 -|------- run30 ------xxxxxxxxxx|
            <--------------- period 20 -------------->
                                                       <slack-5>

  if global option "cumulative_slack" is true, all the intermediate slacks of a
  phase with multiple timers are accumulated and reported when the phase
  completes

- c_duration: sum of the configured duration of run/runtime events [us]
- c_period: sum of the timer(s) period(s) [us]
- wu_lat: sum of wakeup latencies after timer events [us]

Below is an extract of a log:

# Policy : SCHED_OTHER priority : 0
#idx     perf      run   period           start             end          rel_st      slack c_duration   c_period     wu_lat
   0    92164    19935    98965    504549567051    504549666016            2443      78701      20000     100000        266
   0    92164    19408    99952    504549666063    504549766015          101455      80217      20000     100000        265
   0    92164    19428    99952    504549766062    504549866014          201454      80199      20000     100000        264
   0    92164    19438    99955    504549866060    504549966015          301452      80190      20000     100000        265
   0    92164    19446    99952    504549966061    504550066013          401453      80093      20000     100000        264
   0    92164    19415    99953    504550066060    504550166013          501452      80215      20000     100000        263
   0    92164    19388    99954    504550166059    504550266013          601451      80242      20000     100000        264
   0    92164    19444    99956    504550266060    504550366015          701452      80185      20000     100000        265

Some gnuplot files are also created to generate charts based on the log files
for each thread and for each kind of metrics. The format of the chart that
will be generated by gnuplot is eps (text art has been used for the chart
below)

                  Measured thread0 Loop stats
                                                Load [nb loop]
  120000 ++--+----+---+----+---+---+----+---+----+--++ 506000     load  ******
         +   +    +   +    +   +   +    +   +    +   +             run  ######
         |$$ :    :   :    :   $$$ :    :   :    :   |          period  $$$$$$
  110000 ++.$$$........$$$$...$...$..........$$$.$$$$+
         |   : $$$$$$$$    $$ $:   $ $$$$$$$$   $:   $
         |   :    :   :    : $ :   :$   :   :    :  ++ 504000
         |   :    :   :    :   :   :    :   :    :   |
  100000 ++.........................................++
         |   :    :   :    :   :   :    :   :    :   |
         |   :    :   :    :   :   :    :   :    :   |
   90000 ++.........................................++
         |   :    :   :    :   :   :    :   :    :  ++ 502000
         |   :    :   :    :   :   :    :   :    :   |
   80000 ++.........................................++
         |   :    :   :    :   :   :    :   :    :   |
         |   :    :   :    :   :   :    :   :    :   |
   70000 +******************************************** 500000
         |   :    :   :    :   :   :    :   :    :   |
         |   :    :   :    :   :   :    :   :    :   |
         |   :    :   :    :   :   :    :   :    :   |
   60000 ++.........................................++
         |   :    :   :    :   :   :    :   :    :   |
         |   :    :   :    :   :   :    :   :    :  ++ 498000
   50000 ++.........................................++
         |   :    :   :    :   :   :    :   :    :   |
         |   :    :   :    :   :   :    :   :    :   |
   40000 ++.........................................++
         |   :    :   :    :   :   :    :   :    :  ++ 496000
         |#  :    :   :    :   :   :    :   :    :   |
         | # :    :   :    :   ### :    :   :    :   |
   30000 ++.###.###....####...#...#..........###.####+
         |   : #  :####    ### :   # ########   #:   #
         +   +    +   +    +   +   +#   +   +    +   +
   20000 ++--+----+---+----+---+---+----+---+----+--++ 494000
       17560556057560556057560656065606756065606756075607
                    Loop start time [msec]


