Is there a rule governing generation of RTX task ids OS_TID? It would be interesting to know:
a. if they are incremented sequentially, and b. if there is a max value for OS_TID
I am writing a lib which will cache some data in a buffer. That lib can be called from multiple tasks at the same time, so I want to keep separate buffers for each task. If OS_TID is deterministic then it will be a good differentiator for the per-task buffers.
Any other suggestions are welcome too.
Thanks
"Even Unix systems which are much more complex, make process ids sequential."
An interesting statement.
First off - you need a huge table to handle all PID of a modern Unix system. And that Unix normally have lookup tables because it needs to perform translations from the "sequential" values.
If you read the Linux man page for getpid(), you'll get an indication how hard that function has to perform the task - just because it caches the value to avoid additional system calls. And it's not trivial for a C library to keep track of the unique memory regions for different instances.
Next thing - your Unix system will not promise you what PID a new process will get. The promise is that it will get a PID which will fit in the pid_t data type. And that will not collide with the PID of any other, currently running, process. A new PID can be higher than the PID of the calling process. Or it can be lower.
Have you, by the way looked at the complexities involved in thread local storage? Lots of seemingly trivial functionality is way more complex when you do look under the hood.
"If you read the Linux man page for getpid(), you'll get an indication how hard that function has to perform the task"
But RTX is already doing it! OS_TID's implementation, as it stands, is exactly what we want. Only thing that's missing is an explicit commitment from RTX that this won't change in future. Unix makes that promise.
"Next thing - your Unix system will not promise you what PID a new process will get."
And I'm not asking for that.
I'm not in any way trying to trivialise the effort involved in building RTX. But holding back on solidifying an aspect of the system - generation of OS_TIDs - makes life harder for users. It feels even worse when the runtime itself relies on it. I gave the Unix example to argue that such a commitment can be made in practice, not on how to implement pids :)
"Have you, by the way looked at the complexities involved in thread local storage?"
The question doesn't ask for TLS. It asks whether generation of OS_TIDs is reliable.
No. Unix does not make any promise about the sequence of numbers in use. The only promise you have is the data type pid_t. And on my machine, that value is a 32-bit integer. So if you keep an array with the PID as index, then you need to be prepared for having 2^32 array entries. If we then assume that you can manage with 16 byte per possible PID, then you would "only" need a 64 GB large table.
The machine also happens to have an entry /proc/sys/kernel/pid_max with the value 32768. But that value is not a commitment to how Linux will work - it's just information how the specific implementation is currently configured. If I switch Linux kernel, I can't assume the value will be the same. I can't even assume that there will even be a pid_max file to read, or that the kernel will have any maximum value lower than what the pid_t data type can store.
The POSIX requirements is basically that a PID is a signed identifier not larger than the long data type.
So in a Unix system, you can't assume that you can get away with the PID as an index into an array because you have no promise that the possible range will be just a subset of what the pid_t data type can store. Not only that - you can't even assume that the PID you read out is the actual PID of the process. You can have translation layers that remaps the PID values to allow the use of sandboxes where a specific process will only be allowed to see a local mapping of a small subset of the full number of processes in use. This allows applications to run in firewalled containers without requiring virtual machines.
Why should RTX have to make a commitment how to actually allocate thread identifiers, when you use Unix as an example - and Unix does not make any such commitment? An additional commitment means limitations in what improvements that may be introduced at a later time. So commitments should only be made when it is really important to lock down some functionality.
If an OS promises you a nice integer data type of a size that works well with the programming language/hardware architecture, then that really is all commitment that is meaningful. Any additional commitment will be counter-productive because it will permanently add additional "must" clauses without giving any gain for the huge majority of the users of the API. If Keil finds a way where they can cut 100ns from a task switch, then it will be a big advantage to a lot of customers. If that 100ns task switch improvement clashes with any additional thread-id commitment, then lots of users will fail to get that 100ns gain because of the 0.1% users who could actually gain anything from additional commitments of how thread identifiers are allocated.
There is a reason why the software industry focuses on black box designs, where the implementation is allowed to change as long as the new implementation continues to fulfill the promised functionality/performance.
Just an addendum. The code to find a free thread ID is almost certainly deterministic. I don't think they are using a pseudorandom number generator when locating a suitable ID value. It's just that Keil has no reason to document how the code finds a non-colliding value within the possible number range. And Keil has no reason to document the possible number range in any other way than to document a data type for storing the values.
So in the end, neither *nix nor RTX relies on the ID:s being sequentially increasing. Just that they can handled by the intended data type, and that all active ID (you can see) are unique.
"Just an addendum. The code to find a free thread ID is almost certainly deterministic."
Yes. But it's not promised to remain same so users can't rely on it. Unix promise to keep way pids are generated same so they can be relied on.
Regarding your suggested approach...</p>
I thought you were suggesting having separate memory block for each task that was going to use the library. I am not suggesting this at all. My suggestion was to just make the library thread safe and re-entrant.
Cumbersome because of the semantics of such an API. If I understood correctly, first you need define per-task buffer and then have the calling task pass in a pointer to that buffer to the function that actually wants to keep track of per-task data. E.g. in a logging API which likes to buffer some data before flushing to file, the calling task will have to pass in a pointer to its buffer to log() function. That creates unwanted dependencies and tight coupling between caller (task) and callee (log() function) - a fertile ground for breeding spaghetti code. Moreover, buffering is responsibility of callee. Why should the caller worry about it?
No, you did not understand at all. The application does not need any knowledge of how this is implemented. (My guess is that you probably had no clue the Keil was doing this for the C runtime library. IF they changed the API for the C run time library my guess is that you would have had a clue about this and thought they were insane for changing the interface. Only the library uses this buffer. When the library needs to use one of these per tasks buffers, it uses the TID to get a pointer to the buffer.
// Also in RTX_CM_Lib.n void *__user_perthread_libspace (void) { /* Provide a separate libspace for each task. */ uint32_t idx; idx = (os_running != 0U) ? runtask_id () : 0U; if (idx == 0U) { /* RTX not running yet. */ return (&__libspace_start); } return ((void *)&std_libspace[idx-1]); }
this function above returns the unique buffer associated with the task that called the library. The API for the library does not change.
// There is nothing extra to pass in printf() { TaskUniqueBuffer = __user_perthread_libspace(); // This task the currently running task, which by definition needs to be the currently running task..., then returns a pointer that is unique for that task. The API does not change .... }
That being said, I want to re-iterate that I am not suggesting using this method at all. I thought you were looking for something like this. You had said....
"I am writing a lib which will cache some data in a buffer. That lib can be called from multiple tasks at the same time, so I want to keep separate buffers for each task.
and that is exactly what this does.
My interest was to provide knowledge on how some things worked. You had 2 questions about TID's so I answered them. I was not making any suggestions for how you use that knowledge. I pointed out that Keil used something similar to what you were suggesting (I was really not suggesting this) to make the C runtime library thread safe. Since it only required 1 variable and 1 function and no change to the API. My guess is that I would not be choosing to implement it this way, but that does not mean that in your specific situation this is not appropriate.
You can assign, store and compare thread ID:s if using the specified data type. So any code that stores the returned value in a variable of that type will be fine.
Maybe you could explain exactly what additional promise you think you get from Windows or Linux or BSD or OS X? They don't promise that you can predict what the next value will be. And they do not promise if they will use a subset or the full range the data type can store.
This seems to be what Windows promises: "Numeric identifier used to distinguish one process from another. ProcessIDs are valid from process creation time to process termination. Upon termination, that same numeric identifier can be applied to a new process. This means that you cannot use ProcessID alone to monitor a particular process. For example, an application could have a ProcessID of 7, and then fail. When a new process is started, the new process could be assigned ProcessID 7."
On Win32, the process ID is a uint32.
I haven't checked if it has changed on Win64, but the Win64 systems I have observed seems to limit itself to 16-bit values. But I have no way to prove if this is true, without trying to create 64k+ processes - and MS doesn't seem to give any such promise.
What Keil is doing, is quite similar to how thread local storage is implemented - which was the reason I did mention this concept earlier.
And this is because thread local storage tries to solve the similar problem that Keil wants to solve when they make their CRTL thread-safe.
The other common route to make library functions thread-safe is functions like gmtime_r(), gethostbyname_r() etc where a specific thread-safe variant of standard functions takes a pointer to a thread-local structure. Which obviously doesn't work for malloc() and all other standard C functions where you can't break the API.