MapuSoft’s OS Abstraction Layer (OSAL)
Nowadays embedded programmers are including OS Abstraction in their next generation designs to prevent code from getting locked into the current OS. They are anticipating future changes to the underlying platform due to OS version upgrades, changing of the OS due to hardware changes or changes in market demand or popularity of the original OS. However, these internal OS Abstraction designs will include limited OS functions and the wrapper-based design will introduce performance issues all while the application will remain closely tied to the original OS.
MapuSoft’s OS Abstractor product is a commercial-grade OS Abstraction Layer (OSAL) designed without a layered implementation in order to provide better performance at lower cost when compared to an in-house OS Abstraction solution. OS Abstractor eliminates the wrapper approach by performing many OS functions without using the underlying OS. For functions that need to be wrapped, it eliminates the run-time impact by allowing the functions to re-use objects and resources. Its implementation is based on a C/C++ source-level virtualization technology custom generated and optimized for each specific application and target platform. OS Abstractor frees your application from getting locked to the OS so that when you upgrade or change the OS there will be no impact to your application. OS Abstractor is offered in full source code format with a royalty-free licensing model.
Developing applications with OS Abstractor is especially important if your product is anticipated to have a long life. Operating systems are often up-graded and will eventually need to be replaced when hardware reaches its end-of-life. OS Abstractor eliminates time consuming manual updates to the application when there are changes to the underlying OS. In addition, applications developed with OS Abstractor are easily transitioned to new operating systems and hardware because your software will support multiple operating systems and versions from the start.
Using OS Abstractor in Your Application
OS Abstractor was designed for embedded application development and provides an important level of abstraction to your application. It also provides performance enhancement features like re-using kernel resources and allowing optimization of the abstraction code that is specific to your application. OS Abstractor also provides software design abstraction without affecting your application’s performance.
Here are a few additional advantages to developing applications with OS Abstractor:
- It protects your software investment
- Provides advanced development features and capable of adding functionality to an existing OS
- Adds API flexibility to run applications written for different OS API’s on one or many OS’s
- Generates interface code that is specific to your application during code generation
- It is easily extended to support a new commercial OS or your in-house OS/in-house abstraction
- Comes with a Profiler for API and application profiling
Frequently Asked Questions about Cross-OS Development Platform and OS Abstractor
Q: What is OS Abstractor and Cross OS Development Platform
A: Cross-OS Development Platform™ is a C/C++ source-level virtualization technology that provides a robust and industry standard OS interface architecture for flexible real-time application development, while allowing theuser to protect the software from being locked to one OS. OS ABSTRACTOR is a component of Cross-OS Development Platform™ along with POSIX and uITRON Interface that provides user to develop portable applications using Industry standard generic API's
Q: What value does Cross-OS Development Platform provide?
A: By using Cross-OS Development Platform, your embedded application can run on many real time (RTOS) and non-real time operating systems to negate any porting issues in the future when your platform changes. OS Abstractor Interface from Mapusoft provides you a robust and standard OS interface architecture for flexible application development while eliminating the risks associated with selecting an OS and dependency on a single vendor. OS Abstractor Interface makes your application adapt to multiple operating system platforms with a standard OS interface, thereby reducing cost associated with code maintenance and learning multiple operating systems. POSIX Interface enhances the OS Abstractor Interface architecture with the addition of optimized non-proprietary and industry standard POSIX Application Programming Interface (API) to facilitate using open source POSIX/Linux in your design.
Q: Can I start developing my application with the OS Abstractor APIs without a RTOS?
A: Yes, the OS Abstractor APIs are available in AppCOE to enable the development of applications on a host x86 platform.
Q:How does the OS changer abstraction layer works? Is it VM? Does HW support needed?
A: The implementation is not dependent on any hardware. The code is 100% written in ‘C’. Our abstraction is based on source-level virtualization. OS Abstractor features compete with the native implementation in many situations to offer better performance.
Q: Why not use an in-house abstraction?
A: Developing in-house OS abstraction requires considerable
- Time, resources and money
- Planning to support multiple OS
- Detailed knowledge of low level OS functions and interfaces
- Up front cost associated with purchase of various OS and tools in order to validate your product
Q: Why develop portable code?
A: Developing portable code will Protect software investment by enabling efficient software re-use across multiple platforms Eliminate manual updates to applications when upgrading to newer versions of OS Allow you to easily switch your OS platform for more cost effective development
Q: Why use a standardized OS interface?
A: Using a standardized OS interface architecture will
- Reduce the learning curve associated with adopting a new OS by using intuitive, flexible and standard APIs across multiple operating systems
- Protect software investment by enabling efficient software re-use across multiple platforms
- Eliminate manual updates to applications when upgrading to newer versions of OS
Q: Why support multiple operating systems?
A: Developing software to run on multiple operating systems will
- Expand your market share and opportunities
- Provide flexibility to your customer to use your software on their preferred OS platform
- Protect your software investment from OS platform changes in the future
- Allow you to easily switch your OS platform for more cost effective development
Q: Why should I leverage open source code in my platform?
A: Leveraging open source code will give you the following benefits:
- Efficiently add feature rich services in a cost effective manner regardless of the underlying OS
- Get to market faster with compelling open-source applications and content in your design
- Tap into the large talent pool of available engineers with POSIX/Linux experience
- POSIX Interface enhances the OS Abstractor OS Interface architecture with the addition of optimized non-proprietary and industry standard POSIX Application Programming Interface (API) to facilitate using open source POSIX/Linux in your design.
Q: Why should I use OS Abstractor with my proprietary OS?
A: Using OS Abstractor with your proprietary OS will give you the following benefits:
- Leverage re-usable open source POSIX/Linux code to efficiently add feature rich services in a cost effective to your proprietary OS
- Make your application more compelling and valuable to your end users and customers
- Make your proprietary OS more adaptable for your customers
Q: Are the Cross-OS Development Platform APIs CPU dependent?
A: Cross-OS Development Platform APIs are not CPU dependent, since there is not any target specific code and OS Abstractor is 100% C code. We have customers using Cross-OS Development Platform APIs on many different CPU platforms. While, there should be no problems across platforms, MapuSoft does offer validation services if requested.
Q: Which POSIX Standard is your product conformant to?
A: We are not conformant in terms of supporting required POSIX functionalities in POSIX profile 51, 52, 53, and 54. For some of the POSIX features (such as, run-time libraries, math libraries, and so on) we are dependent on the underlying OS, but we plan to provide all of this directly from MapuSoft in the future. For example, OS Abstractor APIs for VxWorks is pretty close to profile 53. OS Abstractor APIs for LynxOS would be close to profile 54. However, the listed APIs in the POSIX Interface API list comes from MapuSoft libraries.
Q: How do you support Process Model?
A:We support process models for all the supported target OS (around 25 of them). If the target OS offers hardware protection & true process, then we will use that model, when it does not, we offer software process model where each process is independent with software protection features.
Q: Could you give one example of target OS which gives hardware protection & true process and which does not provide?
A:] Linux (also VxWorks 6.x and higher) provides hardware protection and full process. ThreadX (also MQX, etc.) does NOT provide full hardware protection and process support.
Q: Could you give any key difference between software process model and hardware assisted model as supported by OS abstraction layer? Please also give details of any key difference vs Linux process model
A:] The software process model uses private memory area that is dedicated to each process for local memory. It creates all resources (eg timers) by default private (however the scope can be changed to “system” if the resource needs to be shared across process) and provides the handle . The memory is divided in case if it is single kernel memory OS like ThreadX, otherwise true process feature provided by the OS is used. As far as the Linux process and OS Abstractor, they are identical. I suggest you read the system configuration manual documentation for further information (this system manual is available within AppCOE (help menu->help_contents-> AppCOE). You can download the software from our website via the download link. We also offer ability for processes to specifically create resources with public scope (which means another process will be able to use it, but cannot delete it, e.g. timers)
Q: How do you support Memory Management - VM, Shared Library, demand paging etc? If yes, is it "arm-linux-eabi" complaint?
A:] Mapusoft support ansi memory, dynamic pool memory, partitioned memory, tiered pool memory (dynamic memory build using layers of partitions like a wedding cake) and then shared tiered pool memory and also general shared memory area. Memory is fully managed by us, however we take advantage of the MMU feature offered by OS if available. Application basically gets configured for so much memory usage, which we guaranty and also ensure application will not go beyond its allocated memory and start impacting other applications. We do not use EABI features.
Q: Could you give more details of support levels for "Shared Library" & "demand paging" in the OS Abstraction layer?
A:] Under NDA we can provide the reference manuals for our shared memory support. You can use the native Linux shared memory ‘as is’ in addition to what OS Abstractor offers. We do not do any demand paging. Also, the virtual address differences in shared memory access across process will also not be a problem when using OS Abstractor.
Q: How do you support Linux Extensions like - SysV IPC/cgroups/inotiy/futext/readahead/udev etc? Q: Is it possible for Mapusoft to check feasibility to implement these features at the OS Abstraction layer?
A:] Currently our OS Abstractor runs on over 25 different target OS and their versions. We need to know the additional levels of support and also on what needed target OS to be supported then we can work this under a NRE and support this.
A:] Our level of support is posix 1003.1, 1003.1b and 1003.1c . In addition, the POSIX that we provide will automatically be able to take advantage of many of the real-time features from the specific OS Abstractor component for a target OS. OS Abstractor module provide the universal OS interfaces and OS features for many OS.
Q: Is it possible for Mapusoft to check feasibility to implement these features at the OS Abstraction layer? A:] Currently our OS Abstractor runs on over 25 different target OS and their versions. We need to know the additional levels of support and also on what needed target OS to be supported then we can work this under a NRE and support this.
Q: Do you support extensions to include TCP/IP stack which may not be provided as part of underlying RTOS?
A:] Yes. But for selected OS, we offer BSD compliance implemented using the net stack from the appropriate vendors offering. For example, we offer BSD complaint interface for Nucleus NET stack.
Q: Do you support necessary WiFI, BT kernel Interfaces, so that native OS Connectivity frameworks can be re used? Ex: BlueZ, Connman etc.
A:] As far as the device drivers itself, we do not offer our own across every OS. What we offer is a unix device driver I/O system framework to connect to many target OS so that the application code can be portable.
Q: How do you schedule process in multi-process application? Does this depend on target OS?
A:] We do not schedule process. All OS Abstractor processes are created equal and each thread within a process competes directly with the threads from the other. Also, we do not have a concept of parent/child/grandchild, etc. when it comes to processes. All the processes stand in a single level. To answer your second question, it does not depend on target OS. Note that uITRON and T-Kernel scheduler is First Come First Serve and preemptive. I am not sure of other OS scheduling policies. Most of them must be roundrobin. We provide support for all the scheduling (refer to OS_Create_Task API) as defined when you create a task using OS Abstractor API. If an underlying target OS does not support, we find a way to support the required scheduling.
Q: Are there possible software modes where function or performance would be anomalous? If so, how does the software enter such modes?
A:] OS Abstractor does not enter into any anomalous modes (though the target OS may do so, which is beyond OS Abstractor’s control). OS Abstractor returns failure codes if a function fails. For serious problems (e.g. exceed max configured resources) it will generate an error (in this case, based on user configuration, OS Abstractor will ignore or stop on error and can also print the error message string).
Q: How would you classify the OS Abstractor version of Linux? For example, do these versions have significant code reuse when compared to previous versions? Or, can they be considered child or rehosted versions of a mature product?
A:] OS Abstractor releases have significant code re-use when we add support to new OS. Once the product is validated for a specific OS (not specific OS versions), the code base is fully separated for each target OS. Bug fixes and enhancements that are generic will be carried out to all code bases for each supported target OS and goes through independent validation. Keeping the code base separate provides us the ability implement further target OS specific optimizations and also ensures that the new code changes do not impact solutions supported on the other OS platforms. We also have an automated test framework containing over 100 test suites that exercise OS Abstractor usage across various target operating systems and versions. Adding a new OS support is pretty much a cookie-cutter type of operation for us. We start off with a stable and suitable code base and make the target specific optimizations and then perform the validation. This effort is normally anywhere from 2 to 4 weeks for us on a stable new OS platform.
Q: Do you have safety data that you can provide us with? Examples include but are not limited to: Traceability matrix, Code review records, Functional Testing Results, Stress Testing Results, Stability Testing Results, Condition/Decision Coverage
A:] Yes. Please let us know what information that you would specifically need. Here are some examples of what we usually do:
- Test logs: We run the test suite and capture the log results under specific target environment. Release is made only when all the functional & regression level tests have passed repeatedly for a period of time.
- Test cases: We are constantly adding new test suites as new test cases arise.
- Test Configuration: Test can be run light (few resources) or heavy (more resources and large data sizes). Can also configure test loop count, duration between test runs, print results in summary or verbose, run only selected tests or all tests, etc.
Q: I generated source code for Windows XP as the target in AppCOE, compiled, and executed the program in VC++. Are the resultant tasks scheduled by the Windows scheduler?
A: OS Abstractor does not do scheduling on any port. We use the OS scheduler. However, we configure the threads to be real-time priority (e.g. on Windows we use the threads priorities that are normally not used by MFC app).
Q: If I select uITRON as a target OS, who schedules the task? Do the AppCOE libraries call uITRON dispatcher?
A: uITRON scheduler will schedule the thread.
Q: What is the default maximum stack size for QNX in the main thread?
A: The stack size defined under OS Abstractor for QNX is as follows: #define OS_MIN_STACK_SIZE 16384 #define OS_MAX_STACK_SIZE 0xffffffff The above defines are in cross_os_def.h. It is recommended that you always use the default OS_MIN_STACK_SIZE which is defined across various target OS. For example, under Linux, this is directly mapped to pthread_stack_min define.
Q: If three same priority processes, A, B and C are created, which process starts execution first? When will it switch to another process?
A: OS Abstractor processes do not compete with each other, but only the threads within them. The process will be started First In First Out (FIFO).
Q: If a task is created with priority 100, and it is changed to priority 50, then will the created task start executing? Is it not possible to set higher priority than the parent process?
A: You can set any priority when creating tasks. If you want it to run immediately, you pass along the flag parameter START, it will resume, irrespective of the priority.
Q: There are three processes, A, B and C, created of the same priority and are in ready state. If process A create new task D, with a priority higher than other three processes, then will Process D start first? If priority D is lower, then will process A keep executing? If priority D is equal to other processes, will the application execution be in this series: A -> B -> C -> D -> A,,,, ?
A: OS Abstractor Processes do not compete.
Q: OS Abstractor document says that we can use target OS native IO functions if I set MAP_OS_ANSI_IO to OS_FALSE. Does this mean that I can also call other target OS APIs from application program?
A: You should be able to use any native APIs directly from the target OS (as long as the API names are not the same).
Q: If target OS is uITRON, can I call set_tim (set system time API)?
A: You should be able to set_tim native uITRON API, however, that line of code will not be portable. It is recommended to use the clock routines provided by either the OS Abstractor APIs or the POSIX APIs.
Q: How about sig_sem (signal semaphore)?
A: You should be able to use sig_sem native API. But it is recommended to use the signal APIs from OS Abstractor APIs to ensure that your code be portable.
Q: If target OS is uITRON and it has Windows compatible file system and TCP/IP which support BSD Socket, then can the developer use open, close, read, write functions for file related operation?
A: The developer has the options to use OS_open, OS_close, etc (OR) they can set MAP_ANSI to TRUE and have open(), close() calls map to OS Abstractor APIs. The developer also has the option to use both APIs OS_open and open() by setting the MAP_ANSI to false.
Q: If target OS is uITRON and it has Windows compatible file system and TCP/IP which support BSD Socket, can the developer use socket functions such as socket, accept, listen., etc by calling native OS system call?
A: OS Abstractor API’s does not provide socket APIs, so the target OS needs to supply them. We have developed a BSD compliance wrapper for many netstacks on a custom solution basis if needed.
Q: What do I do if I get OS_ERR_RESOURCE error code when creating the OS Abstractor resources?
A: This error code indicates that the application is exceeding the max-allowed resources (see section Resource Configuration) and there are no free resource control blocks. To prevent this error from occurring, adjust max-allowed resource configuration for the appropriate resource, which is causing the error.
Q: What do I do if I get OS_ERR_NOT_OWNER error code when using any OS Abstractor resources?
A: This error code indicates that the resource was created with a Process scope and only the process, which created this resource, can use it. If you need the resource to be shared across processes, then use OS_SCOPE_SYSTEM option during creation of system resources.
Q: I see some functions inside the OS Abstractor library that starts with “INT_”. Can we also use these functions in our applications?
A: The functions starting with “INT_” are functions internal to the OS Abstractor, In order to keep the application portable, application should not use those functions (use only the functions that are exposed in the OS Abstractor manual).
Q: Is all access to a memory pool serialized among all threads (tasks)? Does each thread lock the memory pool when allocating or deallocating from that memory pool?
A: The memory allocation is FIFO based across all OS. There are two types of memory allocation to be discussed: Fixed Memory Allocation: OS Abstractor does NOT synchronize the alloc/de-alloc request. Neither does it searches memory to find a free block. Even in other types of resource creation/deletion we do NOT sequential. That being said, we have proprietary mechanism in place to ensure that everything will work concurrently under a multi thread/process environment with better performance. Dynamic or Variable Memory Allocation: OS Abstractor normally depends on the OS to do the allocation and de-allocation. It is expected that the OS may lock a thread momentarily during the period acquiring a free block, but this should not be viewed as sequential operation (it's only for data protection under multithreaded environment). Under Linux, in order to utilize the MMU defrag feature, OS Abstractor does allocation of memory only during the allocation time and not the pool creation time.
Q: What happens if a memory pool is deleted while there are still open handles to that memory pool? Or what happens to the memory behind the pointers that are still held?
A: All the resources including memory allocation gets automatically freed up when the process gets deleted. Also, all memory gets freed up when the pool goes away. If applications still use the memory pointers then we will have unpredictable results (seg faults are typical). Since OS Abstractor actually tracks the amount of memory allocated and available, it is easy to check its control block for memory leaks in application prior to actually deleting these pools.
Q: Does the application have direct access to OS Abstractor control blocks?
A: Applications will not have direct access to the control blocks if the option OS_INCLUDE_PROCESS is set to true. However, the application could corrupt the control block if the underlying OS/hardware does not have support for MMU.
Q: Does OS Abstractor dynamically allocate memory at run-time?
Answer: OS Abstractor library allocates memory for all the control blocks, application memory pools and such either statically or during initialization. This is done to ensure that OS Abstractor provides a stable environment for the application. The run time allocations that occur are only to cater to applications needs for queue buffers, stacks, pools and such). But this will be automatically come from the specific process pool, unless you are creating shared memory partitions and such. Applications can also create additional memory fixed/dynamic memory pools locally using the appropriate OS_Create APIs to bypass the use of process memory pool as an added flexibility. There may be few dynamic allocations that may take place (please refer to section on Internal run-time memory allocations by OS Abstractor and Interface)
Q: Can I set up a memory pool at a specific physical address and share it across multiple process and processors?
A: Contact MapuSoft for appropriate advice for your target environment.
Q: Why do the POSIX function names in OS Abstractor starts with “PX_” first?
A: This is intentionally done so that there would be no conflict at link time with the POSIX functions provided by the OS itself and the ones by the OS Abstractor. Please refer to the POSIX Interface section in the reference manual for more information.
Q: What are the important design goals of OS Abstractor?
A: Portability and Performance are key factors, which go hand in hand. Next is the API simplicity and development flexibility. The APIs are designed in such a way that they can be extended easily with new features and still preserve backward compatibility of the API interface. Each OS Abstractor API is developed as an independent building block (such as lego blocks) so that any complex functionality could be built using them.
Q: Can OS Abstractor and Interface libraries be built as shared libraries? What about as DLLs?
A: OS Abstractor and Interface libraries can be built as shared libraries. This would be the recommended option if you develop multi-process applications. As far as DLL, the current release does not support this feature. However, MapuSoft can provide custom help for your specific target environments.
Q: Does OS Abstractor support DLL calls?
A: Yes, under certain target OS that normally does not provide DLL support. DLL support is not packaged with the OS Abstractor release. If you need specific DLL support for your target environment, please contact MapuSoft.
Q: How can I use the process feature within a single or multiple executable images?
A: If it is a single image, then you can use OS_Create_Process() function to create new processes and this way you can break down your single application to run as many individual processes. In case of multiple images, OS Abstractor automatically creates a process for every application main ().
Q: How can I pass parameters (such as void *, Char *, etc.) to task functions?
A: OS Abstractor task accepts a parameter type UNSIGNED. If you need to pass any pointer, you can assign the pointer value to a variable of type UNSIGNED and pass that to the task routine. You will then retrieve the value within the task function and initialize the value to a local pointer of the appropriate type and then can use the pointer.
Q: How is memory defragmentation handled in the OS Abstractor product on Linux and VxWorks?
A: OS Abstractor always prefers to use the OS defrag methods for pools created using OS_Create_Memory_Pool API. Under VxWorks, the pool management and defrag is handled by the OS itself. Under Linux, we have two options: Option 1: This is a preferred option if the pool is big and varied size allocation. OS Abstractor does partial memory management but relies on the MMU assisted defrag and block allocation mechanism offered by the OS. Memory is not pre-allocated during pool creation but instead allocated at the time of request. OS Abstractor ensures that the applications do not over allocate system memory beyond what they are allowed (via OS_Application_Init API init call) for when they are created. As such failure of a memory request is unlikely even though allocations are done at request time. The disadvantage is that, there may be a non-OS Abstractor based application running along with OS Abstractor based applications eating up all the memory suddenly, which then could impact the OS Abstractor applications. Option 2: This is an option offered by MapuSoft for a smaller size pool in lieu of the above. Here the memory allocation is based on first fit. Automatic and limited defrag happens during each time the memory is freed. With every memory free request, the block adjacent to the freed block is checked and if free, then it gets combined into one bigger block. This defrag mechanism is no way close to how things are done via Option 1 (this is a custom request from MapuSoft).
Q: Since I do not have a source code for OS Abstractor, it is difficult to understand your products. Is it possible to send me the full source code?
A: Yes, you can generate source code for all the Mapusoft products if you have the required License. For more information on our OS Abstractor manual and/or for an overview of our licensing please contact MapuSoft for more information by following this link: http://mapusoft.com/contact/
Q: Are the Cross-OS Development Platform APIs CPU dependent?
A: Cross-OS Development Platform APIs are not CPU dependent, since there is not any target specific code as it is 100% ‘C’ code. We have customers using Cross-OS Development Platform APIs on many different CPU platforms. While, there should be no problems across platforms, MapuSoft does offer validation services if requested. AppCOE also provides feature called App/Platform Profiler that allows the application to collect performance data. By default, this feature is disabled, however when it is enabled, there is small architecture specific code that retrieves CPU clock frequency. Right now, we only support x86 architectures for this feature and we will add support for other architectures in subsequent releases.
Q: Which latest Linux kernel version do you support?
A:] We basically support every version and distribution out there for Linux
Q: Linux comes with varity of scheduling policy. Do you support any of them? These may not be supported in underlying RTOS
A:] We support, priority, time slice, combination and then run-to-completion scheduling at the OS Abstractor level. In addition we offer support for various scheduling done by OS like VxWorks, POSIX, etc.. We do not require the underlying OS to support all these scheduling as we contral the scheduling (except for the save/restore task context).
Q: Do you provide any NAND flash based file systems?
A:] We provide ANSI interface to various file systems offered by the RTOS vendors
Q: What kind of security abstraction features are supported in the OS Abstraction layer ? Ex: SMACK, SELinux, DAC etc
A:] We do not have any encryption support. Access to objects contained in shared memory are restricted only when they are accessed using our APIs.