Written by Jeffrey “Jefro” Osier-Mixon, ELISA Project Ambassador and member of the TSC and Senior Principal Community Architect at Red Hat
This blog originally ran on the Red Hat website. For more content like this, click here.
The automotive computing world, like many other industries, is going through a transformation. Traditionally discrete computing systems are becoming more integrated, with workloads consolidated into systems that look remarkably more like edge systems than embedded devices. The ideas driving this shift come from open source, but will Linux be part of this future, given that the existing standards for functional safety do not currently accommodate Linux-based operating systems?
Red Hat safety expert Gabriele Paoloni will present our safety methodology to the Automotive Linux Summit as an update to the presentation given at the recent ELISA Workshop. This methodology outlines the path Red Hat is taking toward creating an in-vehicle OS that incorporates modern ideas around workload orchestration, secure process isolation, and consolidation of mixed-criticality workloads, field-updatable and continuously certified for functional safety. We recognize that this is a huge task, but we believe it to be possible by adhering to these development pillars:
Open methodology development within ELISA
Participation in the ISO 26262 update process
Open code development within the CentOS Automotive SIG
ELISA (Enabling Linux in Safety Applications) is a vital part of the Linux universe, and it provides a community where those of us who care deeply about functional safety can address the challenges of certification and create solutions to resolve those challenges. ELISA is the cornerstone open source community for functional safety, and automotive is a big focus as the industry is clamoring for transformation and advanced computing facilities within the car. Red Hat recognizes that value and has emerged as a leader in the community.
ISO 26262 was originally developed in a time when automotive computing was managed through Electronic Control Units (ECUs), black boxes that had a deterministic output when given specific inputs. The standard is notably lacking support for pre-existing complex systems, including Linux. Red Hat is a member of an ongoing effort to update ISO 26262, known as ISO-PAS 8926, which has been accepted as a new working item proposal to the ISO committee.
Finally, Red Hat continues its commitment to work transparently as well as upstream first by forming an automotive special interest group (SIG) within CentOS. This Automotive SIG meets twice a month to collaboratively discuss automotive issues, including safety, and to produce a reference automotive OS based on CentOS Stream. We hope you will join us on this journey.
You can view Gabriele Paoloni’s “Functional Safety certification methodology for Red Hat In Vehicle OS” video from Automotive Linux Summit below.
Written by Shuah Khan, Chair of the ELISA Project Technical Steering Committee, and Milan Lakhani, member of the ELISA Medical Devices Working Group
Key Points
Understanding system resources necessary to build and run a workload is important.
Linux tracing and strace can be used to discover the system resources in use by a workload.
Once we discover and understand the workload needs, we can focus on them to avoid regressions and evaluate safety.
OpenAPS is an open source Artificial Pancreas System designed to automatically adjust an insulin pump’s insulin delivery to keep Blood Glucose in a safe range at all times.
It is an open and transparent effort to make safe and effective basic Automatic Pancreas System technology widely available to anyone with compatible medical devices who is willing to build their own system.
Broadly speaking, the OpenAPS system can be thought of performing 3 main functions. Monitoring the environment and operational status of devices with as much data relevant to therapy as possible collected, predicting what should happen to glucose levels next, and enacting changes through issuing commands, emails and even phone calls.
ELISA Medical Devices WG team has set out to discover the Linux kernel subsystems used by OpenAPS. Understanding the kernel footprint necessary to run a workload helps us focus on the subsystem and modules that make up the footprint for safety. We set out to answer the following questions:
What happens when an OpenAPS workload runs on Linux?
What are the subsystems and modules that are in active use when OpenAPS is running?
What are the interactions between OpenAPS and the kernel when a user checks how much insulin is left in the insulin pump?
So how do we discover the Linux kernel subsystem in use? The Linux kernel has several features and tools that can help discover which modules and functions are being used by an application during run-time. Using these tools, we can gather the system state while the OpenAPS workload is running to determine which parts of the kernel are being used.
Let’s talk a bit about kernel states. The kernel system state can be viewed as a combination of static and dynamic features and modules. Let’s first define what static and dynamic system states are and then explore how we can visualize the static and dynamic system parts of the kernel.
Static System View comprises system calls, features, static modules and dynamic modules enabled in the kernel configuration. Supported system calls and Kernel features are architecture dependent. System call numbering is different on different architectures. We can get the supported system call information using the auditd package tool. ausyscall –dump prints out the supported system calls on a system. You can install the auditd package by running “sudo apt-get install auditd” on Debian systems. Linux kernel script scripts/checksyscalls.sh can be used to check if current architecture is missing any function calls compared to i386. scripts/get_feat.pl <get_feat.pl list> can be used to list the Kernel feature support matrix for a system or get_feat.pl list –arch=arm for example lists the Kernel feature support matrix of the ‘arm’ architecture
Dynamic System View comprises system calls, ioctls invoked, and subsystems used during the runtime. A workload could load and unload modules and also change the dynamic system configuration to suit its needs.
One easier way to understand its runtime characteristics is watching the system state while a workload is running. We determined that the following methodology and tools would work well for us to observe the system activity.
What is the methodology?
The first step is gathering the default system state such as the dynamic and static modules loaded on the system. lsmod command prints out the dynamically loaded modules on a system. Statically configured modules can be found in the kernel configuration file. Understanding the default system is necessary to determine the changes if any made by the workload.
The next step isdiscovering the Linux kernel footprint used by OpenAPS by enabling event tracing before starting the workload, gathering dynamic modules while the workload is running. Once the workload is stopped, gather the event logs, kernel messages.
Once we have the necessary information, we can extract the system call numbers from the event trace log and map them to the supported system calls.
Putting our methodology to test
Our initial plan was to use strace to trace system calls and signals used by OpenAPS commands in strace. strace runs the specified command until it exits. It intercepts and records the system calls which are called by the process and the signals which are received by the process. It gives insight into the syscalls OpenAPS commands use. However, we realized quickly that OpenAPS employs setup scripts to launch its workload. As a result, using strace was not an option.
We modified OpenAPS oref0-setup.sh in https://github.com/OpenAPS/oref0.git to enable event tracing before OpenAPS starts its workload (processes, shell scripts). This approach gives us an overall information about OpenAPS. We will develop a higher level view and then dive into individual OpenAPS commands..
We were able to gather traces with the above modification to oref0-setup.sh. As mentioned earlier, the ausyscall tool dumps out mapping for syscalls and their corresponding syscall table numbers. The mapping is architecture dependent for some system calls. The trace data includes NR followed by a number in the trace file. These are the syscalls that are run during the time the tracing was on. Using the tracing and system call information, we determined the system calls used by the OpenAPS workload. In addition we used scripts/checksyscalls.sh to check for system call support status on RasPi.
What did we do to gather traces and system state?
Start OpenAPS workload with modified the modification to enable tracing
Let the workload run for 30 minutes
Discard last 5 minutes of trace from analysis to account for interference (rsync and plugging devices) with trace file extraction
Stop OpenAPS.
Extract trace file from the system – cat /sys/kernel/debug/tracing/trace > trace.out
Run lsmod after OpenAPS workload starts to gather module information
Run “ausyscall –dump > syscalls_dump.out”
Analyzing traces:
Map the NR (syscal) numbers from the trace to syscalls from the syscalls dump.
Categorize system calls and map them to Linux subsystems.
Findings and observations:
ioctls in use are all spidev ioctls. OpenAPS python modules use spidev interfaces.
Raspi SPI Dev and Python Spidev use (914 spidev ioctls invoked)
CONFIG_SPI, SPI_BCM2835 options enabled.
Heavy use of syscall32
Use scripts/checksyscalls.sh to check for support status
Kernel module usage:
Module Name
Usage count (OpenAPS)
Usage count (default)
cmac
Not loaded
1
ecdh_generic
1 (bluetooth)
2 (bluetooth)
ecc
1 (ecdh_generic)
1 (ecdh_generic)
spidev
2
Not loaded
i2c_bcm2835
1
Not loaded
spi_bcm2835
0
Not loaded
i2c_dev
2
Not loaded
ipv6
26
24
Subsystem usage:
Subsystem
# of calls
kmem_*
200990
mm_*
182471
sched_*
195241
rcu_*
223011
irq_*
1781503
kmalloc
79801
cpu_idle
62623
rss_stat
22130
ipi_*
42514
sys_*
148034
vm_*
60489
task_*
72813
timer_
58572
hrtimer_*
152271
softirq_*
28860
workqueue_*
6129
writeback_*
3933
ext4_*
34461
jbd2_*
2062
block_*
3590
dwc_otg*
27701
arch_timer
13446
Updated System view:
Conclusion
This tracing activity was a good way of identifying which parts of the kernel are used by OpenAPS. This helped to generate the Updated system view, so it is useful for our goal to do an STPA analysis of OpenAPS Operating System activity. We plan to theoretically analyse how these different subsystems can interact unsafely, while using fault injection and mocked components to collect more traces.
As mentioned earlier, the approach we used so far gave us a higher level of information about the OpenAPS usage. It isn’t ideal that this higher information doesn’t tell us the system usage by individual OpenAPS commands. As an example, we won’t be able to clearly identify which system calls are invoked when a user queries insulin pump status.
We are in the process of gathering fine grained information about individual OpenAPS commands and important use-cases. As an example, what subsystems are used when a user queries the insulin pump status. We are using the strace commandto trace theOpenAPS commands. We will share our findings in our next blog on this topic.
SPDX-License-Identifier: CC-BY-4.0
This document is released under the Creative Commons Attribution 4.0 International License, available at https://creativecommons.org/licenses/by/4.0/legalcode. Pursuant to Section 5 of the license, please note that the following disclaimers apply (capitalized terms have the meanings set forth in the license). To the extent possible, the Licensor offers the Licensed Material as-is and as-available, and makes no representations or warranties of any kind concerning the Licensed Material, whether express, implied, statutory, or other. This includes, without limitation, warranties of title, merchantability, fitness for a particular purpose, non-infringement, absence of latent or other defects, accuracy, or the presence or absence of errors, whether or not known or discoverable. Where disclaimers of warranties are not allowed in full or in part, this disclaimer may not apply to You.
To the extent possible, in no event will the Licensor be liable to You on any legal theory (including, without limitation, negligence) or otherwise for any direct, special, indirect, incidental, consequential, punitive, exemplary, or other losses, costs, expenses, or damages arising out of this Public License or use of the Licensed Material, even if the Licensor has been advised of the possibility of such losses, costs, expenses, or damages. Where a limitation of liability is not allowed in full or in part, this limitation may not apply to You.
The disclaimer of warranties and limitation of liability provided above shall be interpreted in a manner that, to the extent possible, most closely approximates an absolute disclaimer and waiver of all liability.
Written by Kate Stewart and Jason Smith, ELISA Project Medical Devices Working Group Members
The ELISA Project has several working groups with different focuses including Automotive, Linux Features for Safety-Critical Systems, Medical Devices, Open Source Engineering Process, Safety Architecture and Tool Investigation and Code Improvement. The Medical Devices Working Group consists of experts in Linux, medical, and functional safety applications that work together on activities and deliverables intended to help the safe development of medical devices that include Linux-based software. These activities include, but are not limited to, white papers describing best practices and safety requirements for medical devices using operating systems such as Linux, and conducting safety analyses of open source medical device projects that use Linux such as OpenAPS.
The safety of medical devices is very important, and can be influenced to a great extent by any software that is contained in the medical device. Failure of software in a medical device can unfortunately cause harm to persons or worse, as demonstrated in the incidents involving the Therac-25 several decades ago. Therefore, if a medical device is using an operating system such as Linux, the performance and safety of Linux then comes under scrutiny.
In the context of medical device safety standards such as IEC 62304, when Linux is incorporated into a medical device, it is considered to be something called Software of Unknown Provenance (SOUP). In this case, the medical device manufacturer incorporating Linux into their device did not develop Linux and therefore does not fully know what level of quality processes were used to develop Linux in the first place. Standards like IEC 62304 allow the usage of SOUP such as Linux; however, IEC 62304 requires that risks associated with the failure of SOUP have been considered and addressed by the manufacturer.
The Medical Devices Working Group is in the process of developing a white paper summarizing requirements from IEC 62304 pertaining to SOUP to assist medical device manufacturers. If you have experience in Linux, medical, or functional safety applications, the Medical Devices Working Group welcomes your input on this white paper.
One of the interesting challenges with medical devices is that often most of the source for the system is restricted, and not openly available. This presents a challenge when trying to do analysis on how Linux is being used in such systems.
The OpenAPS project is a hobbyist project to create a feedback system between an insulin pump and glucose monitor to aid the Type 1 diabetes users to build systems to help manage their blood glucose levels. That the project is open source means that we can see the code and have a starting point for analysis.
The Medical Devices Working Group has been using System Theoretic Process Analysis (STPA) to analyze the system, which they call a “rig”, and the Linux system interactions within it. A rig consists of the Raspberry PI (running Linux and algorithms), glucose monitor (commercial) , insulin pump (commercial), and some data logging device. How to set up a rig and use it is documented by the OpenAPS project, which has significantly aided our analysis.
At this point, we’ve applied the STPA analysis through a couple of levels and have iterated on the analysis a few times (STPA process helped us identify some factors we’d not considered in diagraming the system initially). The team is now working on collecting traces of the system interacting with the Linux kernel. Tracing will let us continue to take the STPA analysis into the kernel subsystems.
We are interested in learning of other open source projects using Linux in the context of a medical device. If you know of such a project, or are interested in working with our team of volunteers, please feel free to reach out at medical-devices@lists.elisa.tech.
Happy New Year! We hope that everyone in the ELISA ecosystem and community had a wonderful and safe holiday season. As we take a look at the blank slate of the new year, the Linux Foundation’s Shuah Khan, ELISA Project Chair of the Technical Steering Committee, shares a few of the predictions of what the project will achieve in 2022. She chats with Swapnil Bhartiya, The Fourth Industrial Revolution Creator and Host, in this video. Watch it or read the predictions below.
Swapnil Bhartiya:
Hi, this is your host, Swapnil Bhartiya. And welcome back to TFiR’s predictions for 2022. And today we have with us, once again, Shuah Khan, Linux fellow and chair of the ELISA Project Technical Steering Committee. Shuah, it’s great to have you on the show.
Shuah Khan:
Thank you for having me.
Swapnil Bhartiya:
Before we ask you to grab your crystal ball and share your predictions, I want to know a little bit about the project. Tell us what is the project all about?
Shuah Khan:
ELISA project is all about enabling Linux and safety critical applications. What that means is that at ELISA project, what we are doing is we are bringing safety experts and Linux experts together to collaborate on developing best practices and resources for people that are enabling Linux in their products, on their safety critical applications.
Swapnil Bhartiya:
Now, if I ask you, please grab the crystal ball and tell me what predictions you have for 2022.
Shuah Khan:
My first prediction is ELISA community will keep growing. We’ll continue to add new members and will continue to engage with the kernel and safety communities.
My second prediction is we will expand our critical spaces. Right now, we focus on several like medical and automotive. We will expand into other industries by adding members from aviation and industrial spaces.
Swapnil Bhartiya:
Thanks for sharing these two predictions. Now, if I ask you what is going to be the focus of the project in 2022?
Shuah Khan:
The focus for us in 2020 do is continuing to work with Automotive Grade Linux (AGL) and Autostar in the automotive space and continue engaging the kernel and safety communities. Our second focus is to continue harmonizing best practices for our members. We want to be able to make best practices, processes, and resources available to our members that are enabling Linux on safety clinical applications.
Swapnil Bhartiya:
Shuah, thank you so much for sharing these predictions and also, the focus for the project for 2022. And as usual, I would love to have you back on the show. Thank you.
ELISA Ambassadors are technical leaders who are passionate about the mission of the ELISA Project, recognized for their expertise in functional safety and linux kernel development, and willing to help others to learn about the community and how to contribute.
Each month, we’ll put a spotlight on an ELISA Ambassador. Today, we’re excited to highlight Elana Copperman, PhD, ELISA ambassador, Chair of the Linux Features for Safety-Critical Systems (LFSCS) Working Group and TSC voting member. Elana works as a System Safety Architect in Mobileye, an Intel company, designing features to support safety as well as security features in Mobileye vision products for the automotive domain.
Background Details:
As a System Safety Architect at Mobileye, Elana provides support for designing critical system features in Mobileye products, including system boot; drivers; and Linux infrastructure. Before working at Mobileye, she worked as a Security Architect for Cisco-Il (formerly NDS) and more recently as a security consultant for major European automotive concerns on behalf of various Israeli startups. Most recently, merging with safety constraints in the automotive domain, to deploy secure as well as safe systems. Her research interests focus on software engineering methodologies and security engineering.
Q&A
How long have you been active in open source?
I have been involved in “open source” before it was formalized as we know it today. During my time as an undergraduate student, online software source code was commonly shared as freeware. In fact, even early Unix versions were provided at no cost to academics, and collaborative efforts were supported to some degree.
My PhD research focused on Object-Oriented Programming, including some investigations on Java source code and features. Open source software development, Linux in particular, has evolved since then, including many new challenges and opportunities. For example, setting up and working with Apache Server over Linux OS in the early 1990’s.
Over the last 15 years, Linux has grown to what we know & love today, with its amazing powers. As a system architect, I have been designing systems that empower Linux in embedded systems, first in Set-Top Boxes (for digital broadcasting) and currently in Automotive.
Tell us about your favorite open source project and what problems did it aim to solve?
My favorite open source project focused on security code review with ST (chip vendor) engineers for the optee code, which had been released to Linaro. To see first-hand how open source aligns vendor, kernel and user requirements with features to resolve complex security challenges was mind-blowing.
How long have you been active in the ELISA Project?
What roles and/or working groups do you have or participate in?
I was the Chair of the Kernel Development Process Working Group, and am currently the new Linux Features for Safety-Critical Systems WG. I am learning from the safety experts on board, and my primary contribution is to represent the viewpoint of the application designers and developers who will build Linux-based safety critical systems.
Where do you see the ELISA Project in three years?
My dream for ELISA is to see a community of developers providing kernel modules which may be leveraged for use in safety critical systems. For example, a suite of memory protection mechanisms derived from well defined requirements that may be deployed to protect safety-critical data. Another example, which is in our planned agenda, are test suites for concurrency issues such as deadlock and race conditions, focusing on test plans derived from safety requirements.
What is the biggest strength of the ELISA community?
I think ELISA has matured since its early days and we are currently more structured in our analysis and goals. Our strength is in current efforts to progress and making real contributions to the Linux kernel community.
What is the best piece of advice you have ever received?
Never give up. Be flexible, adapt, adjust – but keep up the hard work.
What technology can you not live without? Why?
My home coffee machine. Drop in for a fresh cup and you will know why.
What part of the world do you live in? Why do you love where you live?
I live in Israel, a tiny multicultural nation at the junction of 3 continents (Europe, Asia, Africa). It has also become a major highway for birds – and a pilgrimage site for bird watchers. A wide variety of species can be viewed making their way south in the fall and back north in the spring. Thomas Krumenacker (German journalist and photographer) wrote a book on “Birds in the Holy Land”.
Now is a great time of year to take a trip to the Hula Valley Nature Reserve, to get a view of some of the estimated 500 million migrating birds passing over the Hula Valley and stopping by for a drink of water. This exemplifies the amazing diversity of the people and wildlife of our small fast-paced country.
What’s your favorite quote?
3 Fun facts:
My first computer was a PCJr, with 64 Kb of memory, and with virtually no hardware or software. I had to install all my own hardware add-ons and get it to work, including etching bit maps for the Hebrew alphabet in memory and getting it to work from right to left. After adding more and more components to the motherboard, one day it blew up. But I had a lot of fun with it until that happened.
As a grad student at NYU, we had loads of fun playing with early internet protocols and collaboration tools, including gopher and kermit; getting our hands dirty (“finger”) working with each other’s systems; and Mosaic (an early Web browser). The crazy things we did over the internet taught me a lot about potential security issues related to networking. This expertise was the foundation for my more recent work in security and safety engineering.
As chair of the Computer Department at the Jerusalem College, we aimed high but with a low budget. Trying to keep up to date with educational (i.e., no support or documentation) versions of closed source software was a real challenge. I became adept at finding contacts working with major software vendors who were willing to break down the walls, expose the source code and support my customization for our needs. In a sense, challenging proprietary software in open-source development mode …
To learn more about ELISA ambassadors, please click here.
Written by Philipp Ahmann, ELISA Project Ambassadorand TSC member
The 8th ELISA Workshop, which took place on November 8-10, had 158 registrants that learned more about the various working groups and networked with ambassadors. Participants were active and stable in the technical discussions, which helped plan for next year, and proved that there is continuous interest and motivations in these topics. Additionally, new ideas were shared and can be considered as sources of inspiration and for critical thinking with e.g. talks about STPA, the z-model or open source and the community problem along with new approaches to safety and importance of processes and testing.
Overall, ELISA workshops have more of a conference character presenting major achievements and results of achievements from the last quarter. Interested newcomers as well as community participants receive a very good status update in which direction ELISA moves forward on its way to enable Linux in safety applications.
On the other hand, this brilliant forum to sync up and align in a virtual format, misses a bit of the in-person discussion and brainstorming that made ELISA’s first workshops so enjoyable. With a community spread across the world longer working sessions are seldom and almost not possible. Prime time slots are used to align to reach as many people as possible.
As this year’s workshops come to a close, we hope that travelling and hybrid workshops become possible again. In this type of format, the core community can take joint working sessions, while talks and presentations can still attract newcomers and other interested to get an overview about what is going on in the community and what to target next. We hope to see you in-person next year if possible.
In case you missed the November virtual workshop, let us take a look at a few selected sessions from the conference.
Day 1 started with the newcomer and welcome session. It provided an overview about what Linux is and how safety comes into picture to form the name ELISA. The statistics shown during the talk on the usage of Linux in industry are quite impressive and speak for themselves. Linux is everywhere. (Or almost everywhere.)
How important such Linux features and development processes are could be slightly depicted from the presentation about Linux in Safety Systems explained by Christopher Temple from arm. He illustrates quite well which different levels of complexity a system could have. He also reminded the audience that there can be a difference between simply following a safety integrity standard for the necessary processes and actually creating a safe product. It is interesting to see that you may be able to show standard conformance without having safety properly implemented and creating a safe system which is not certifiable or in conformance with a safety standard.
On the second day the slides about Certification Using the New Approach to Safety presented by Paul Albertella showed that modern safety certification and assessments need to bring a good tool support towards tool support and automation along with proper tool classification and qualification as a major step to achieve efficient certification and generation of evidence, irrespective if you are running proprietary or open source software.
In another session Lukas Bulwahn from Elektrobit helped the participants on understanding the z-model in which the challenge on pre-existing software is illustrated, where a logical model would be to write requirements, go over to integration and verification to derive software architectural design, leading to software unit verification to finally create a “z” by moving to the originally existing Software unit design and implementation. This can help to approach existing stacks which do not fall into strict recommendations such as demanded by e.g. ISO26262. This said, it has to be kept in mind that the “z” fills the gap and can create a complete “v” in the end, although you may run into issues during an ASPICE assessment.
Day two was concluded by Shaun Mooney from Codethink, who gave insights on how to use STPA for ISO26262. STPA has been used within the medical devices work group since a long time and recently found its way also into the automotive work group, where it serves as an alternative to a traditional HARA. A very important element of the STPA, beside others, is the identification of unsafe control actions (UCA) to unveil potential harms/hazards and risks in a very structured and visual way.
Day three was a nice mixture of technical insights into the Linux kernel and new approaches and directions towards safety qualification of Linux application. The strong demand to consider both Linux software development within the community and the strict regulations by safety integrity standards to come to a certifiable product, were brought to the point by Lukas Bulwahn’s talk. It was thought to encourage critical thinking on safety integrity standards and the community problem. Let us hope that the work of ELISA can make a difference and the effort we take will direct in the solution of this problem, eventually even with updated or new safety integrity standards, which include state of the art software development process and quality measures, so much needed for complex systems.
As for the previous workshops the last session wraps up and includes the goal settings for the next quarter along with the request to not let the discussions stop here…
If you reach this point of reading the blog post, you seem to be really interested in joining the ELISA community, so don’t miss to register to your mailing list of choice.
Short TL:DR summarizing words about the workshop:
Less registered people, but very stable number of attendees during the workshop and on level of last workshop
Good “take home” messages letting you think about the challenges of Linux and open source communities to approach safety integrity standards
New approaches in fields of architectural analysis, tools, development process, testing and engineering show the demands where Linux and open source need to go different ways and where safety integrity standards need to evolve to keep up with the complexity of software written by a large scale community.
ELISA community would benefit from a hybrid approach enabling in person working sessions to let the workshop be a workshop and have less conference style.
The ELISA community grows and reaches a point where harmonization is needed. Brainstorming times are over and everybody shares concepts and proposals how to achieve the goal to enable Linux in safety applications.
Videos and recordings of the workshop presentations can be found here.
Written by Dan Brown, Senior Manager, Content & Social Media, LF Training
As we approach a new year, this is the perfect time to consider what you want your career to look like in 2022. Job openings are at record highs, and this is especially true in the IT field, where the 2021 Open Source Jobs Report found that 92% of hiring managers are unable to find enough talent to meet their organizations’ needs. A primary mission of The Linux Foundation is helping close the talent gap so the industry has the talent necessary to carry out digital transformation activities and continue innovating, while also creating accessible pathways for anyone who wants to start an IT career to do so.
We are excited to once again offer our best pricing of the year on our entire catalog of training courses, certification exams, bundled programs, and bootcamps for Cyber Monday. From now through December 6, 2021, all these fantastic offerings covering hot topics like cloud computing, system administration, networking, blockchain, web development, embedded systems, and more are available at significantly reduced cost. As the home of some of the most important open source technologies like Kubernetes, Linux, Node.js, Hyperledger, and more, The Linux Foundation provides vendor-neutral training directly from the experts helping build these projects.
You can check out the full details of everything that is on offer on our Cyber Monday Landing Page. Begin your journey to a long-term, successful career in IT today!
Open Source Summitand Embedded Linux Conference, held in Seattle, Washington as well as virtually on September 27-30, gathered 1,944 total attendees. Approximately, 482 of those attended in person from 760 organizations across 68 countries around the globe. Learn more about the event in the post-event report here.
The ELISA Project was represented by Gabriele Paoloni, Chair of the ELISA Project Governing Board and Open Source Tech Lead (Functional Safety) at Red Hat, and Daniel Bristot de Oliveira, a member of the ELISA community and Principal Software Engineer at Red Hat. Gab and Daniel presented a talk about how to create a maintainable, scalable and verifiable SW qualification approach for automotive in linux. Watch the video below.
Over the last years, many discussions took place in Linux Foundation’s ELISA Working Groups about possible approaches to qualify Linux for safety-critical systems. To achieve this goal, an architectural description of the Linux kernel is required.
The challenge though is to find the adequate granularity for description: It must be precise enough to support safety analyses, but it cannot be too fine-grained to the point of being unmanageable. A promising approach is to leverage the ISO26262-6 and ISO26262-8 together, in a hierarchical incremental approach. Optimizing the amount of produced documentation and collaterals.
In this video, the foundations of this approach were presented. Gab and Daniel showcase why this approach is suitable for safety application as well as out-of-context using assuming safety requirements and why it provides natural scalability across different use-cases. Finally, considerations will be made with respect to available tools and mechanisms already implemented or proposed in Linux that can significantly help with the above-mentioned approach – including a detailed discussion about how to cross verify, and monitor, the documentation and the kernel using the Runtime Verification subsystem.
Written by Elana Copperman, ELISA project ambassador, Chair of the Linux Features for Safety-Critical Systems Working Group and System Safety Architect at Mobileye (Intel)
The Linux Features for Safety-Critical Systems (LFSCS) WG aims to feed into the OSEP and other WGs, working together as a team. LFSCS invites engineers, architects and integrators who actually develop and deploy Linux-based safety-critical systems to contribute from their practical experience and knowledge. In particular, to identify existing Linux kernel features that may be leveraged for use in safety-critical systems.
For example:
Mechanisms for protection of various memory types; e.g. protection from faults due to uninitialized variables or stack overflow.
Dynamic analysis for multi-threaded systems; e.g. tests based on tools such as TSAN or ASAN.
Kernel profiling using ebpf-based tools; e.g. perf-tools or bpftrace
AER (Advanced Error Reporting) for fault handling; e.g. PCIe fault handling
Safety extensions to Linux drivers; e.g. fault handling support and bridging the gap between hardware-based safety features and application layer fault handling.
The WG mailing list is open to registration here, and we are seeing an amazing group of contributors who can demonstrate use of such features in real systems, and help ELISA to learn from these experiences. Initially, we will investigate existing features but will also propose enhancements to such features and to work as a community to design / implement / deploy kernel patches. The goal of such patches will be to help make those features more amenable for use in safety critical systems. Our Github playground is here.
The alliance with ELISA, and with the new Open Source Engineering Process Working Group in particular, is a critical aspect of this effort. We will be working together to help ensure that those patches and features can be used by designers and integrators producing safety critical systems.
The scope of this WG does not include safety qualification or any safety claims on how the integrator can or should use these features or patches. The only claims that would be made are a description of the feature and its functional impact.
The WG will be formally kicked off at the upcoming ELISA workshop (November 8-10). We will be giving an overview of the working group and answer any questions on November 8 at 3 pm CET. We will be scheduling weekly meetings following the workshop. If the technical challenge of enabling real change in deploying open source Linux-based software for safety critical systems excites you, come join and help us meet the challenges!
You can still register for the Fall workshop, which is being held virtually and is free to attend. All registrants will be able to watch the sessions on-demand. Register here today!
Written by Paul Albertella, Chair of the ELISA Project Open Source Engineering Process Working Group
The ELISA Project’s new Open-Source Engineering Process (OSEP) Working Group focuses on the role of engineering processes in creating safety-related systems based on Linux and other FOSS.
Engineering processes are very important in safety, because we rely heavily on them to provide confidence in a system and its components. We achieve confidence by undertaking risk analysis to identify how harm may result from the use (or misuse) of the system, and then constructing a safety argument, which describes how these risks are managed.
When we apply this approach to a specific element of the system, such as a software component like Linux, the argument can be broken down into a number of claims that we want to make regarding that element and its role in the safety of our system. Some claims will relate to the functional responsibilities that the element has in the system; others will relate to the processes that we use to create and refine it.
Importantly, we also need to produce evidence to support these claims. Almost all of this evidence will be produced by an engineering process; some of it will be evidence relating to those processes themselves..
Safety standards like ISO 26262 and IEC 61508 describe reference processes that can act as a template for safety arguments like this. They identify the engineering practices that are seen be necessary (e.g. code review, verification through software testing), the formal processes that are used to control these (e.g. verification management), and the evidence needed to confirm that these have been applied (e.g. test plans, test results).
These reference processes are based on the V-model, which emphasises the formal specification of requirements, architecture and design, and the ability to trace formal verification processes back to these. For software, the standards focus on the processes used when developing new components for a safety-related system, although they include some guidance on applying the principles to pre-existing components, such as software libraries.
Open source projects like Linux have their own development processes, which may be sophisticated and make use of sound software engineering practices. However, it is difficult to map these directly to the reference processes described by the safety standards, because open source development models have very different goals and organizational models, which tend to emphasize refinement by rapid iteration, peer review and community contribution.
In order to address this, OSEP aims to identify and evaluate practices, processes and tools that FOSS developers, system integrators and product creators can use to bridge this gap. We plan to accomplish this by:
Selecting Linux topics and safety-related claims that we want to make about them
Identifying and evaluating practices, processes and tools to answer:
What risks are associated with the topic and claims?
To what extent are these risks addressed or mitigated by (or for) Linux?
How can we manage risks that are not sufficiently addressed or mitigated?
How can we show evidence to support our claims?
Collaborating with other WGs for technical investigations
Documenting and sharing our results as we go
If you would like to learn more about OSEP, join us for an overview presentation on November 8 at 3 pm CET at the ELISA Workshop. The Fall workshop, being held virtually on November 8-10, is free to attend and all registrants will be able to watch the sessions on-demand. Register here.
If you would like to contribute to OSEP, please join the mailing list here, where you can also find details of weekly meetings on the working group calendar.
