Category

Blog

ELISA Project Launches Call for Papers for November 8-10 Workshop

By Announcement, Blog, Workshop

Since launching in 2019, the ELISA Project has continued to grow in membership, community contributions and working groups. The project’s more than 20 member companies, which include ADIT, AISIN AW CO., arm, Automotive Grade Linux, BMW Car IT GmbH, Codethink, Elektrobit, Horizon Robotics, Huawei Technologies, Intel, Kuka, Linuxtronix, Mentor, NVIDIA, OTH Regensburg, Red Hat, Suzuki, Toyota and Wind River, collaborate to define and maintain a standardized set of processes and tools that can be integrated into Linux-based, safety-critical systems seeking safety certification.

Today, the ELISA Project is excited to announce that its next technical workshop will take place virtually on November 8-10. The event is free and open to developers, users and contributors of ELISA from around the globe looking to learn, network and collaborate. 

The Call for Papers is now open and accepting submissions that will tackle technical strategies for development and deployment as well as real-world applications and use cases. Submit a speaking proposal by Friday, October 1 here: https://events.linuxfoundation.org/elisa-workshop/program/cfp/

The last workshop took place in May with 239 participants from 37 different countries. It featured sessions that showcased working group milestones, open discussions about projects and use cases in automotive and medical. Additionally, this workshop involved more collaboration with adjacent communities, such as Xen, Real Time Linux and AUTOSAR. For a complete recap of the workshop, click here

The ELISA Workshops are hosted once a quarter and are focused on education and outreach for new community members, the exchange of ideas and feedback from the linux kernel and safety communities, as well as productive collaboration to make tangible progress toward achieving the mission and goals of the ELISA Project.

Registration for the event is also open. You can register here

Linux in Basic Safety Applications

By Blog, Workshop

In May, the ELISA Project hosted its 7th Workshop with 239 participants from 37 different countries. For a complete recap of the workshop, click here. Today, we’ll take a look at one of the sessions led by Jason Smith, Principal Engineer at UL LLC, about the “Linux in Basic Safety Applications.”

Linux is more often being used in applications with safety relevance:

  • Complex safety-related functions necessitating the advantages of an OS, or
  • Complex end application necessitating the advantages of an OS, now being asked to perform one or more basic safety-related functions (for example: voltage, current, temperature monitoring)

In both cases, software used to implement the safety-related functions may be required to conform/comply with applicable functional safety standards.

In this presentation, Jason will discuss linux in basic safety applications, the goals and progress of the white paper the ELISA Project is working on and details about how to get involved.

Click here learn more about the ELISA Project, here for the Working Groups and here to join our mailing list. 

Open Source Software Safety Concept Tooling in Freeplane

By Blog, Workshop

In May, the ELISA Project hosted its 7th Workshop with 239 participants from 37 different countries. For a complete recap of the workshop, click here. Today, we’ll take a look at one of the sessions led by Jochen Kall, Expert Engineer Safety at ITK Engineering on behalf of Toyota about the “Open Source Software Safety Concept Tooling in Freeplane.”

The Automotive Working Group uses an open source software mindmap based tooling for developing and documenting Safety Concepts as well as managing the requirements therein. In this session, an introduction to the tool, its capabilities, and use cases is given, followed by a setup/tutorial session guiding the audience through installation and setup of the tool as well as a demonstration of how it can be used in safety engineering.

Freeplane is available on github ((https://github.com/Jochen-Kall/Safety_concept_tool) and helps Safety/Requirements Engineering tasks with support for:
– Avoiding duplication of repeated requirements, leveraging clones
– Managing artifact types, ASILs ,etc and their respective constraints
– Allocating to architectural elements
– Code tagging
– Safety Consistency checking
– Tainting/Changing Propagation in the tree
– Exporting / Importing [WIP]

Watch the video below and let us know if you have questions!

Click here learn more about the ELISA Project, here for the Working Groups and here to join our mailing list. 

Usability of ISO 26262 2nd Edition for an Open Source Design

By Blog, Workshop

In May, the ELISA Project hosted its 7th Workshop with 239 participants from 37 different countries. For a complete recap of the workshop, click here. Today, we’ll take a look at one of the sessions led by Roberto Paccapeli, Functional Safety Manager at Intel and Vito Magnanimo, Functional Safety Architect at BMW Group, about the “Usability of ISO 26262 2nd Edition for an Open Source Design.”

In the automotive domain, the reference standard for Functional Safety is ISO 26262. The normative does not currently provide a clear distinction between new Software design and pre-existing ones. This limitation directly impacts on open source designs, developed in accordance with non-standardized development process (e.g. Linux operations system). This video presents some of the gaps observed in the standard and introduces hints that can be jointly addressed with ELISA members without losing the cornerstone of the ISO (or in contrast with its clauses).

Click here learn more about the ELISA Project, here for the Working Groups and here to join our mailing list. 

Updates for ELISA’s Tooling Investigation and Code Improvement WorkGroup

By Blog, Workshop

The ELISA Project has several working groups each dedicated to a focus or use case. In today’s blog, we’ll take a look at the Tool Investigation and Code Improvement WorkGroup. The Tool Investigation and Code Improvement WG focuses on application of tools, handling the tool results, and improving the kernel based on the tools’ feedback.

Lukas Bulwhan, Safety Software Key Expert at Elektrobit GmbH, leads the Tool Investigation and Code Improvement WorkGroup and recently gave an update about their mission, achievements and roadmap at the last ELISA Project Workshop. You can watch the presentation below.

ELISA Project Workshop May 2021: Tooling Investigation and Code Improvement Working Group Update

If you have questions or would like to join the Working Group, they meet weekly on Tuesdays. Subscribe to the mail list here: https://lists.elisa.tech/g/tool-investigation.

Xen Project: How we do functional safety

By Blog, Workshop

In May, the ELISA Project hosted its 7th Workshop with 239 participants from 37 different countries. For a complete recap of the workshop, click here. Today, we’ll take a look at one of the sessions led by Artem Mygaiev, Director of Technology Solutions at EPAM Systems, Stefano Stabellini, Principal Engineer at Xilinx, about the Xen Project.

Tailored versions of Xen Hypervisor are used in mission-critical systems for years, but this was never the case for Xen’s mainline. Starting 2019, Special Interest Group in Xen Project works on identifying and eliminating obstacles on the way to safety-certify Xen. In this video, Artem and Stefano will talk about their approach, progress so far and collaboration with other groups within Linux Foundation.

Click here learn more about the ELISA Project, here for the Working Groups and here to join our mailing list. 

The Safety Architecture Working Group: Achievements & Plans

By Blog, Workshop

The ELISA Project has several working groups each dedicated to a focus or use case. In today’s blog, we’ll take a look at the Safety Architecture Working Group, which aim’s to determine critical Linux subsystems and components in supporting safety functions, define associated safety requirements and scalable architectural assumptions, deliver corresponding safety analyses for their individual qualification and their integration into the safety critical system.

Gabriele Paoloni, Governing Board Chair for the ELISA Project, leads the Safety Architecture Working Group and recently gave an update about their mission, achievements and roadmap at the last ELISA Project Workshop. You can watch the presentation below.

ELISA Project Workshop May 2021: Safety Architecture Working Group Update

If you have questions or would like to join the Safety Architecture Working Group, they meet weekly on Tuesdays from 8-9 am ET (2-3 pm CET). Subscribe to the mail list here: https://lists.elisa.tech/g/safety-architecture.

We invite you to get your hands dirty with the Automotive Working Group!

By Blog

Written by Philipp Ahmann, ELISA Project Ambassador and Manager at ADIT

Where it all started – The automotive WG 

The ELISA Project was launched two years ago by the Linux Foundation. We had our first workshop in person at the BMW training center (Munich, Germany) and the majority of participants with automotive focuses were screaming, “Enable Linux in safety application within the car!” But what happened then?

Since then, the following workshops as well as our weekly meetings, had a strong focus on automotive use cases. There were a lot of participants and a lot of interest but not a lot of volunteers to help with tasks. We kept receiving requests from Toyota, Suzuki, BMW and Automotive Grade Linux (AGL)… In response to this, the Automotive Working Group was established a little more than a year after the launch of the ELISA Project.

From the beginning, while looking for datasheets, reference designs, documentation, and technical concepts, the words “NDA” and “IP” are something we always have in our minds. As a result, we approached the work cautiously as a group:

  • Concentrated on what ISO26262 showcased about functional safety;
  • Focused our work with a simulation that is open for everybody;
  • Stopped saying “could and should” and started using practical examples; and 
  • Pause lengthy discussions about problems that are not Linux specific.

Gaining momentum – The telltale use case

Following these principles, the Automotive Working Group started making progress.  We got a good mixture of safety expertise, Linux know-how and automotive backgrounds. We also frequently talk about new things with the curiosity and questioning mindset of a child, which has helped us create a healthy learning environment that is engaging and productive. 

Due to Suzuki’s and AGL’s introduced use case, we decided to concentrate on the enablement of telltales (often referred to also as tell-tale) based on a Linux instrument cluster. Thanks to AGL a demo and some high-level ideas were already available. 

As we continued our momentum as a group, we recognized that we were spreading our key learnings around in different formats – a bit of source code in a git, diagrams in PlantUML, PowerPoint, or other tools. Documentation was spread over presentations and google docs, so it was hard to create materials and engage interested participants outside the working group. We were determined to continue our momentum and began leveraging tools that would enable others to reproduce and understand our work.

Public means public – The tools

Functional safety projects typically have a very limited set of tools used in the development flow, which have run through a tool qualification. This is expensive because of the license fees and proprietary tools. Putting everything in plain text is good version control and a good baseline, which is key. But monolithic documents make it hard to maintain relationships and traceability – you may even find yourself lost in long text passages. 

To make documentation reviews easier and put them under proper version control, we changed from initial sketches in google docs to documentation in GitHub. While also taking requirements in GitHub, we saw they are hard to maintain, put in the relationship and maintain traceability. So the transition was done to maintain them in Freeplane with a plugin developed by Jochen Kall, who is the Automotive WG lead. This plugin also includes e.g. an export script that renders requirements in markdown. Also, the ReqIF exporter is under preparation.

Similar to text, we also had architectural diagrams that the working group converted. We worked to take initial sketches in slide decks and presentations into a storable format. In this case, PlantUML was efficient and easy for us to use.

After this, we recognized that the use case designs end up in the same issue – no relationship between elements within the single PlantUML diagrams, so it was time to change the tool again. The OSS tool we use now is Papyrus based on Eclipse. The files are stored in XML format and in this way can also be put under proper version control. 

In the end, all of this hard work has led us to a steady set of tools:

  • Github for all source code and documentation;
  • Freeplane to maintain requirements (storable in version control and exportable to text also stored in version control); and 
  • Papyrus for Eclipse. 

We are aware that our tools currently used will not survive a safety assessment out of the box, but this is not our intention. The generated artifacts should be shareable so that they can be re-used by others in their established infrastructure. Also, we are targeting to enable others to build safe Linux-based systems and follow the development process for safety integrity standards accordingly. However, in the end, our telltale example will remain an example. A fully qualified product is out of the scope of the ELISA project.

What’s next

So, here we are. Out of creativity and storming team spirit, we settle and start to standardize the tools we use. Version control, review, traceability became major elements of our work. 

The practical demo provided by AGL was enhanced to serve the fundamental demands of the telltale use case with a watchdog and a safety app as a codebase. The build can be reproduced with the help of a docker image and the binary can run on qemu. 

We still have a long way to go but our goals for the next quarter are:

  • The source code analysis and interaction with the ELISA Architecture Working Group will be enhanced; 
  • The use case will be benchmarked against Autosar Adaptive safety requirements and its demands on the operating system; and 
  • Documentation needs to reach a draft state good enough to share with an external audience and to stand critical questions.
  • The existing Kernel config will be cleaned up towards a slim config (by throwing out unused things) and feedback on our changes to AGL

To learn more about the Automotive Working Group, please subscribe to the mailing list, join our weekly calls and become an active member. Never underestimate what you can achieve with a group. We are happy to welcome additional contributors – get ready to get your hands dirty and have fun with a passionate group of people. 

A Recap of the 7th ELISA Workshop

By Blog, Workshop

Written by Gabriele Paoloni, Chair of the ELISA Project Governing Board and Lead Software Architect at Intel, and Paul Albertella, Contributor and Member of the ELISA Project and Consultant at Codethink

The latest ELISA workshop, hosted virtually on May 18-20, was a great reflection of how fast the community has grown and evolved over the last few months. Participation was almost double the previous workshop in February with 239 participants from 37 different countries. Additionally, we’ve seen more collaboration with other groups such as AUTOSAR and AGL. The existing working groups have been exploring an extensive range of topics and initiatives, and there are plans to add new working groups to help take some of these forward.

A number of presentations focused on the challenges of qualifying or certifying Linux for functional safety, and the limitations of the established routes presented in standards such as IEC62304, IEC61508 and ISO 26262, and innovative approaches to addressing these. One proposed strategy included a more comprehensive look at a Linux Architectural design, and using test and tracing techniques to verify system behaviour against a derived model. Another proposal, focused on top-down hazard analysis to define safety requirements, statistical analysis of tests on historical kernel versions to show where Linux satisfies these, and fault injection techniques to validate the safety mechanisms of the wider system.

There were also talks on how some of these ideas are being applied in the working groups, focussing on collaborative efforts in the Automotive, Safety Architecture and Development Process groups based on the Telltale use case. Other interesting sessions focused on technologies with possible applications for functional safety, including an introduction to real time configurations for Linux, and the use of authorisation hooking in security modules. 

Discussions during these sessions made it clear that the community has a lot of new ideas to explore over the coming months and a lot of new participants eager to get involved. Work continues on the ELISA technical strategy, which will provide an important direction to this work, but there’s also a need to consolidate the innovative ideas and methodologies for qualifying Linux into the current working group activities, and evaluate the need for new working groups. As ELISA becomes more mature we need to define and refine the publication strategy for the outputs of working groups. There are also plans to develop ‘onboarding’ material for the project to help enable new participants to start contributing more quickly.

You can view the some of the presentation materials here when you click on each session. Some of the videos will be accessible too in the next few weeks.  

Tuesday, May 18

Shuah Khan, the Chair of the ELISA Project Technical Steering Committee, kicked off the workshop with an overview of the project, the working group activities and the recent whitepaper summarizing their interactions and deliverables.

As the different working group updates were presented, it became clear that there is a great deal of collaboration between each group:

  • The Automotive WG refined the safety concept following feedback from the Safety Architecture WG and is working with the Tools Subgroup to optimize the active Kernel image footprint;
  • The Safety Architecture WG is working with the Development Process WG on safety analyses and on a new hybrid qualification approach;
  • The Medical Device WG is coming to a point where they need to hand over the safety requirements to the Safety Architecture WG for deeper Kernel analyses; 
  • The Tools WG released a static code analysis framework that can be used along the qualification activities of the different WGs.

Additionally, Artem Mygaiev and Stefano Stabellini gave an introduction and update about the Functional Safety Special Interest Group (SIG) in the Xen project. This session was engaging as we shared feedback and ideas about functional safety from different perspectives. 

Wednesday, May 19

Philipp Ahmann introduced the engagement between the Automotive WG and the Autosar Adaptive consortium. We have many common interests and goals that should easily help us build a solid foundation for future collaboration. 

Then Roberto Paccapeli and Vito Magnanimo presented the current limitation of ISO26262 in qualifying a complex pre-existing SW component, like Linux, and the need for overcoming such limitations.


Gabriele Paoloni and Daniel Bristot de Oliveira presented an innovative approach (Hybrid Approach) that could be used as a scalable way to qualify Linux to be used in automotive safety critical applications; hence a proposal to overcome the above mentioned limitations.

Elana Copperman and Gabriele Paoloni presented the out of context analysis of the Linux Watchdog subsystem as a practical example of applying the Hybrid Approach, and how this is beneficial in the context of the Automotive WG’s Telltale use case.

Finally, Thomas Gleixner introduced the Linux Real-Time project, the challenges that they faced to meet timing constraints and all the different solutions they put in place to overcome them. It was a really nice tour of the project with lots of possible intercepts with functional safety systems.

Thursday, May 20

On the last day, Shuah Khan and Elana Copperman presented the work done to analyze Kernel configuration parameters (Kconfig) and their impact on Functional Safety, starting from some similar work done for Security (CWE).

Chris Temple then presented an overview of the possible SW qualification routes in

Functional Safety ranging from ISO26262 to IEC61508 reinforcing the current limitations of safety standards with respect to the qualification of complex SW components already discussed in the previous day.

Following this, Paul Sherwood and Paul Albertella presented yet another approach to overcome such limitations: an in-context approach based on a mix of safety analysis, testing of historical kernel versions and fault injection. This approach sparked a lot of interest and a need to further consider and discuss it across the different ELISA WGs was widely agreed.

STPA diagram from New Approach presentation

The final day closed with some wrap-up sessions discussing future activities to advertise ELISA and encourage new members to join, ELISA goals for the next quarter and a few stats about the current workshop. 

It was wonderful to get together virtually as a community. With more than 200 participants, we hope that attendees were engaged in our work and welcome their thoughts and participating in any of our technical meetings and working groups. Click here learn more about the ELISA Project, here for the Working Groups and here to join our mailing list. 

Interview with Shuah Khan, Kernel Maintainer & Linux Fellow

By Blog
Shuah Khan, Kernel Maintainer, Linux Fellow and Chair of the ELISA Project Technical Steering Committee

Jason Perlow, Director of Project Insights and Editorial Content at the Linux Foundation, had an opportunity to speak with Shuah Khan about her experiences as a woman in the technology industry. She discusses how mentorship can improve the overall diversity and makeup of open source projects, why software maintainers are important for the health of open source projects such as the Linux kernel, and how language inclusivity and codes of conduct can improve relationships and communication between software maintainers and individual contributors. This blog originally ran on the Linux Foundation website. For more content like this, click here.

JP: So, Shuah, I know you wear many different hats at the Linux Foundation. What do you call yourself around here these days?

SK: <laughs> Well, I primarily call myself a Kernel Maintainer & Linux Fellow. In addition to that, I focus on two areas that are important to the continued health and sustainability of the open source projects in the Linux ecosystem. The first one is bringing more women into the Kernel community, and additionally, I am leading the mentorship program efforts overall at the Linux Foundation. And in that role, in addition to the Linux Kernel Mentorship, we are looking at how the Linux Foundation mentorship program is working overall, how it is scaling. I make sure the LFX Mentorship platform scales and serves diverse mentees and mentors’ needs in this role. 

The LF mentorships program includes several projects in the Linux kernel, LFN, HyperLedger, Open MainFrame, OpenHPC, and other technologies. The Linux Foundation’s Mentorship Programs are designed to help developers with the necessary skills–many of whom are first-time open source contributors–experiment, learn, and contribute effectively to open source communities. 

The mentorship program has been successful in its mission to train new developers and make these talented pools of prospective employees trained by experts to employers. Several graduated mentees have found jobs. New developers have improved the quality and security of various open source projects, including the Linux kernel. Several Linux kernel bugs were fixed, a new subsystem mentor was added, and a new driver maintainer is now part of the Linux kernel community. My sincere thanks to all our mentors for volunteering to share their expertise.

JP: How long have you been working on the Kernel?

SK: Since 2010, or 2011, I got involved in the Android Mainlining project. My first patch removed the Android pmem driver.

JP: Wow! Is there any particular subsystem that you specialize in?

SK: I am a self described generalist. I maintain the kernel self-test subsystem, the USB over IP driverusbip tool, and the cpupower tool. I contributed to the media subsystem working on Media Controller Device Allocator API to resolve shared device resource management problems across device drivers from different subsystems.

JP: Hey, I’ve actually used the USB over IP driver when I worked at Microsoft on Azure. And also, when I’ve used AWS and Google Compute. 

SK: It’s a small niche driver used in cloud computing. Docker and other containers use that driver heavily. That’s how they provide remote access to USB devices on the server to export devices to be imported by other systems for use.

JP: I initially used it for IoT kinds of stuff in the embedded systems space. Were you the original lead developer on it, or was it one of those things you fell into because nobody else was maintaining it?

SK: Well, twofold. I was looking at USB over IP because I like that technology. it just so happened the driver was brought from the staging tree into the Mainline kernel, I volunteered at the time to maintain it. Over the last few years, we discovered some security issues with it, because it handles a lot of userspace data, so I had a lot of fun fixing all of those. <laugh>.

JP: What drew you into the Linux operating system, and what drew you into the kernel development community in the first place?

SK: Well, I have been doing kernel development for a very long time. I worked on the LynxOS RTOS, a while back, and then HP/UX, when I was working at HP, after which I transitioned into  doing open source development — the OpenHPI project, to support HP’s rack server hardware, and that allowed me to work much more closely with Linux on the back end. And at some point, I decided I wanted to work with the kernel and become part of the Linux kernel community. I started as an independent contributor.

JP: Maybe it just displays my own ignorance, but you are the first female, hardcore Linux kernel developer I have ever met. I mean, I had met female core OS developers before — such as when I was at Microsoft and IBM — but not for Linux. Why do you suppose we lack women and diversity in general when participating in open source and the technology industry overall?

SK: So I’ll answer this question from my perspective, from what I have seen and experienced, over the years. You are right; you probably don’t come across that many hardcore women Kernel developers. I’ve been working professionally in this industry since the early 1990s, and on every project I have been involved with, I am usually the only woman sitting at the table. Some of it, I think, is culture and society. There are some roles that we are told are acceptable to women — even me, when I was thinking about going into engineering as a profession. Some of it has to do with where we are guided, as a natural path. 

There’s a natural resistance to choosing certain professions that you have to overcome first within yourself and externally. This process is different for everybody based on their personality and their origin story. And once you go through the hurdle of getting your engineering degree and figuring out which industry you want to work in, there is a level of establishing credibility in those work environments you have to endure and persevere. Sometimes when I would walk into a room, I felt like people were looking at me and thinking, “why is she here?” You aren’t accepted right away, and you have to overcome that as well. You have to go in there and say, “I am here because I want to be here, and therefore, I belong here.” You have to have that mindset. Society sends you signals that “this profession is not for me” — and you have to be aware of that and resist it. I consider myself an engineer that happens to be a woman as opposed to a woman engineer.

JP: Are you from India, originally?

SK: Yes.

JP: It’s funny; my wife really likes this Netflix show about matchmaking in India. Are you familiar with it?

SK: <laughs> Yes I enjoyed the series, and A Suitable Girl documentary film that follows three women as they navigate making decisions about their careers and family obligations.

JP: For many Americans, this is our first introduction to what home life is like for Indian people. But many of the women featured on this show are professionals, such as doctors, lawyers, and engineers. And they are very ambitious, but of course, the family tries to set them up in a marriage to find a husband for them that is compatible. As a result, you get to learn about the traditional values and roles they still want women to play there — while at the same time, many women are coming out of higher learning institutions in that country that are seeking technical careers. 

SK: India is a very fascinatingly complex place. But generally speaking, in a global sense, having an environment at home where your parents tell you that you may choose any profession you want to choose is very encouraging. I was extremely fortunate to have parents like that. They never said to me that there was a role or a mold that I needed to fit into. They have always told me, “do what you want to do.” Which is different; I don’t find that even here, in the US. Having that support system, beginning in the home to tell you, “you are open to whatever profession you want to choose,” is essential. That’s where a lot of the change has to come from. 

JP: Women in technical and STEM professions are becoming much more prominent in other countries, such as China, Japan, and Korea. For some reason, in the US, I tend to see more women enter the medical profession than hard technology — and it might be a level of effort and perceived reward thing. You can spend eight years becoming a medical doctor or eight years becoming a scientist or an engineer, and it can be equally difficult, but the compensation at the end may not be the same. It’s expensive to get an education, and it takes a long time and hard work, regardless of the professional discipline.

SK: I have also heard that women also like to enter professions where they can make a difference in the world — a human touch, if you will. So that may translate to them choosing careers where they can make a larger impact on people — and they may view careers in technology as not having those same attributes. Maybe when we think about attracting women to technology fields, we might have to promote technology aspects that make a difference. That may be changing now, such as the LF Public Health (LFPH) project we kicked off last year. And with LF AI & Data Foundation, we are also making a difference in people’s lives, such as detecting earthquakes or analyzing climate change. If we were to promote projects such as these, we might draw more women in.

JP: So clearly, one of the areas of technology where you can make a difference is in open source, as the LF is hosting some very high-concept and existential types of projects such as LF Energy, for example — I had no idea what was involved in it and what its goals were until I spoke to Shuli Goodman in-depth about it. With the mentorship program, I assume we need this to attract fresh talent — because as folks like us get older and retire, and they exit the field, we need new people to replace them. So I assume mentorship, for the Linux Foundation, is an investment in our own technologies, correct?

SK: Correct. Bringing in new developers into the fold is the primary purpose, of course — and at the same time, I view the LF as taking on mentorship provides that neutral, level playing field across the industry for all open source projects. Secondly, we offer a self-service platform, LFX Mentorship, where anyone can come in and start their project. So when the COVID-19 pandemic began, we expanded this program to help displaced people — students, et cetera, and less visible projects. Not all projects typically get as much funding or attention as others do — such as a Kubernetes or  Linux kernel — among the COVID mentorship program projects we are funding. I am particularly proud of supporting a climate change-related project, Using Machine Learning to Predict Deforestation.

The self-service approach allows us to fund and add new developers to projects where they are needed. The LF mentorships are remote work opportunities that are accessible to developers around the globe. We see people sign up for mentorship projects from places we haven’t seen before, such as Africa, and so on, thus creating a level playing field. 

The other thing that we are trying to increase focus on is how do you get maintainers? Getting new developers is a starting point, but how do we get them to continue working on the projects they are mentored on? As you said, someday, you and I and others working on these things are going to retire, maybe five or ten years from now. This is a harder problem to solve than training and adding new developers to the project itself.

JP: And that is core to our software supply chain security mission. It’s one thing to have this new, flashy project, and then all these developers say, “oh wow, this is cool, I want to join that,” but then, you have to have a certain number of people maintaining it for it to have long-term viability. As we learned in our FOSS study with Harvard, there are components in the Linux operating system that are like this. Perhaps even modules within the kernel itself, I assume that maybe you might have only one or two people actively maintaining it for many years. And what happens if that person dies or can no longer work? What happens to that code? And if someone isn’t familiar with that code, it might become abandoned. That’s a serious problem in open source right now, isn’t it?

SK: Right. We have seen that with SSH and other security-critical areas. What if you don’t have the bandwidth to fix it? Or the money to fix it? I ended up volunteering to maintain a tool for a similar reason when the maintainer could no longer contribute regularly. It is true; we have many drivers where maintainer bandwidth is an issue in the kernel. So the question is, how do we grow that talent pool?

JP: Do we need a job board or something? We need X number of maintainers. So should we say, “Hey, we know you want to join the kernel project as a contributor, and we have other people working on this thing, but we really need your help working on something else, and if you do a good job, we know tons of companies willing to hire developers just like you?” 

SK: With the kernel, we are talking about organic growth; it is just like any other open source project. It’s not a traditional hire and talent placement scenario. Organically they have to have credibility, and they have to acquire it through experience and relationships with people on those projects. We just talked about it at the previous Linux Plumbers Conference, we do have areas where we really need maintainers, and the MAINTAINERS file does show areas where they need help. 

To answer your question, it’s not one of those things where we can seek people to fill that role, like LinkedIn or one of the other job sites. It has to be an organic fulfillment of that role, so the mentorship program is essential in creating those relationships. It is the double-edged sword of open source; it is both the strength and weakness. People need to have an interest in becoming a maintainer and also a commitment to being one, long term.

JP: So, what do you see as the future of your mentorship and diversity efforts at the Linux Foundation? What are you particularly excited about that is forthcoming that you are working on?

SK: I view the Linux Foundation mentoring as a three-pronged approach to provide unstructured webinars, training courses, and structured mentoring programs. All of these efforts combine to advance a diverse, healthy, and vibrant open source community. So over the past several months, we have been morphing our speed mentorship style format into an expanded webinar format — the LF Live Mentorship series. This will have the function of growing our next level of expertise. As a complement to our traditional mentorship programs, these are webinars and courses that are an hour and a half long that we hold a few times a month that tackle specific technical areas in software development. So it might cover how to write great commit logs, for example, for your patches to be accepted, or how to find bugs in C code. Commit logs are one of those things that are important to code maintenance, so promoting good documentation is a beneficial thing. Webinars provide a way for experts short on time to share their knowledge with a few hours of time commitment and offer a self-paced learning opportunity to new developers.

Additionally, I have started the Linux Kernel Mentorship forum for developers and their mentors to connect and interact with others participating in the Linux Kernel Mentorship program and graduated mentees to mentor new developers. We kicked off Linux Kernel mentorship Spring 2021 and are planning for Summer and Fall.

A big challenge is we are short on mentors to be able to scale the structured program. Solving the problem requires help from LF member companies and others to encourage their employees to mentor, “it takes a village,” they say.

JP: So this webinar series and the expanded mentorship program will help developers cultivate both hard and soft skills, then.

SK: Correct. The thing about doing webinars is that if we are talking about this from a diversity perspective, they might not have time for a full-length mentorship, typically like a three-month or six-month commitment. This might help them expand their resources for self-study. When we ask for developers’ feedback about what else they need to learn new skill sets, we hear that they don’t have resources, don’t have time to do self-study, and learn to become open source developers and software maintainers. This webinar series covers general open source software topics such as the Linux kernel and legal issues. It could also cover topics specific to other LF projects such as CNCF, Hyperledger, LF Networking, etc.

JP: Anything else we should know about the mentorship program in 2021?

SK: In my view,  attracting diversity and new people is two-fold. One of the things we are working on is inclusive language. Now, we’re not talking about curbing harsh words, although that is a component of what we are looking at. The English you and I use in North America isn’t the same English used elsewhere. As an example, when we use North American-centric terms in our email communications, such as when a maintainer is communicating on a list with people from South Korea, something like “where the rubber meets the road” may not make sense to them at all. So we have to be aware of that.

JP: I know that you are serving on the Linux kernel Code of Conduct Committee and actively developing the handbook. When I first joined the Linux Foundation, I learned what the Community Managers do and our governance model. I didn’t realize that we even needed to have codes of conduct for open source projects. I have been covering open source for 25 years, but I come out of the corporate world, such as IBM and Microsoft. Codes of Conduct are typically things that the Human Resources officer shows you during your initial onboarding, as part of reviewing your employee manual. You are expected to follow those rules as a condition of employment. 

So why do we need Codes of Conduct in an open source project? Is it because these are people who are coming from all sorts of different backgrounds, companies, and ways of life, and may not have interacted in this form of organized and distributed project before? Or is it about personalities, people interacting with each other over long distance, and email, which creates situations that may arise due to that separation?

SK: Yes, I come out of the corporate world as well, and of course, we had to practice those codes of conduct in that setting. But conduct situations arise that you have to deal with in the corporate world. There are always interpersonal scenarios that can be difficult or challenging to work with — the corporate world isn’t better than the open source world in that respect. It is just that all of that happens behind a closed setting.

But there is no accountability in the open source world because everyone participates out of their own free will. So on a small, traditional closed project, inside the corporate world, where you might have 20 people involved, you might get one or two people that could be difficult to work with. The same thing happens and is multiplied many times in the open source community, where you have hundreds of thousands of developers working across many different open source projects. 

The biggest problem with these types of projects when you encounter situations such as this is dealing with participation in public forums. In the corporate world, this can be addressed in private. But on a public mailing list, if you are being put down or talked down to, it can be extremely humiliating. 

These interactions are not always extreme cases; they could be simple as a maintainer or a lead developer providing negative feedback — so how do you give it? It has to be done constructively. And that is true for all of us.

JP: Anything else?

SK: In addition to bringing our learnings and applying this to the kernel project, I am also doing this on the ELISA project, where I chair the Technical Steering Committee, where I am bridging communication between experts from the kernel and the safety communities. To make sure we can use the kernel the best ways in safety-critical applications, in the automotive and medical industry, and so on. Many lessons can be learned in terms of connecting the dots, defining clearly what is essential to make Linux run effectively in these environments, in terms of dependability. How can we think more proactively instead of being engaged in fire-fighting in terms of security or kernel bugs? As a result of this, I am also working on any necessary kernel changes needed to support these safety-critical usage scenarios.

JP: Before we go, what are you passionate about besides all this software stuff? If you have any free time left, what else do you enjoy doing?

SK: I read a lot. COVID quarantine has given me plenty of opportunities to read. I like to go hiking, snowshoeing, and other outdoor activities. Living in Colorado gives me ample opportunities to be in nature. I also like backpacking — while I wasn’t able to do it last year because of COVID — I like to take backpacking trips with my son. I also love to go to conferences and travel, so I am looking forward to doing that again as soon as we are able.

Talking about backpacking reminded me of the two-day, 22-mile backpacking trip during the summer of 2019 with my son. You can see me in the picture above at the end of the road, carrying a bearbox, sleeping bag, and hammock. It was worth injuring my foot and hurting in places I didn’t even know I had.

JP: Awesome. I enjoyed talking to you today. So happy I finally got to meet you virtually.